How to remove subset from a file efficiently?
fynali
iladijas at gmail.com
Sat Jan 14 03:51:15 EST 2006
$ cat cleanup_use_psyco_and_list_compr.py
#!/usr/bin/python
import psyco
psyco.full()
postpaid_file = open('/home/sajid/python/wip/stc/2/PSP0000333')
outfile = open('/home/sajid/python/wip/stc/2/PSP-CBR.dat.psyco',
'w')
barred = {}
for number in open('/home/sajid/python/wip/stc/2/CBR0000333'):
barred[number] = None # just add it as a key
outfile.writelines([number for number in postpaid_file if number
not in barred])
postpaid_file.close(); outfile.close()
--
$ time ./cleanup_use_psyco_and_list_compr.py
real 0m39.638s
user 0m5.532s
sys 0m0.868s
This was run on my machine (w/ Python 2.4.1), can't install psyco on
the actual server at the moment.
I guess using generators & newer Python is indeed faster|better.
--
fynali
More information about the Python-list
mailing list