how to build a dict including a large number of data

wanzathe wanzathe at gmail.com
Fri Jan 4 08:57:50 EST 2008


hi everyone
i'm a newbie to python :)
i have a binary file named test.dat including 9600000 records.
the record format is int a + int b + int c + int d
i want to build a dict like this: key=int a,int b  values=int c,int d
i choose using bsddb and it takes about 140 seconds to build the dict.
what can i do if i want to make my program run faster?
or is there another way i can choose?
Thanks in advance.

My Code:
-----------------------------------------------------------------------------------
my_file = file('test.dat','rb')
content = my_file.read()
record_number = len(content) / 16

db  = bsddb.btopen('test.dat.db','n',cachesize=500000000)
for i in range(0,record_number):
    a = struct.unpack("IIII",content[i*16:i*16+16])
    db['%d_%d' % (a[0],a[1])] = '%d_%d' % (a[2],a[3])

db.close()
my_file.close()



More information about the Python-list mailing list