Is there any way to minimize str()/unicode() objects memory usage [Python 2.6.4] ?

dmtr dchichkov at gmail.com
Sat Aug 7 04:45:45 EDT 2010


I guess with the actual dataset I'll be able to improve the memory
usage a bit, with BioPython::trie. That would probably be enough
optimization to continue working with some comfort. On this test code
BioPython::trie gives a bit of improvement in terms of memory. Not
much though...

>>> d = dict()
>>> for i in xrange(0, 1000000): d[unicode(i).encode('utf-8')] = array.array('i', (i, i+1, i+2, i+3, i+4, i+5, i+6))

1000000 keys, ['VmPeak:\t  125656 kB', 'VmSize:\t  125656 kB'],
3.525858 seconds, 283618.896034 keys per second


>>> from Bio import trie
>>> d = trie.trie()
>>> for i in xrange(0, 1000000): d[unicode(i).encode('utf-8')] = array.array('i', (i, i+1, i+2, i+3, i+4, i+5, i+6))

1000000 keys, ['VmPeak:\t  108932 kB', 'VmSize:\t  108932 kB'],
4.142797 seconds, 241382.814950 keys per second



More information about the Python-list mailing list