How to make this unpickling/sorting demo faster?

Steve Bergman sbergman27 at gmail.com
Thu Apr 17 18:33:49 EDT 2008


Thanks for the help.  Yes, I see now that gdbm is really superfluous.
Not having used pickel before, I started from the example in "Python
in a Nutshell" which uses anydbm.  Using a regular file saves some
time.  By far, the biggest improvement has come from using marshall
rather than cPickel. Especially for writing. gc.disable() shaved off
another 20%.  I can't use psyco because I'm on x86_64. And my goal is
to demonstrate how Python can combine simplicity, readability, *and*
speed when the standard library is leveraged properly.  So, in this
case, calling C or assembler code would defeat the purpose, as would
partitioning into smaller arrays.  And my processor is single core.

I started at about 7 seconds to just read in and sort, and at 20
seconds to read, sort, and write.  I can now read in, sort, and write
back out in almost exactly 5 seconds, thanks to marshal and your
suggestions.

I'll look into struct and ctypes.  Although I know *of* them, I'm not
very familiar *with* them.

Thank you, again, for your help.




More information about the Python-list mailing list