is cPickle really this slow?

Dag dag at animagicnet.no
Mon Sep 30 05:53:21 EDT 2002


I have a script which starts by parsing a large file and turning into
a data stucture (a dictonary of lists) which I then use in the script.  
This operation however takes about 12-15 seconds.  Since the file hardly 
ever changes , and if it changes it's because I changed it, thought I'd 
save some time by pickeling the datastructure and and simply getting that 
at the start of each script.  So I pickle my dictionary, which ends up
being a 3.2MB file and try to run my script again, this time getting 
the pickled dictonary instead of building it at runtime.  This time
however the script takes over 35 seconds to run, close to three times
as long.  Am I doing something wrong is loading and parsing larger
pickled files really very slow.

Dag



More information about the Python-list mailing list