Python example: possible speedup?

[Doug Hellmann]

Hrvoje Niksic wrote:
> 
> Timothy R Evans <tre17 at pc142.cosc.canterbury.ac.nz> writes:
> 
> > I had a very similar problem, I have this University project to
> > visualise the Debian package file, so I needed to read in into
> > useful language (C was discard as fast but too much work).  The read
> > tiem was really slow, so I only do it once.  After loading the file
> > and creating a data structure that stores the information, I pickle
> > it.  Read time drops by an order of magnitude.  Not so applicable to
> > your situation possibly but you could use it if you don't install
> > new packages often.
> 
> Thanks for the suggestion.  However, I do install new packages often,
> so that's not really an option.

Probably not every time you run your app, though.  You could parse the
data, pickle the results, and keep up with the mod times on the two
files.  If the source data is newer than the pickle database you have to
update the pickle data.  That would slow down some of your processing,
but in most cases would still let you take advantage of the faster
pickle version.

Doug