Python example: possible speedup?

[Hrvoje Niksic]

Doug Hellmann <doughellmann at home.com> writes:

> > Thanks for the suggestion.  However, I do install new packages
> > often, so that's not really an option.
> 
> Probably not every time you run your app, though.

I use the application to see which packages waste too much space, so
that I know which ones to remove.  Not to mention that I run nightly
updates of my Debian system.  It is practically *guaranteed* that the
files will have changed between two runs of the program.

Sorry.  I like the idea of pickling/marshalling, and I'd love to be
able to use it here.  But it can't work that way.

> You could parse the data, pickle the results, and keep up with the
> mod times on the two files.  If the source data is newer than the
> pickle database you have to update the pickle data.

The problem is that to "update" the pickle data (which I'd have to do
every time, as explained above), I have to parse the text files.  And
it's the parsing that's slow, not the post-processing.  If I need to
parse them every time, then I don't need pickling.