[Python-Dev] Unpickling memory usage problem, and a proposed solution

[Antoine Pitrou]

Dan Gindikin <dgindikin <at> gmail.com> writes:
> 
> Antoine Pitrou <solipsis <at> pitrou.net> writes:
> > Does cPickle bytecode have some kind of NOP instruction?
> > You could keep track of which PUTs weren't necessary and zero them out at the
> > end. It would be much cheaper than writing a whole other "optimized" stream.
> 
> For a large file, I'm not sure it is much faster to edit it in place
> than to rewrite it, and also since it's a large file, you are going
> to probably have it compressed, in which case you are out of luck anyway.

Depends whether you really care about disk occupation or not. Disk space is
often cheap (much more so than RAM).
Also, I'm quite sure overwriting a couple of blocks in a file is cheaper than
rewriting it entirely. Of course, if you must overwrite every other block, it
might not be true anymore :-)

Regards

Antoine.