[Python-Dev] Unpickling memory usage problem, and a proposed solution

[Alexandre Vassalotti]

On Fri, Apr 23, 2010 at 2:38 PM, Alexandre Vassalotti
<alexandre at peadrop.com> wrote:
> Collin Winter wrote a simple optimization pass for cPickle in Unladen
> Swallow [1]. The code reads through the stream and remove all the
> unnecessary PUTs in-place.
>

I just noticed the code removes *all* PUT opcodes, regardless if they
are needed or not. So, this code can only be used if there's no GET in
the stream (which is unlikely for a large stream). I believe Collin
made this trade-off for performance reasons. However, it wouldn't be
hard to make the current code to work like pickletools.optimize().

-- Alexandre