[issue14775] Slow unpickling of certain dictionaries in python 2.7 vs python 2.6

[stw]

stw <silktw at googlemail.com> added the comment:

Thanks for the nod to the pickletools module. The structure of the streams do change between python 2.6 and 2.7, at least for files created with cPickle. You can see this from the file sizes that I quoted in my previous post. Below are some snippets of the disassemblies of pickle files generated using make_file.py:

pickle, python 2.6 and 2.7:

    0: \x80 PROTO      2
    2: }    EMPTY_DICT
    3: q    BINPUT     0
    5: (    MARK
    6: U        SHORT_BINSTRING 'ArLLX'
   13: q        BINPUT     1
   15: \x85     TUPLE1
   16: q        BINPUT     2
   18: K        BININT1    0
   ...

cPickle, python 2.6:

    0: \x80 PROTO      2
    2: }    EMPTY_DICT
    3: q    BINPUT     1
    5: (    MARK
    6: U        SHORT_BINSTRING 'XYoGB'
   13: \x85     TUPLE1
   14: q        BINPUT     2
   16: K        BININT1    0
   ...

cPickle, python 2.7:

    0: \x80 PROTO      2
    2: }    EMPTY_DICT
    3: q    BINPUT     1
    5: (    MARK
    6: U        SHORT_BINSTRING 'QGgRa'
   13: \x85     TUPLE1
   14: K        BININT1    1
   ...

As you can see, pickle memoizes both the string and the tuple. cPickle2.6 just memoizes the tuple, while cPickle2.7 doesn't memoize either. It therefore seems that the problem is somehow related to the memo...

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue14775>
_______________________________________