[issue19754] pickletools.optimize doesn't reframe correctly

[Antoine Pitrou]

New submission from Antoine Pitrou:

pickletools.optimize() can output arbitrarily large frames (much larger than then 64 KB heuristic). This may be annoying for memory use when pickling -- on the other hand if you keep the whole pickle in memory for optimize() perhaps it's not a problem keeping it in memory for load() :-)

$ ./python -c "import pickle, pickletools; d = pickle.dumps(list(range(100000)), 4); pickletools.dis(d)"  | head
    0: \x80 PROTO      4
    2: \x95 FRAME      65537
   11: ]    EMPTY_LIST
   12: \x94 MEMOIZE
   13: (    MARK
   14: K        BININT1    0
   16: K        BININT1    1
   18: K        BININT1    2
   20: K        BININT1    3
   22: K        BININT1    4

$ ./python -c "import pickle, pickletools; d = pickle.dumps(list(range(100000)), 4); pickletools.dis(pickletools.optimize(d))"  | head
    0: \x80 PROTO      4
    2: \x95 FRAME      368875
   11: ]    EMPTY_LIST
   12: \x94 MEMOIZE
   13: (    MARK
   14: K        BININT1    0
   16: K        BININT1    1
   18: K        BININT1    2
   20: K        BININT1    3
   22: K        BININT1    4

----------
messages: 204213
nosy: alexandre.vassalotti, pitrou, tim.peters
priority: low
severity: normal
status: open
title: pickletools.optimize doesn't reframe correctly
type: resource usage
versions: Python 3.4

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19754>
_______________________________________