[New-bugs-announce] [issue29368] Optimize unpickling list-like objects

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

According to PEP 307 the extend method can be used for appending list items to the object.

    listitems    Optional, and new in this PEP.
                 If this is not None, it should be an iterator (not a
                 sequence!) yielding successive list items.  These list
                 items will be pickled, and appended to the object using
                 either obj.append(item) or obj.extend(list_of_items).
                 This is primarily used for list subclasses, but may
                 be used by other classes as long as they have append()
                 and extend() methods with the appropriate signature.
                 (Whether append() or extend() is used depends on which
                 pickle protocol version is used as well as the number
                 of items to append, so both must be supported.)

Proposed patch makes the extend method be used in the APPENDS opcode. To avoid breaking existing code the use of the extend method is optional.

Microbenchmark:

$ ./python -m timeit -s "import pickle, collections; p = pickle.dumps(collections.deque([None]*10000), 4)" -- "pickle.loads(p)"
Unpatched:  100 loops, best of 5: 2.02 msec per loop
Patched:    500 loops, best of 5: 833 usec per loop

----------
components: Library (Lib)
files: pickle-appends-extend.patch
keywords: patch
messages: 286237
nosy: alexandre.vassalotti, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Optimize unpickling list-like objects
type: performance
versions: Python 3.7
Added file: http://bugs.python.org/file46412/pickle-appends-extend.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue29368>
_______________________________________