[issue25761] Improve unpickling errors handling

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Proposed patch allows unpickler to detect errors related to reading a stack behind a mark.

For now Python implementation just pops a sentinel used as a mark. This can cause TypeError, AttributeError or UnpicklingError besides IndexError:

>>> pickle._loads(b'}(NNs.')
Traceback (most recent call last):
  ...
TypeError: 'object' object does not support item assignment
>>> pickle._loads(b'](Na.')
Traceback (most recent call last):
  ...
AttributeError: 'object' object has no attribute 'append'

Or can silently expose the mark object:

>>> pickle._loads(b')(.')
<object object at 0xb71084b0>
>>> pickle._loads(b']](a.')
[<object object at 0xb71084d8>]

C implementation just ignores incorrect mark:

>>> pickle.loads(b'}(NNs.')
{None: None}
>>> pickle.loads(b'](Na.')
[None]
>>> pickle.loads(b')(.')
()
>>> pickle.loads(b']](a.')
[[]]

But in case of complex data this can cause errors later.

With the patch C implementation always raises UnpicklingError with relevant message and Python implementation always raises IndexError.

>>> pickle.loads(b'}(NNs.')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'pickle' is not defined
>>> import pickle, pickletools
>>> pickle.loads(b'}(NNs.')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
_pickle.UnpicklingError: unexpected MARK found
>>> pickle.loads(b'](Na.')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
_pickle.UnpicklingError: unexpected MARK found
>>> pickle.loads(b')(.')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
_pickle.UnpicklingError: unexpected MARK found
>>> pickle.loads(b']](a.')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
_pickle.UnpicklingError: unexpected MARK found
>>> pickle._loads(b'}(NNs.')
Traceback (most recent call last):
  ...
IndexError: list index out of range
>>> pickle._loads(b'](Na.')
Traceback (most recent call last):
  ...
IndexError: list index out of range
>>> pickle._loads(b')(.')
Traceback (most recent call last):
  ...
IndexError: pop from empty list
>>> pickle._loads(b']](a.')
Traceback (most recent call last):
  ...
IndexError: pop from empty list

----------
keywords: +patch
stage:  -> patch review
versions: +Python 3.6
Added file: http://bugs.python.org/file41188/unpickling_mark_errors.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25761>
_______________________________________