[New-bugs-announce] [issue38770] Pickle handle self references in classes

[Saim Raza]

New submission from Saim Raza <saimorsaim at gmail.com>:

If the __qualname__ of a class is set to have a circular reference to itself, pickle behaves differently based on protocol. Following script demonstrates the issue:

======================================================
from __future__ import print_function

import pickle, sys

class Foo:
    __name__ = __qualname__ = "Foo.ref"
    pass

Foo.ref = Foo

print(sys.version_info)

for proto in range(0, pickle.HIGHEST_PROTOCOL + 1):
    print("{}:".format(proto), end=" ")
    try:
        pkl = pickle.dumps(Foo, proto)
        print("Dump OK,", end=" ")
        assert(pickle.loads(pkl) is Foo)
        print("Load OK,")
    except Exception as err:
        print(repr(err))
======================================================
OUTPUT:
Python2.7:
sys.version_info(major=2, minor=7, micro=16, releaselevel='final', serial=0)
0: Dump OK, Load OK,
1: Dump OK, Load OK,
2: Dump OK, Load OK,

Python3.7:
sys.version_info(major=3, minor=7, micro=3, releaselevel='final', serial=0)
0: RecursionError('maximum recursion depth exceeded while pickling an object')
1: RecursionError('maximum recursion depth exceeded while pickling an object')
2: RecursionError('maximum recursion depth exceeded while pickling an object')
3: RecursionError('maximum recursion depth exceeded while pickling an object')
4: Dump OK, Load OK,
======================================================

This was introduced as a side effect of issue#23611 (?). I can think of the following approaches to fix the issue and make the behavior consistent:

1. Check if the class has a self-reference and raise an error for all protocols.
2. Use memoization to handle self-references. I am not sure what should be dumped in this case. In the example above `Foo` will exist in the namespace but not `Foo.ref`. 
3. Dump such classes similar to Python 2 pickle and Python 3 pickle protocol >= 4.

I had a stab at pickle.py and had a bit of success in doing point 3 above. Posting this issue for discussions. I would be happy to submit a PR for this issue.

Thanks,
Saim Raza

----------
components: Library (Lib)
messages: 356388
nosy: Saim Raza, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Pickle handle self references in classes
type: behavior
versions: Python 2.7, Python 3.7

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38770>
_______________________________________