weird pickle behavior in Python 3.1.2 + Eclipse 3.5.2

[Peter Otten]

Fabio Zadrozny wrote:

> On Mon, Jun 7, 2010 at 6:30 AM, Peter Otten <__peter__ at web.de> wrote:
>> kirby.urner at gmail.com wrote:
>>
>>> On Jun 4, 9:47 am, Peter Otten <__pete... at web.de> wrote:
>>>
>>>> I can provoke the error in "naked" Python 3 by changing the
>>>> Example.__module__ attribute:
>>>>
>>>> Python 3.1.1+ (r311:74480, Nov  2 2009, 15:45:00)
>>>> [GCC 4.4.1] on linux2
>>>> Type "help", "copyright", "credits" or "license" for more information.
>>>
>>>> >>> import pickle
>>>> >>> class Example:
>>>>
>>>> ...     pass
>>>> ...
>>>> >>> pickle.dumps(Example())
>>>>
>>>> b'\x80\x03c__main__\nExample\nq\x00)\x81q\x01}q\x02b.'
>>>
>>>> >>> Example.__module__ = "builtins"
>>>> >>> pickle.dumps(Example())
>>>>
>>>> Traceback (most recent call last):
>>>> File "<stdin>", line 1, in <module>
>>>> File "/usr/lib/python3.1/pickle.py", line 1358, in dumps
>>>> Pickler(f, protocol, fix_imports=fix_imports).dump(obj)
>>>> _pickle.PicklingError: Can't pickle <class 'Example'>: attribute lookup
>>>> builtins.Example failed
>>>>
>>>> What's the value of __module__ when you run your code in Eclipse?
>>>>
>>>> Peter
>>>
>>> Thank you for replying.
>>>
>>> Here's from Eclipse console:
>>>
>>>>>> Example.__module__
>>> 'builtins'
>>>
>>>>>> __name__
>>> 'builtins'
>>>
>>> Duplicating your result in naked Python:
>>>
>>> Python 3.1.2 (r312:79149, Mar 21 2010, 00:41:52) [MSC v.1500 32 bit
>>> (Intel)] on win32
>>> Type "copyright", "credits" or "license()" for more information.
>>>>>> class Example:
>>> pass
>>>
>>>>>> import pickle
>>>>>> Example.__module__
>>> '__main__'
>>>>>> f = open('testpickle.pkl','wb')
>>>>>> obj = Example()
>>>>>> obj
>>> <__main__.Example object at 0x02A26690>
>>>>>> pickle.dump(obj, f)
>>>
>>>>>> Example.__module__ = 'builtins'
>>>>>> obj2 = Example()
>>>
>>>>>> pickle.dump(obj2, f)
>>> Traceback (most recent call last):
>>> File "<pyshell#12>", line 1, in <module>
>>> pickle.dump(obj2, f)
>>> File "C:\Python31\lib\pickle.py", line 1354, in dump
>>> Pickler(file, protocol, fix_imports=fix_imports).dump(obj)
>>> _pickle.PicklingError: Can't pickle <class 'Example'>: attribute
>>> lookup builtins.Example failed
>>>
>>> So what if I'm in an Eclipse pydev console and
>>> change the Example.__module__ to '__main__'....
>>>
>>>>>> import sys; print('%s %s' % (sys.executable or sys.platform,
>>>>>> sys.version))
>>> C:\Python31\python.exe 3.1.2 (r312:79149, Mar 21 2010, 00:41:52) [MSC
>>> v.1500 32 bit (Intel)]
>>>>>> import pickle
>>>>>> class Example:
>>> ...     pass
>>> ...
>>>>>> Example.__module__
>>> 'builtins'
>>>>>> Example.__module__ = '__main__'
>>>>>> obj = Example()
>>>>>> obj
>>> <__main__.Example object at 0x029E8FD0>
>>>>>> f = open('testpickle.pkl','wb')
>>>>>> pickle.dump(obj, f)
>>> Traceback (most recent call last):
>>> File "<console>", line 1, in <module>
>>> File "C:\Python31\lib\pickle.py", line 1354, in dump
>>> Pickler(file, protocol, fix_imports=fix_imports).dump(obj)
>>> _pickle.PicklingError: Can't pickle <class '__main__.Example'>:
>>> attribute lookup __main__.Example failed
>>>>>>
>>>
>>> Dang.
>>>
>>> Any insights?
>>
>> Sorry, no. Consider a bug report to the pydev project.
> 
> I've taken a look at the pydev code and the issue seems to be in
> python itself. See: http://bugs.python.org/issue8943

Thanks for taking a look!

While this is being worked on you could put a workaround into pydev: you'd 
just have to pass the global dict of a real module to InteractiveConsole:

$ cat fast_test.py
import sys
from code import InteractiveConsole

if "-m" in sys.argv:
    console = InteractiveConsole(globals())
else:
    console = InteractiveConsole()

contents = '''
import pickle
class Example:
    def __init__(self):
        self.name = "Hello"
    def __repr__(self):
        return "an Example object named " + self.name


obj = Example()
obj

f = open("testpickle.pkl", 'wb')
pickle.dump(obj, f)
f.close()

pickle.load(open("testpickle.pkl", "rb"))
'''

for line in contents.splitlines():
    console.push(line)
$ python2.4 fast_test.py -m
an Example object named Hello
an Example object named Hello
$ python2.5 fast_test.py -m
an Example object named Hello
an Example object named Hello
$ python2.6 fast_test.py -m
an Example object named Hello
an Example object named Hello
$ python3.1 fast_test.py -m
an Example object named Hello
an Example object named Hello
$

Peter