[Python-Dev] Pickling instances of nested classes

[Samuele Pedroni]

Walter Dörwald wrote:

> Samuele Pedroni wrote:
>
>>> [...]
>>> And having the full name of the class available would certainly help 
>>> in debugging.
>>
>>
>> that's probably the only plus point but the names would be confusing wrt
>>  modules vs. classes.
>
>
> You'd propably need a different separator in repr. XIST does this:
>
> >>> from ll.xist.ns import html
> >>> html.a.Attrs.href
> <attribute class ll.xist.ns.html:a.Attrs.href at 0x8319284>
>
>> My point was that enabling reduce hooks at the metaclass level has
>> propably other interesting applications, is far less complicated than
>> your proposal to implement, it does not further complicate the notion of
>> what happens at class creation time, and indeed avoids the
>> implementation costs (for all python impls) of your proposal and still
>> allows fairly generic solutions to the problem at hand because the
>> solution can be formulated at the metaclass level.
>
>
> Pickling classes like objects (i.e. by using the pickling methods in 
> their (meta-)classes) solves only the second part of the problem: 
> Finding the nested classes in the module on unpickling. The other 
> problem is to add additional info to the inner class, which gets 
> pickled and makes it findable on unpickling.
>
>> If pickle.py is patched along these lines [*] (strawman impl, not much
>> tested but test_pickle.py still passes, needs further work to support
>> __reduce_ex__ and cPickle would need similar changes) then this 
>> example works:
>>
>>
>> class HierarchMeta(type):
>>   """metaclass such that inner classes know their outer class, with 
>> pickling support"""
>>   def __new__(cls, name, bases, dic):
>>       sub = [x for x in dic.values() if isinstance(x,HierarchMeta)]
>
>
> I did something similar to this in XIST, but the problem with this 
> approach is that in:
>
> class Foo(Elm):
>    pass
>
> class Bar(Elm):
>    Baz = Foo
>
> the class Foo will get its _outer_ set to Bar although it shouldn't.

this should approximate that behavior better: [not tested]

   import sys

  ....
  def __new__(cls, name, bases, dic):
      sub = [x for x in dic.values() if isinstance(x,HierarchMeta)]
      newtype = type.__new__(cls, name, bases, dic)
      for x in sub:
          if not hasattr(x, '_outer_') and 
getattr(sys.modules.get(x.__module__), x.__name__, None) is not x:
               x._outer_ = newtype
      return newtype

  .....

we don't set _outer_ if a way to pickle the class is already there