[Python-Dev] Evil setattr hack

[Phillip J. Eby]

At 02:33 PM 4/15/03 -0400, Guido van Rossum wrote:
>[Guido]
> >  >I've checked in what I believe is an adequate block for at least
> >  >this particular hack.  wrap_setattr(), which is called in response
> >  >to <type>.__setattr__(), now compares if the C function it is
> >  >about to call is the same as the C function in the built-in base
> >  >class closest to the object's class.  This means that if B is a
> >  >built-in class and P is a Python class derived from B,
> >  >P.__setattr__ can call B.__setattr__, but not A.__setattr__ where
> >  >A is an (also built-in) base class of B (unless B inherits
> >  >A.__setattr__).
>
> > From: "Phillip J. Eby" <pje@telecommunity.com>
>
> > Does this follow __mro__ or __base__?
>
>It follows __base__, like everything concerned about C level instance
>lay-out.
>
> > I'm specifically wondering about the implications of multiple
> > inheritance from more than one C base class; this sort of thing
> > (safety checks relating to heap vs. non-heap types and the "closest"
> > method of a particular kind) has bitten me before in relation to
> > ZODB4's Persistence package.
>
>It is usually impossible to inherit from more than one C base class,
>unless all but one are mix-in classes, meaning they add nothing to the
>instance lay-out of a common base class.
>
> > In that context, mixing 'type' and 'PersistentMetaClass' makes it
> > impossible to instantiate the resulting metaclass, because neither
> > type.__new__ nor PersistentMetaClass.__new__ is considered "safe" to
> > execute.
>
>You're referring to this error message from tp_new_wrapper(), right:
>
>     "%s.__new__(%s) is not safe, use %s.__new__()"

Yep, that's the one.


> > My "evil hack" to fix that was to add an extra PyObject *
> > to PersistentMetaClass so that it has a larger tp_basicsize than
> > 'type' and Python then considers it the '__base__' type, thus
> > causing its '__new__' method to be accepted as legitimate.
>
>Is this because the algorithm in best_base() picks the wrong base
>otherwise?

Yes, at least for Python 2.2.  However, the problem with ZODB4 was only an 
issue on 2.2; on 2.3, PersistentMetaClass *is* 'type', because it is there 
to workaround C layout issues in 2.2 that don't exist in 2.3.  So this is 
probably all moot.

Anyway...  if I recall correctly, even if you got best_base() to pick the 
right base by changing the order of mixing in the classes, you got a 
*different* safety error message; I think it might have been in the 
resulting class, though, rather than in the metaclass.  This was all back 
in November, so my memory is a little hazy.  I think there might have been 
more details in the Zope3-Dev collector issue (#86), but I think Jeremy 
showed that info to you previously and said that it wasn't enough for you 
to understand what the problem was.  I think part of the complexity had to 
do with the fact that one of the types (my subclass of 'type') was a "heap 
type", and PersistentMetaClass was not.

But as you pointed out, subclassing from multiple C bases is a rarity, so I 
don't see any point to following this up further, unless you have some 
perverse desire to have yet another new-style class layout algorithm change 
for Python 2.2.3.  :) It's probably better just to leave my "make it 
bigger" hack in ZODB4, since PersistentMetaClass itself is one big Python 
2.2 backward compatibility hack anyway. <wink>