New+old-style multiple inheritance

[stephenpas at googlemail.com]

Thank Gabriel,

That sounds exactly right and your work-around provides confirmation.
This code is an expedient solution for us so I expect we'll go with
forcing a slot update.  Hopefully the library concerned is going to
fix the problem in a future release.

Cheers,
Stephen.

On  Dec 18, 7:40 pm, Gabriel Genellina <gagsl-... at yahoo.com.ar> wrote:
> On 18 dic, 12:08, "stephen... at googlemail.com"
>
>
>
> <stephen... at googlemail.com> wrote:
> > We are trying to monkey-patch a third-party library that mixes new and
> > old-style classes with multiple inheritance.  In so doing we have
> > uncovered some unexpected behaviour:
>
> > <quote>
> > class Foo:
> >     pass
>
> > class Bar(object):
> >     pass
>
> > class Baz(Foo,Bar):
> >     pass
>
> > # Monkey-patch Foo to add a special method
> > def my_nonzero(self):
> >     print "my_nonzero called"
> >     return False
> > Foo.__nonzero__ = my_nonzero
>
> > b = Baz()
>
> > print "doing the test on Baz(Foo,Bar).  Should return false"
> > if b:
> >     print "true"
> > else:
> >     print "false"
> > </quote>
>
> > Produces this output:
>
> >   doing the test on Baz(Foo,Bar).  Should return false
> >   true
>
> > With some experimentation it is clear that this behaviour only occurs
> > when you combine new+old-style multiple inheritance, monkey-patching
> > and special methods.  If Foo and Bar are either old or new-style it
> > works.  calling b.__nonzero__() directly works.  Defining __nonzero__
> > within Foo works.
>
> > I know this level of messing with python internals is a bit risky but
> > I'm wondering why the above code doesn't work.
>
> I think I can barely explain what happens here (but I may be
> absolutely wrong! Please someone with greater knowledge of Python
> innards correct whatever is wrong on my description!)
>
> type objects contain "slots" (function pointers) corresponding to the
> "magic" methods like __nonzero__ (this one is stored into the
> nb_nonzero slot). new-style classes are types; old-style classes are
> not (they're all instances of classobj).
> The slots are populated when a new type is created (e.g., when
> creating a new-style class) and are updated when a magic attribute is
> set onto the type. By example, setting the __nonzero__ attribute on a
> new-style class updates the nb_nonzero slot. old-style classes just
> store the magic attribute in its __dict__. Note that if you patch Foo
> *before* defining Baz, it works fine, because Baz sees the magic
> attribute and can populate its nb_nonzero slot when the new type is
> created.
>
> When you define Baz, neither Foo nor Bar have a __nonzero__ at this
> time, so the nb_nonzero slot on Baz is empty.
> Later, when you alter Foo, Baz cannot notice it (Foo has no way to
> notify Baz that something changed).
>
> If Foo were a new-style class, things are different: new-style classes
> maintain a list of subclasses, and the subclasses can then be notified
> of changes. In particular, setting a magic attribute on a base class
> notifies all its subclasses, and the corresponding slots are updated.
>
> The problem seems to be exactly that: old-style base classes can't
> notify its derived new-style classes when magic methods are added, so
> the corresponding slots aren't updated. Always asking the base class
> whether it has a magic method or not would slow down all method
> lookups.
>
> To force a slot update:
>
> py> Baz.__nonzero__ = "xxx"
> py> del Baz.__nonzero__
> py> bool(b)
> my_nonzero called
> False
>
> (setting or deleting __nonzero__ triggers the slot update)
>
> This is *why* it happens. How to avoid this... well, don't do that in
> the first place :) Or try to patch the base class *before* the new-
> style derived class is defined. Or replace the derived class with a
> new version of itself (once the base was patched). Or, if you know all
> the derived classes, force a slot update on them as above.
>
> --
> Gabriel Genellina