New+old-style multiple inheritance

[Gabriel Genellina]

On 18 dic, 12:08, "stephen... at googlemail.com"
<stephen... at googlemail.com> wrote:

> We are trying to monkey-patch a third-party library that mixes new and
> old-style classes with multiple inheritance.  In so doing we have
> uncovered some unexpected behaviour:
>
> <quote>
> class Foo:
>     pass
>
> class Bar(object):
>     pass
>
> class Baz(Foo,Bar):
>     pass
>
> # Monkey-patch Foo to add a special method
> def my_nonzero(self):
>     print "my_nonzero called"
>     return False
> Foo.__nonzero__ = my_nonzero
>
> b = Baz()
>
> print "doing the test on Baz(Foo,Bar).  Should return false"
> if b:
>     print "true"
> else:
>     print "false"
> </quote>
>
> Produces this output:
>
>   doing the test on Baz(Foo,Bar).  Should return false
>   true
>
> With some experimentation it is clear that this behaviour only occurs
> when you combine new+old-style multiple inheritance, monkey-patching
> and special methods.  If Foo and Bar are either old or new-style it
> works.  calling b.__nonzero__() directly works.  Defining __nonzero__
> within Foo works.
>
> I know this level of messing with python internals is a bit risky but
> I'm wondering why the above code doesn't work.

I think I can barely explain what happens here (but I may be
absolutely wrong! Please someone with greater knowledge of Python
innards correct whatever is wrong on my description!)

type objects contain "slots" (function pointers) corresponding to the
"magic" methods like __nonzero__ (this one is stored into the
nb_nonzero slot). new-style classes are types; old-style classes are
not (they're all instances of classobj).
The slots are populated when a new type is created (e.g., when
creating a new-style class) and are updated when a magic attribute is
set onto the type. By example, setting the __nonzero__ attribute on a
new-style class updates the nb_nonzero slot. old-style classes just
store the magic attribute in its __dict__. Note that if you patch Foo
*before* defining Baz, it works fine, because Baz sees the magic
attribute and can populate its nb_nonzero slot when the new type is
created.

When you define Baz, neither Foo nor Bar have a __nonzero__ at this
time, so the nb_nonzero slot on Baz is empty.
Later, when you alter Foo, Baz cannot notice it (Foo has no way to
notify Baz that something changed).

If Foo were a new-style class, things are different: new-style classes
maintain a list of subclasses, and the subclasses can then be notified
of changes. In particular, setting a magic attribute on a base class
notifies all its subclasses, and the corresponding slots are updated.

The problem seems to be exactly that: old-style base classes can't
notify its derived new-style classes when magic methods are added, so
the corresponding slots aren't updated. Always asking the base class
whether it has a magic method or not would slow down all method
lookups.

To force a slot update:

py> Baz.__nonzero__ = "xxx"
py> del Baz.__nonzero__
py> bool(b)
my_nonzero called
False

(setting or deleting __nonzero__ triggers the slot update)

This is *why* it happens. How to avoid this... well, don't do that in
the first place :) Or try to patch the base class *before* the new-
style derived class is defined. Or replace the derived class with a
new version of itself (once the base was patched). Or, if you know all
the derived classes, force a slot update on them as above.

--
Gabriel Genellina