[Python-ideas] An even simpler customization of class creation

[Nick Coghlan]

On 2 March 2015 at 20:57, Martin Teichmann <lkb.teichmann at gmail.com> wrote:
>>> Fortunately, this is easily changeable: it is possible to move the
>>> initialization of __class__ into type.__new__. As the class is
>>> (typically) created there, that is about as early as possible.
>>
>> Unfortunately, it's too early - we can't control whether or not subclasses
>> actually return the result of type.__new__ from their own __new__
>> implementation, so that object could have been replaced by something else by
>> the time __init__ gets called.
>
> Well, this is actually just a question of definition and documentation.
> Sure, a metaclasses __new__ may return whatever it pleases, but this
> is not in contradiction of setting __class__ of the methods in type.__new__.
>
> There are two possible scenarios:
>
> 1. metaclass.__new__ does not call type.__new__. Once we document that
> it is type.__new__'s job to set __class__, nobody can complain if __class__
> is not set if type.__new__ is not called, as this would be documented behavior.
> In my changes I actually still left in the old code, so __build_class__ would
> still try to set __class__ even if type.__new__ did not get called.
>
> 2. somehow type.__new__ does get called (most likely by metaclass.__new__).
> It will do its job and set the __class__ of the methods of the class being
> created. Whatever metaclass.__new__ returns then doesn't matter much,
> because what sense would it make to set __class__ of that object it creates?
>
> To give an example:
> Imagine a soliton-generating metaclass:
>
>     class Soliton(type):
>        def __new__(cls, name, bases, ns):
>            self = super().__new__(name, bases, ns)
>            return self()
>
> And generate such a soliton:
>
>     class A(metaclass=Soliton):
>         def f(self):
>             print(__class__)
>
> As of now, writing "A.f()" interestingly prints "<__main__.A object>", so
> __class__ is indeed set to what Soliton.__new__ returns, the object,
> not the class.
>
> This is currently correct behavior, but I think it actually is not what one
> expects, nor what one desires. (Does anyone out there make use of
> such a construct? Please speak up!) super() certainly won't work.
> So I think it would actually be a change for the better to let
> type.__new__ set the __class__ of the generated class.

Thinking about this a bit more, I think a good way to look at it is to
consider the impact of:

1. Using explicit super() rather than implicit super()
2. Using explicit decorators

Currently, in your example, "__class__" and "A" will both refer to an
*instance* of the class.

If, instead, you used a "singleton" explicit decorator, then
"__class__ " would refer to the actual class object, while "A" would
refer to the instance.

With your proposed change, then "__class__" will always refer to the
actual class object, even if the metaclass __new__ plays implicit
decoration games.

It also means that class methods that rely on __class__ (whether
explicitly or implicitly through super()) will "just work" in
metaclass __init__ and __new__ methods, although only after the call
up to the base class implementation in the latter case.

> I know that it technically breaks backward compatibility. But I ran
> all the tests and none failed, so apparently until now nobody got
> weird ideas like me... I even tried to remove the old code from
> __build_class__ and still all tests run. (just a note, if anyone else
> here is trying to do so, I tampered with the compiler, so you better
> delete your .pyc files before running the tests)

Aye, and the discrepancy I was concerned about can already happen when
using explicit decorators. That suggest to me that even if someone
*was* relying on __class__ pointing to the same object as the bound
name when a metaclass does something odd, they can likely switch to
referring to it by name instead.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia