[Python-Dev] PEP 487 vs 422 (dynamic class decoration)

[PJ Eby]

On Wed, Apr 1, 2015 at 10:39 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 2 April 2015 at 07:35, PJ Eby <pje at telecommunity.com> wrote:
>> I recently got an inquiry from some of my users about porting some of
>> my libraries to Python 3 that make use of the Python 2 __metaclass__
>> facility.  While checking up on the status of PEP 422 today, I found
>> out about its recently proposed replacement, PEP 487.
>>
>> While PEP 487 is a generally fine PEP, it actually *rules out* the
>> specific use case that I wanted PEP 422 for in the first place:
>> dynamic addition of callbacks or decorators for use at class creation
>> time without requiring explicit inheritance or metaclass
>> participation.  (So that e.g. method decorators can access the
>> enclosing class at class definition time.)
>
> How hard is the requirement against relying on a mixin class or class
> decorator to request the defining class aware method decorator
> support? Is the main concern with the fact that failing to apply the
> right decorator/mixin at the class level becomes a potentially silent
> failure where the class aware method decorators aren't invoked
> properly?

The concern is twofold: it breaks proper information hiding/DRY, *and*
it fails silently.  It should not be necessary for clients of package
A1 (that uses a decorator built using package B2) to mixin a metaclass
or decorator from package C3 (because B2 implemented its decorators
using C3), just for package A1's decorator to work properly in the
*client package's class*.  (And then, of course, this all silently
breaks if you forget, and the breakage might happen at the A1, B2, or
C3 level.)

Without a way to hook into the class creation process, there is no way
to verify correctness and prevent the error from passing silently.
(OTOH, if there *is* a way to hook into the creation process, the
problem is solved: there's no need to mix anything in anyway, because
the hook can do whatever the mixin was supposed to do.)

The only way PEP 487 could be a solution is if the default
`object.__init_subclass__` supported one of the earlier __decorators__
or __autodecorate__ proposals, or if the PEP were for an
`__init_class__` that operated on the defining class, instead of
operating only on subclasses.  (I need to hook the creation of a class
that's *being defined*, not the definition of its future subclasses.)


> My preference at this point would definitely be to introduce a mixin
> class into the affected libraries and frameworks with an appropriate
> PEP 487 style __init_subclass__ that was a noop in Python 2 (which
> would rely on metaclass injection instead), but implemented the
> necessary "defining class aware" method decorator support in Python 3.

If this were suitable for the use case, I'd have done it already.
DecoratorTools has had a mixin that provides a __class_init__ feature
since 2007, which could be ported to Python 3 in a straighforward
manner as a third-party module.  (It's just a mixin that provides a
metaclass; under 3.x it could probably just be a plain metaclass with
no mixin.)


> The question of dynamically injecting additional base classes from the
> class body to allow the use of certain method decorators to imply
> specific class level behaviour could then be addressed as a separate
> proposal (e.g. making the case for an "__append_mixins__" attribute),
> rather than being linked directly to the question of how we going
> about defining inherited creation time behaviour without needing a
> custom metaclass.

Then maybe we should do that first, since PEP 487 doesn't do anything
you can't *already* do with a mixin, all the way back to Python 2.2.

IOW, there's no need to modify the core just to have *that* feature,
since if you control the base class you can already do what PEP 487
does in essentially every version of Python, ever.  If that's all PEP
487 is going to do, it should just be a PyPI package on a
stdlib-inclusion track, not a change to core Python.  It's not
actually adding back any of the dynamicness (dynamicity?
hookability?) that PEP 3115 took away.