[Python-Dev] PEP 487: Simpler customization of class creation

[Nick Coghlan]

On 24 June 2016 at 00:41, Martin Teichmann <lkb.teichmann at gmail.com> wrote:
> Hi list,
>
> just recently, I posted about the implementation of PEP 487. The
> discussion quickly diverted to PEP 520, which happened to be
> strongly related.
>
> Hoping to get some comments about the rest of PEP 487, I took
> out the part that is also in PEP 520.

Good idea :)

=========================
> Proposal
> ========
>
> While there are many possible ways to use a metaclass, the vast majority
> of use cases falls into just three categories: some initialization code
> running after class creation, the initalization of descriptors and
> keeping the order in which class attributes were defined.
>
> Those three use cases can easily be performed by just one metaclass.

This section needs to be tweaked a bit to defer to PEP 520 for
discussion of the 3rd case.

> If
> this metaclass is put into the standard library, and all libraries that
> wish to customize class creation use this very metaclass, no combination
> of metaclasses is necessary anymore. Said metaclass should live in the
> ``types`` module under the name ``Type``. This should hint the user that
> in the future, this metaclass may become the default metaclass ``type``.

As long as the PEP still proposes phased integration into the standard
library and builtins (more on that below) I'd suggest being explicit
here in the proposal section that the non-default metaclasses in the
standard library (abc.ABCMeta and enum.EnumMeta) should be updated to
inherit from the new types.Type.

> The three use cases are achieved as follows:

"The three ..." -> "These ..."

> 1. The metaclass contains an ``__init_subclass__`` hook that initializes
>    all subclasses of a given class,
> 2. the metaclass calls a ``__set_owner__`` hook on all the attribute
>    (descriptors) defined in the class, and

This part isn't entirely clear to me, so you may want to give some
Python pseudo-code that:

- is explicit regarding exactly when this new code runs in the type
creation process
- whether the __set_owner__ hooks are called before or after
__init_subclass__ runs, or only when the subclass calls up to
super().__init_subclass__, and the implications of each choice (either
descriptors see a partially initialised class object, or init_subclass
sees partially initialised descriptor objects, or that choice is
delegated to individual subclasses)
- how the list of objects to be checked for "__set_owner__" methods is
retrieved (presumably via "ns.items()" on the class definition
namespace, but the PEP should be explicit)

For the second point, my personal preference would be for descriptors
to have their owner set first and independently of __init_subclass__
super calls (as it seems more likely that __init_subclass__ will
depend on having access to fully initialised descriptors than the
other way around).

> Reduced chance of metaclass conflicts
> -------------------------------------
>
> One of the big issues that makes library authors reluctant to use metaclasses
> (even when they would be appropriate) is the risk of metaclass conflicts.
> These occur whenever two unrelated metaclasses are used by the desired
> parents of a class definition. This risk also makes it very difficult to
> *add* a metaclass to a class that has previously been published without one.
>
> By contrast, adding an ``__init_subclass__`` method to an existing type poses
> a similar level of risk to adding an ``__init__`` method: technically, there
> is a risk of breaking poorly implemented subclasses, but when that occurs,
> it is recognised as a bug in the subclass rather than the library author
> breaching backwards compatibility guarantees.

This section needs some additional explanation of how it fares given
the proposed migration plan below. I *think* it would be fine,
assuming that in 3.7, the types module gains the lines:

    Type = type
    Object = object

As that would collapse the hierarchy again, even for classes that had
declared inheritance from types.Object or the direct use of types.Type
as their metaclass in 3.6

Honestly though, I'm not sure this additional user-visible complexity
is worth it - "The default type metaclass has this new behaviour" is a
lot easier to document and explain than "We added a new opt-in
alternate metaclass that you can use if you want, and in the next
version that will just become an alias for the builtin types again".
We'd also end up being stuck with types.Type and types.Object as
aliases for the type and object builtins forever (with the associated
"How does 'class name:' or 'class name(object)' differ from 'class
name(types.Object)'?" question and "It doesn't, unless you're using
Python 3.6" answer for folks learning the language for the first
time).

If we decide __init_subclass__ and __set_owner__ are good ideas, let's
just implement them, with a backport available on PyPI for folks that
want to use them on earlier versions, including in Python 2/3
compatible code.

> A path of introduction into Python
> ==================================
>
> Most of the benefits of this PEP can already be implemented using
> a simple metaclass. For the ``__init_subclass__`` hook this works
> all the way down to Python 2.7, while the attribute order needs Python 3.0
> to work. Such a class has been `uploaded to PyPI`_.

This paragraph should refer to just __init_subclass__ and
__set_owner__ now, since the attribute ordering problem has been moved
out to PEP 520.

[snip: see above for further comments on why I think this additional
complexity in the migration plan might not be worth it]

> Rejected Design Options
> =======================
> Defining arbitrary namespaces
> -----------------------------
>
> PEP 422 defined a generic way to add arbitrary namespaces for class
> definitions. This approach is much more flexible than just leaving
> the definition order in a tuple. The ``__prepare__`` method in a metaclass
> supports exactly this behavior. But given that effectively
> the only use cases that could be found out in the wild were the
> ``OrderedDict`` way of determining the attribute order, it seemed
> reasonable to only support this special case.

Since it isn't tackling definition order any more, this section can
now be left out of this PEP.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia