[Tutor] question about metaclasses

[Steven D'Aprano]

On Thu, Jan 18, 2018 at 05:14:43PM +0000, Albert-Jan Roskam wrote:

> Is a metaclass the best/preferred/only way of doing this? Or is a 
> class decorator an alternative route?

I haven't thought deeply about this, but I suspect a class decorator 
should do the job too.

The general advice is to use the simplest thing that does the job. The 
deeper you have to go into Python's scary internals, the better reason 
you should have:

- ordinary Python code is simpler than...
- code using decorators, which is simpler than...
- classes using custom descriptors, which is simpler than...
- classes with __init_subclass__, which is simpler than...
- code with metaclasses.

So in general you should pick the first technique (starting at the top) 
which solves your problem in a satisfactory manner.

The __init_subclass__ method is new to Python 3.6:

https://docs.python.org/3/reference/datamodel.html#object.__init_subclass__


Remember the excellent advice:

Debugging is harder than programming, so if you write the trickiest and 
most clever code you are capable of, by definition you won't be able to 
debug it.

Having said that, I applaud you for investigating metaclasses!


> Is the following analogy for doing stuff when a class is created ('born') correct?
> Metaclass --> prenatal surgery
> __new__ --> perinatal surgery
> Class decorator --> postnatal surgery


The last one is certainly correct: a class decorator gets called with 
the fully created class object, and then has the opportunity to modify 
it as needed.

It isn't clear to me what you mean by __new__. Do you mean the *class* 
__new__ method or the *metaclass* __new__ method?

If I write this:

class MetaC(type):
    def __new__(meta, *args):
        # pretend this does something useful

class C(metaclass=MetaC):
    def __new__(cls, *args):
        ...


then MetaC.__new__ and C.__new__ are called at very different times.

MetaC.__new__ is called as part of the process of creating C in the 
first place; so at the beginning of MetaC.__new__, C does not exist. 

C.__new__ on the other hand doesn't get called when creating C, it gets 
called when creating instances of C. Think of __new__ as the twin of 
__init__, with the following differences:

* __new__ is called before the instance ("self") exists, 
  and gets to customise the creation of the instance;

* __init__ is called after the instance is created, and
  gets to modify self provided self is mutable.


So writing an __init__ method for (say) a string subclass is usually a 
waste of time, since the string is already set and cannot be changed. 
This does not work:

class UpperString(str):
    # make strings uppercase
    def __init__(self):
        self = self.upper()

But this does:

class UpperString(str):
    def __new__(cls, arg):
        arg = str(arg).upper()
        return super().__new__(cls, arg)


Metaclasses are both extremely powerful *and* mind-blowing (back in 
Python 1.5, they were known as "the Killer Joke") because they can 
customise (nearly?) any part of the class operation. So you can think of 
a metaclass as all of the following:

- prenatal surgery (metaclass.__new__);

- postnatal surgery (metaclass.__init__);

- cybernetic implants (metaclass.__getattribute__, __getattr__ etc);

- mind-meld or telepathy between otherwise unrelated classes 
  (define methods in the metaclass as a kind of alternative to
  inheritence);

- gene therapy (all of the above?);

- (I can't think of a good analogy for this one): metaclass.__call__
  lets you customized what happens when you call your class, allowing 
  you to manipulate the arguments or the returned instance.

For example:

class Meta(type):
    def __call__(self, *args):
        print("called by", self)
        instance = super().__call__(*args)
        instance.foo = 1
        return instance


prints a message and adds a "foo" attribute to every instance of every 
class that belongs to it.

Oh, and just to make it more fun... metaclasses don't have to be a 
class! See if you can predict what this will do:


def meta(*args):
    print("Hi there, I'm your metaclass for today, how can I blow your mind?")
    return 42

class A(metaclass=meta):
    pass


See also The Killer Joke:
https://www.python.org/doc/essays/metaclasses/

That was written back in Python 1.5 days, so a lot of the information in 
it is now obsolete. But the basic techniques should more or less work 
today.



-- 
Steve