magic names in python

[Steven D'Aprano]

On Tue, 05 Jun 2007 18:08:31 +0000, Lenard Lindstrom wrote:

> Steven D'Aprano wrote:
>> On Mon, 04 Jun 2007 22:19:35 +0000, Lenard Lindstrom wrote:
>> 
>>> What is "magic" about __init__ and __repr__? They are identifiers just 
>>> like "foo" or "JustAnotherClass". They have no special meaning to the 
>>> Python compiler. The leading and trailing double underscores represent 
>>> no special incantation. It is just a naming convention.
>> 
>> That's not quite true, as you point out:
> 
> Disassemble the code object of a class statement. "__init__" is just an 
> identifier, a Python name. So there is no "magic" here.

It is not JUST an identifier, it is an identifier with a special, dare I
say *magic*, meaning.

I suspect we have different ideas of what a "magic name" is. You seem to
be under the impression that magic names are a bad thing, and that
therefore one has to minimize their number. Hence you are glossing over
the differences and emphasizing the similarities between methods (say) foo
and __init__.

I don't consider "magic name" in this content to be anything to be ashamed
of, so I'm not concerned about downplaying the differences. Yes, there are
significant similarities between __init__ and random methods you write
yourself, but the differences are equally, if not more, significant.



>>> So a number of method names like __init__ and __repr__ have a 
>>> pre-defined usage. 
>> 
>> That makes them magic, in the best possible way.
> 
> I was careful to use the word "usage". Maybe I should have used 
> "protocol" or "specification" here. A method named "write" is understood 
> to have a particular definition and purpose. Is it "magic"? 

It could be, depending on the context.

If we're discussing file objects compared to str objects, then file
objects have a method "write" and str objects don't, and the method is
just another method.

But in the context of some third function, which takes a file-like object
with a "write" method, then yes it is magic, because any type of object
will work so long as it has a method "write" with the right semantics. Not
very much magic, but a little. 


> These 
> methods are simply callbacks. And yes, these callbacks make Python flexible.
> 
>> 
>> 
>>> In every other respect they are just normal methods. 
>> 
>> That is *almost* true. Or, to put it another way, that is wrong. Some of
>> the double-underscore magic methods are automatically called on the class
>> instead of the instance, bypassing normal inheritance. 
>> 
> 
> They exist as Python functions in the class dictionary. They have no 
> hidden flags that distinguish them from other class level function 
> attributes.

They don't need special flags, because the Python compiler itself already
knows about the existence of them and takes special actions with them.

[snip]


> I do admit that the special methods are given special treatment by the 
> type and ClassType types to ensure they are called by their 
> corresponding operations. But this special treatment is restricted to 
> the types themselves. In CPython an extension type can be written from 
> scratch that treats special methods exactly as a new-style class does. 
> Or an extension type can implement a completely novel approach. The 
> point is, at some level, the machinery which defines special method 
> "magic" is accessible to the programmer. So is it really "magic" or 
> advanced technology?

Does it matter? As Clarke said, any sufficiently advanced technology is
indistinguishable from magic... or to put it another way, as Larry Niven
did, any sufficiently advanced magic is indistinguishable from technology.

Since I don't actually believe in magic, as in "wishing makes it so", of
course it is technology. But most Python programmers aren't capable of, or
have any interest in, writing extension types in C or hacking the compiler. 


> This comes down to the original posting not defining a "magic name". It 
> does not mention what is so objectionable about __init__ and __repr__. I 
> am claiming they are not as "magical" as they may first appear.

I don't believe the Original Poster considers there is anything
objectionable about magic names like __init__ and __repr__, and even if he
does, I certainly don't.

Magic names in this context are those methods which have pre-defined
meanings to the compiler. That is mostly double-underscore methods like
__init__, but also methods like iterator.next(). If you ask yourself "What
methods do I have to over-ride to make a customized sub-class of a
built-in type?", the answer will include mostly magic names. If you ask
"What methods do I have to write for my class to work with Python
operators?", the answer will also include magic names like __add__,
__mul__, and __xor__.



-- 
Steven.