magic names in python

[Lenard Lindstrom]

Steven D'Aprano wrote:
> On Mon, 04 Jun 2007 22:19:35 +0000, Lenard Lindstrom wrote:
> 
>> What is "magic" about __init__ and __repr__? They are identifiers just 
>> like "foo" or "JustAnotherClass". They have no special meaning to the 
>> Python compiler. The leading and trailing double underscores represent 
>> no special incantation. It is just a naming convention.
> 
> That's not quite true, as you point out:

Disassemble the code object of a class statement. "__init__" is just an 
identifier, a Python name. So there is no "magic" here.

> 
>> So a number of method names like __init__ and __repr__ have a 
>> pre-defined usage. 
> 
> That makes them magic, in the best possible way.

I was careful to use the word "usage". Maybe I should have used 
"protocol" or "specification" here. A method named "write" is understood 
to have a particular definition and purpose. Is it "magic"? These 
methods are simply callbacks. And yes, these callbacks make Python flexible.

> 
> 
>> In every other respect they are just normal methods. 
> 
> That is *almost* true. Or, to put it another way, that is wrong. Some of
> the double-underscore magic methods are automatically called on the class
> instead of the instance, bypassing normal inheritance. 
> 

They exist as Python functions in the class dictionary. They have no 
hidden flags that distinguish them from other class level function 
attributes.

> 
>>>> import new
>>>> class Test(object):
> ....     def foo(self):
> ....             return "foo called from the class"
> ....     def __str__(self):
> ....             return "__str__ called from the class"
> ....
>>>> obj = Test()
>>>> obj.foo()
> 'foo called from the class'
>>>> str(obj) # calls obj.__str__
> '__str__ called from the class'
> 
> Now see what happens when we add methods to the instance.
> 
>>>> obj.foo = new.instancemethod(
> .... lambda self: "foo called from the instance", obj, Test)
>>>> obj.foo()
> 'foo called from the instance'
>>>> obj.__class__.foo(obj) # Check the method in the class is still there.
> 'foo called from the class'
> 
> So calling a random method like foo() goes through the usual procedure:
> check the instance, then check the class. Now let's try it with a magic
> method.
> 
> 
>>>> obj.__str__ = new.instancemethod(
> .... lambda self: "__str__ called from the instance", obj, Test)
>>>> obj.__str__()
> '__str__ called from the instance'
>>>> str(obj) # calls obj.__str__() maybe?
> '__str__ called from the class'
> 

Yes, method lookup for an operation differs from attribute lookup in 
new-style classes. But it is a property of new-style classes, not the 
methods themselves. For instance, classic classes behave differently:

 >>> class Test:
	def __str__(self):
		return "__str__ called from the class"

	
 >>> obj = Test()
 >>> obj.__str__()
'__str__ called from the class'
 >>> str(obj)
'__str__ called from the class'
 >>> obj.__str__ = new.instancemethod(
	lambda self: "__str__ called from the instance", obj, Test)
 >>> str(obj)
'__str__ called from the instance'
 >>> obj.__class__.__str__(obj)
'__str__ called from the class'


I do admit that the special methods are given special treatment by the 
type and ClassType types to ensure they are called by their 
corresponding operations. But this special treatment is restricted to 
the types themselves. In CPython an extension type can be written from 
scratch that treats special methods exactly as a new-style class does. 
Or an extension type can implement a completely novel approach. The 
point is, at some level, the machinery which defines special method 
"magic" is accessible to the programmer. So is it really "magic" or 
advanced technology?

This comes down to the original posting not defining a "magic name". It 
does not mention what is so objectionable about __init__ and __repr__. I 
am claiming they are not as "magical" as they may first appear.

--
Lenard Lindstrom
<len-l at telus.net>