Issue with new-style classes and operators

[Bengt Richter]

On Mon, 25 Nov 2002 18:36:45 -0500, "Terry Reedy" <tjreedy at udel.edu> wrote:

>
>"Jan Decaluwe" <jan at jandecaluwe.com> wrote in message
>news:3DE215D1.D9A3FEF9 at jandecaluwe.com...
>> Hi:
>>
>> I'm confused by the following behavior of new-style classes
>> and operators:
>
>I think you have a right to be.
>
>>
>>     class MyInt(object):
>>         def __init__(self, val):
>>             self.val = val
>>         def __getattr__(self, attr):
>>             return getattr(self.val, attr)
>
>> >>> a = MyInt(3)
>> >>> a.__add__(4)
>> 7
>
>This works because '__add__' is seen as just an attribute name,
>nothing special, which is resolved because of __gettattr__ booting
>attribute lookup to self.val.  As far as the interpreter is concerned,
>this might as well have been a.whatever(4).
>
>> >>> a + 4
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in ?
>> TypeError: unsupported operand types for +: 'MyInt' and 'int'
>
>Lookup of special methods invoked by syntax other than explicit
>instance.attr appears to have been changed for new classes by stopping
>the lookup process before calling __getattr__.
>
>This appears to contradict the current new-class doc at
>http://python.org/2.2/descrintro.html
>
>"Note that while in general operator overloading works just as for
>classic classes, there are some differences. "
>
>Which are...?
>
>"(The biggest one is the lack of support for __coerce__; new-style
>classes should always use the new-style numeric API, which passes the
>other operand uncoerced to the __add__ and __radd__ methods, etc.) "
>
>what else?  but then
>
>"There's a new way of overriding attribute access.  The __getattr__
>hook, if defined, works the same way as it does for classic classes:
>it is only called if the regular way of searching for the attribute
>doesn't find it.  But you can now also override __getattribute__, a
>new operation that is called for *all* attribute references."
>
>But you found a difference in __getattr__.  Contrary to inplication of
>doc, __getattribute__ is also not called for a+4.  (Not a big loss
>since attempt to access any attribute, including .__dict__, causes
>infinite loop.)  I opened sf bug report 643841.
>
>>I don't think is can be the intention. Or can it?
>
>Beats me.  We'll see.
>
Adding some print statements:

 Python 2.2.2 (#37, Oct 14 2002, 17:02:34) [MSC 32 bit (Intel)] on win32
 Type "help", "copyright", "credits" or "license" for more information.
 >>> class MyClassicInt:
 ...     def __init__(self, val):
 ...         self.val = val
 ...     def __getattr__(self, attr):
 ...         attv = getattr(self.val, attr)
 ...         print '--(old)--\n%s => %s\n%s\n--' % (
 ...             attr, attv, attv.__doc__ )
 ...         return getattr(self.val, attr)
 ...
 >>> class MyInt(object):
 ...     def __init__(self, val):
 ...         self.val = val
 ...     def __getattr__(self, attr):
 ...         attv = getattr(self.val, attr)
 ...         print '--(new)--\n%s => %s\n%s\n--' % (
 ...             attr, attv, attv.__doc__ )
 ...         return getattr(self.val, attr)
 ...
 >>> ci = MyClassicInt(3)
 >>> ci.__add__(4)
 --(old)--
 __add__ => <method-wrapper object at 0x007D7BB0>
 x.__add__(y) <==> x+y
 --
 7
 >>> ci + 4
 --(old)--
 __coerce__ => <method-wrapper object at 0x007D7BB0>
 x.__coerce__(y) <==> coerce(x, y)
 --
 7
 >>> ni = MyInt(3)
 >>> ni.__add__(4)
 --(new)--
 __add__ => <method-wrapper object at 0x007D7570>
 x.__add__(y) <==> x+y
 --
 7
 >>> ni + 4
 Traceback (most recent call last):
   File "<stdin>", line 1, in ?
 TypeError: unsupported operand types for +: 'MyInt' and 'int'
 >>>

Either way, the code looks different for the two cases:

 >>> from ut.miscutil import disev
 >>> disev('ci.__add__(4)')
           0 SET_LINENO               0
           3 LOAD_NAME                0 (ci)
           6 LOAD_ATTR                1 (__add__)
           9 LOAD_CONST               0 (4)
          12 CALL_FUNCTION            1
          15 RETURN_VALUE
 >>> disev('ci + 4')
           0 SET_LINENO               0
           3 LOAD_NAME                0 (ci)
           6 LOAD_CONST               0 (4)
           9 BINARY_ADD
          10 RETURN_VALUE

But apparently BINARY_ADD reacts differently to being supplied
with classic vs new style object as an argument together with int.

Apparently, with a classic object it tries to get a __coerce__ method.

Note that what comes back from getting the __add__ attribute from an object
*instance* is a bound method that acts like a callable object, and since
the attribute is gotten from self.val, which is an integer, the associated object
is an integer, even though the __getattr__ was supposedly a method of MyInt.
Hence passing the bound method an integer to add naturally works. An this works
the same for both types, when the attribute access is explicit.

When the __coerce__ method is returned, it is similarly bound to the self.val
value which is an int instance. So the coerce works.

If there is an actual __add__ method in the class, it is found before __getattr__
is tried for the new-style class:

 >>> def add(x,y):
 ...     print 'adding %s + %s' % (`x`,`y`)
 ...     return x.val + y
 ...
 >>> MyInt.__add__ = add
 >>> ni + 4
 adding <__main__.MyInt object at 0x007D7BB0> + 4
 7

(Note that __repr__ is found without resort to __getattr__ above).

But binary add works differently for the old:
 >>> MyClassicInt.__add__ = add
 >>> ci + 5
 --(old)--
 __coerce__ => <method-wrapper object at 0x00839730>
 x.__coerce__(y) <==> coerce(x, y)
 --
 8

The coerce succeeded and MyInt.__add__ never got involved.


Of course, explicitly you get there:

 >>> ci.__add__(6)
 --(old)--
 __repr__ => <method-wrapper object at 0x0083AC90>
 x.__repr__() <==> repr(x)
 --
 adding 3 + 6
 9

And of course,  we mustn't be fooled by the names:

 >>> b = MyInt('bbb')
 >>> b.__add__('x')
 adding <__main__.MyInt object at 0x00839730> + 'x'
 'bbbx'
 >>> b + 'x'
 adding <__main__.MyInt object at 0x00839730> + 'x'
 'bbbx'
 >>> del MyInt.__add__
 >>> b + 'x'
 Traceback (most recent call last):
   File "<stdin>", line 1, in ?
 TypeError: unsupported operand types for +: 'MyInt' and 'str'
 >>> b.__add__('x')
 --(new)--
 __add__ => <method-wrapper object at 0x00839D80>
 x.__add__(y) <==> x+y
 --
 'bbbx'

Regards,
Bengt Richter