Working with method-wrapper objects

[Bengt Richter]

On Thu, 28 Apr 2005 11:33:04 +0200, "Dr. Peer Griebel" <griebel at konzept-is.de> wrote:

>Peer Dr. Griebel wrote:
[..]
>> Why has [].__str__ a different type than object.__str__?
>> Why is object.__str__ a routine while object().__str__ not?
Why wouldn't you expect different types? Those are different expressions.

If they were types and instances of your own making,
like class Foo(object) and Foo(), wouldn't you expect
a difference between Foo.method and Foo().method ?

There are a lot of tricky details in the way a method function
is found and what is done with it depending on how it was found.
Descriptors play a huge role in this, so if you are really interested
you must read about those. Note that in order to find an attribute,
you first have to look for the attribute '__getattribute__' to make sure
nothing is overriding the normal function. But what is normal? If you
look for '__getattribute__' anyway, maybe just let that search find
what it will, which will be type.__getattribute__


When you write obj.attr you are kicking off a search for an attribute
named 'attr' starting the search in type(obj).__dict__ and the rest of
the __dict__'s of the base classes in mro order. If the attribute object
has a __get__ method, it is a descriptor, say descr, and the attribute value
finally returned will be whatever the descr returns on being called like
descr.__get__(self, type(self)). So when you look at something like [].__str__
think of it in terms of obj.attr. What kind of obj is []? (it's a list instance,
just as you'd get from list(), analogous to object(), which produces and object instance).

To find the '__str__' attribute, or any attribute, you start in type(obj).mro()[0].__dict__
and see if the immediate base has a method or other descriptor of that name.
If not, you go on through the list with type(obj).mro()[1].__dict__ etc. E.g., for an obj
that is a list instance:

 >>> obj = []
 >>> type(obj).mro()
 [<type 'list'>, <type 'object'>]
 >>> type(obj).mro()[0]
 <type 'list'>
 >>> type(obj).mro()[0].__dict__
 <dictproxy object at 0x02E8159C>
 >>> type(obj).mro()[0].__dict__['__str__']
 Traceback (most recent call last):
   File "<stdin>", line 1, in ?
 KeyError: '__str__'
 >>> type(obj).mro()[1]
 <type 'object'>
 >>> type(obj).mro()[1].__dict__
 <dictproxy object at 0x02E81794>
 >>> type(obj).mro()[1].__dict__['__str__']
 <slot wrapper '__str__' of 'object' objects>
 >>> type(obj).mro()[1].__dict__['__str__']()
 Traceback (most recent call last):
   File "<stdin>", line 1, in ?
 TypeError: descriptor '__str__' of 'object' object needs an argument
 >>> type(obj).mro()[1].__dict__['__str__'](obj)
 '[]'

 >>> obj = [1, 2, '(specific instance)']
 >>> type(obj).mro()[1].__dict__['__str__'](obj)
 "[1, 2, '(specific instance)']"

Now suppose obj is list instead of [], analogous to object instead of object():

 >>> obj = list
 >>> type(obj).mro()
 Traceback (most recent call last):
   File "<stdin>", line 1, in ?
 TypeError: descriptor 'mro' of 'type' object needs an argument
 >>> type(obj).mro(obj)
 [<type 'list'>, <type 'object'>]
 >>> type(obj).mro(obj)[0].__dict__['__str__']
 Traceback (most recent call last):
   File "<stdin>", line 1, in ?
 KeyError: '__str__'
 >>> type(obj).mro(obj)[1]
 <type 'object'>
 >>> type(obj).mro(obj)[1].__dict__['__str__']
 <slot wrapper '__str__' of 'object' objects>
 >>> type(obj).mro(obj)[1].__dict__['__str__']()
 Traceback (most recent call last):
   File "<stdin>", line 1, in ?
 TypeError: descriptor '__str__' of 'object' object needs an argument
 >>> type(obj).mro(obj)[1].__dict__['__str__'](obj)
 "<type 'list'>"

Note that the error message above is saying __str__ is a descriptor, so let's see what methods it has:

 >>> dir(type(obj).mro(obj)[1].__dict__['__str__'])
 ['__call__', '__class__', '__delattr__', '__doc__', '__get__', '__getattribute__', '__hash__', '
 __init__', '__name__', '__new__', '__objclass__', '__reduce__', '__reduce_ex__', '__repr__', '__
 setattr__', '__str__']

It has its own __str__ to show itself:

 >>> type(obj).mro(obj)[1].__dict__['__str__'].__str__()
 "<slot wrapper '__str__' of 'object' objects>"

which is what we get if we let the interactive evaluator show it to us:

 >>> type(obj).mro(obj)[1].__dict__['__str__']
 <slot wrapper '__str__' of 'object' objects>

But we saw it has a __get__ method, and that is acting a lot like the __get__ method
of an ordinary method, e.g., if we give it the object, we get a callable that acts
like a bound method:

 >>> bmlike = type(obj).mro(obj)[1].__dict__['__str__'].__get__(obj)
 >>> bmlike
 <method-wrapper object at 0x02EF196C>
 >>> bmlike()
 "<type 'list'>"

 >>> obj
 <type 'list'>

Note that:

 >>> type(list).mro(list)[1].__dict__['__str__'] is type(object).mro(object)[0].__dict__['__str__']
 True

which is because

 >>> type(list).mro(list)[1] is type(object).mro(object)[0]
 True

I.e., '__str__' is found in the same base class either way

 >>> type(list).mro(list)[1]  , type(object).mro(object)[0]
 (<type 'object'>, <type 'object'>)

If you want to twist your mind some more, note that __getattribute__ is itself an attribute
involved in finding attributes, and __dict__ may not be an ordinary attribute either ;-)

>> 
>> And one again my question: Can I extract some more information about a
>> methed-wrapper object. E.g. can I somehow determine the arg spec?
I believe it is a descriptor, and as such has at least a __get__ method,
and to be callable itself, a __call__ method. I think the latter being a method wrapper
probably effectively does wrapper.__get__(obj, type(obj))(*args, **kw)
when you do wrapper.__call__(obj, *args, **kw)), which you do when you
do wrapper(obj, your_args, your_keywords)

E.g., taking an ordinary class may show it more clearly:

 >>> class Foo(object):
 ...    def m(self, *args, **kw): return self, args, kw
 ...
 >>> foo = Foo()
 >>> type(foo).mro()
 [<class '__main__.Foo'>, <type 'object'>]
 >>> type(foo).mro()[0].__dict__
 <dictproxy object at 0x02E8147C>
 >>> type(foo).mro()[0].__dict__['m']
 <function m at 0x02EE8D4C>
 >>> type(foo).mro()[0].__dict__['m'].__get__
 <method-wrapper object at 0x02EF1A6C>
 >>> type(foo).mro()[0].__dict__['m'].__get__(foo, type(foo))
 <bound method Foo.m of <__main__.Foo object at 0x02EF1BAC>>
 >>> type(foo).mro()[0].__dict__['m'].__get__(foo, type(foo)).__call__
 <method-wrapper object at 0x02EF1CAC>
 >>> type(foo).mro()[0].__dict__['m'].__get__(foo, type(foo)).__call__('arg', k='keyword')
 (<__main__.Foo object at 0x02EF1BAC>, ('arg',), {'k': 'keyword'})

Now you can try the same with inherited methods like __str__ that are built-in, e.g., substitute
foo = [1, 2, 'this is a particular list instance'] and note similarities, if you realize what is a
dscriptor and follow the same kind of logic to the final method call via __call__

 >>> foo = [1, 2, 'this is a particular list instance']
 >>> type(foo).mro()
 [<type 'list'>, <type 'object'>]
 >>> type(foo).mro()[0].__dict__
 <dictproxy object at 0x02E813EC>
 >>> type(foo).mro()[0].__dict__['__str__']
 Traceback (most recent call last):
   File "<stdin>", line 1, in ?
 KeyError: '__str__'
 >>> type(foo).mro()[1].__dict__['__str__']
 <slot wrapper '__str__' of 'object' objects>
 >>> type(foo).mro()[1].__dict__['__str__'].__get__
 <method-wrapper object at 0x02EF1AEC>
 >>> type(foo).mro()[1].__dict__['__str__'].__get__(foo, type(foo))
 <method-wrapper object at 0x02EF1BAC>
 >>> type(foo).mro()[1].__dict__['__str__'].__get__(foo, type(foo)).__call__
 <method-wrapper object at 0x02EF1D2C>
 >>> type(foo).mro()[1].__dict__['__str__'].__get__(foo, type(foo)).__call__()
 "[1, 2, 'this is a particular list instance']"

>> 
>> Thanks
>>   Peer
>
>Isn't there anybody who has something to say about the issue?
>
Ultimately you have to go into the code of ceval.c and object.c and typeobject.c
etc to see what is really happening.

>I think it's not only a problem with inspect. It is aproblem about old 
>style classes vs. new style classes. It seems that the support for new 
>style classes is not complete (yet).
>
>Some more investigation shows that also the module "types" is not 
>universally usable. E.g. InstanceType is only usabel for instances of 
>old classes. How do I test for instances of new classes?

Not certain, but hasattr(inst, '__new__') might be close. And mnemonic ;-)

Regards,
Bengt Richter