AttributeError in "with" statement (3.2.2)

[Steven D'Aprano]

On Wed, 14 Dec 2011 18:13:36 -0500, Terry Reedy wrote:

> On 12/14/2011 3:01 AM, Steven D'Aprano wrote:
>> On Wed, 14 Dec 2011 01:29:13 -0500, Terry Reedy wrote:
>>
>>> To complement what Eric says below: The with statement is looking for
>>> an instance *method*, which by definition, is a function attribute of
>>> a *class* (the class of the context manager) that takes an instance of
>>> the class as its first parameter.
>>
>> I'm not sure that is correct... I don't think that there is anything
>> "by definition" about where methods live.
> 
>  From the Python glossary:
> "method: A function which is defined inside a class body."
> 
> That is actually a bit too narrow, as a function can be added to the
> class after it is defined. But the point then is that it is treated as
> if defined inside the class body.

First off, let me preface this by saying that I'm not necessarily saying 
that the above glossary definition needs to be changed. For most 
purposes, it is fine, since *nearly always* methods are created as 
functions defined inside the class body. But it needs to be understood in 
context as a simplified, hand-wavy definition which covers 99% of the 
common cases, and not a precise, definitive technical definition.

To give an analogy, it is like defining mammals as "hairy animals which 
give birth to live young", which is correct for all mammals except for 
monotremes, which are mammals which lay eggs.

So I'm happy for the glossary entry to stay as is, because complicating 
it would confuse the average coder more than it would educate them.

But the definition as given is strictly wrong, because it fails to 
capture the essence of what distinguishes a method from other objects. It 
mistakes *how* you normally create a method for *what* a method is, a 
little like defining "a hamburger is a foodstuff you get from McDonalds 
by giving them money".

Here are three ways that the definition fails:

(1) You can create a function inside a class, and it remains a function, 
so long as the class constructor (metaclass) never gets to build a method 
object from the function. It is easy to do: just hide it inside a wrapper 
object.

>>> class FunctionInsideClass(object):
...     def func(x, y):  # define a function inside a class
...             return x + 2*y
...     print(type(func))  # confirm is actually is a function
...     attr = (func,)  # hide it from the metaclass
...     del func
... 
<class 'function'>
>>> print(type(FunctionInsideClass.attr[0]))
<class 'function'>


(2) Instead of hiding the function from the metaclass, you can change the 
metaclass to something which doesn't make methods out of functions. I 
won't show an example, because it's tricky to get right (or at least *I* 
find metaclasses tricky).


(3) So the definition is too broad: you can have functions defined inside 
classes that are not methods. But it is also too narrow: you can have 
methods outside of classes. I'm not talking about bound and unbound 
methods, but about *creating* the method from scratch outside of a class. 
When you call the method constructor directly, you can create a method 
from a function defined outside of a class.

>>> def func(self, a):
...     return self + a
... 
>>> type(func)  # Definitely a function.
<class 'function'>
>>> obj = types.MethodType(func, 42)
>>> obj(23)  # Works as expected.
65
>>> type(obj)
<class 'method'>

So there's a method which has never been inside a class, and couldn't 
even if you tried: int is a built-in immutable type.


So what are methods? In Python, methods are wrappers around functions 
which automatically pass the instance to the inner function object. Under 
normal circumstances, you create methods by declaring functions inside a 
class, but that's not the only way to create methods, and it is not the 
only place they can be found.

Note that this definition can easily be adapted to describe class methods 
and static methods too: class methods are wrappers around functions that 
automatically pass the class to the inner function, and static methods 
are wrappers which don't pass any automatic arguments.


>> Particularly not in Python where
>> instance methods can be attributes of the instance itself.
> 
> This is access, not definition or actual location. 

Not so. In the example I gave, the method *really is* inside the 
instance, stored in the instance __dict__ and not the class __dict__.


> The glossary entry go
> on to say: "If called as an attribute of an instance of that class, the
> method will get the instance object as its first argument (which is
> usually called self)." This does *not* happen if a callable is found in
> the instance-specific dictionary.

That's right. Methods are special not because of where they are, but 
because of what they are.


> An instance method is a function
> (callable) attribute of a class that gets special treatment when
> accessed (indirectly) through an instance of that class (or subclass
> thereof).

Methods aren't functions at all, not in the isinstance sense. They are 
functions in the sense that any callable object is a function, i.e. they 
are callable sub-routines. But they're not functions in the sense of 
being instances of type "function". They are wrappers around functions.

Other languages may do things differently, but that's what Python does. 
You can even retrieve the function object from the wrapper:

>>> obj.__func__
<function func at 0xb70fd4ec>


 
>>>>> class Test(object):
>> ...     def method(self):
>> ...             print("This method is an attribute of the class.") ...
>>>>> t = Test()
>>>>> t.method()
>> This method is an attribute of the class.
> 
> The bound method t.method is an instance the class exposed as
> types.MethodType. In other words, isinstance(t.method, types.MethodType)
> == True
> 
>>>>> import types
>>>>> t.method = types.MethodType(
>> ...     lambda self: print(
>> ...     "This method is an attribute of the instance."), t)
> 
> Calling any old fruit an apple does not make it one. Calling any old
> function a method does not make it one.

I'm not *calling* a function a method, I'm *creating* a method from a 
function object. There is no difference between a method created with 
types.MethodType and a method automagically created inside a class except 
for the location where the object is stored. The location is irrelevant.


> 'types.MethodType' is the exposed name of the class the interpreter uses
> to create bound methods from a method and an instance of the class
> containing the method. I believe the interpreter does an isinstance
> check, but it must do that before calling the class, and not in the
> bound method constructor itself. In any case, a bound method is not a
> method.

That's an astonishing statement. What is your evidence for this amazing 
claim that bound methods are not actually methods? What are they then, if 
not methods?


> In this case, the result is not really even a bound method, as the
> function argument is not a method, so we cannot even ask if the second
> arg is an instance of the function class container.  MethodType is a
> special case of functools.partial, which was added later. You could have
> used the latter to the same effect. Or you could have used any old
> function that printed the same thing.

Good grief. Is it really your argument that the types.MethodType isn't 
actually the type of methods, but a fake that lies about returning 
methods? Well, that's easy enough to test:

>>> class K(object):
...     def f(self):
...             pass
... 
>>> k = K()
>>> type(k.f) is types.MethodType
True


Unless you are going to accuse me of faking the interpreter output 
(perhaps I monkey-patched the type built-in?) that is definitive proof 
that types.MethodType is not fake, it actually is the method type used in 
classes.


> There is no relation between the object passed as the second arg of
> MethodType and what you do with the resulting callable. Either 't' could
> be something else. See below.
> 
>>>>> t.method()
>> This method is an attribute of the instance.
> 
> Yes, the callable (which is not a method) is (currently) an attribute of
> the instance. But that is irrelevant to its operation. t.method is just
> a callable, in particular, a pseudo bound method, not a method. It is
> *not* supplying the instance it is called on as the first parameter of
> the callable. 

Of course it is. It is because I made it to be that way.

I encourage you to experiment with Python's introspection tools, perhaps 
put a few calls to print inside a "pseudo bound method" (your words, 
there's nothing pseudo about it) and satisfy yourself that the instance 
passed is the same instance as it is called from *under the circumstances 
shown*.


> The arguemnt (which is not used) has already been
> supplied. These produce the same output:
> 
> class B: pass
> b = B()
> b.method = t.method
> b.method()

Yes. So what? You can take a bound object and attach it to any other 
object and see the same results -- this doesn't mean it isn't a bound 
object. That's how bound objects work! Your objection fails because any 
method will work the same way:

>>> class K: pass
... 
>>> class Example:
...     def foo(self):
...             print(self)
... 
>>> x = Example()
>>> k = K()
>>> k.attr = x.foo
>>> k.attr()
<__main__.Example object at 0xb70fcc0c>

No trickery with types.MethodType, no metaclass magic, no sleight of 
hand, just stock standard Python behaviour.



-- 
Steven