AttributeError in "with" statement (3.2.2)

[Steven D'Aprano]

On Fri, 16 Dec 2011 15:26:30 -0800, Ethan Furman wrote:

> Terry Reedy wrote:
>> On 12/16/2011 4:22 AM, Steven D'Aprano wrote:
[...]

> I think you two are in violent agreement as far as how Python is
> functioning, and the conflict is in the names given to the various
> pieces... I think a glossary would help (please correct me):
> 
> function:  callable code suite

Sure, why not?

Informally: a function is some object which can be called and returns a 
result. Hence we sometimes call str() and int() functions even though 
they are actually implemented as types. "Callable" would be better for 
such informal use, but newbies may not recognise the term.

To be more precise: a function is a particular type of callable object 
created by the def and lambda statements.


> method:  function that lives in a class

Informally, I'm happy with that definition, and use it myself. Under 99% 
of circumstances, it would be overly pedantic to describe spam in the 
following as anything other than a method:

class C:
    def spam(self): pass

Call it the duck-type principle: since spam quacks like a duck, we might 
as well call it a duck. If you only care about the easy cases, then spam 
is a method and we're done.

But as a formal definition, it fails on two fronts: functions that live 
inside classes are not actually methods, they are functions, and while 
methods are commonly found in classes, they can also be found in 
instances, or anywhere else for that matter.


Functions inside classes are still functions
--------------------------------------------

I said that since spam quacks like a method, we might as well call it a 
method. But it doesn't swim like a method: it's actually still a function 
with a remarkable power of mimicry.

There are places and times where the fact that spam is not actually a 
method comes through. The three most important examples in my opinion are:

  * inside the class suite, during class definition time, if you use
    or inspect spam, you will see it is a function and not a method;

  * if you bypass the getattr mechanism for looking up attributes, e.g.
    use C.__dict__['spam'] instead of C().spam, you will see that spam
    is stored in the dict as a function object, not a method object;

  * (Python 3 only) if use the standard getattr mechanism, but call it 
    on the class instead of an instance, you will get the function 
    object instead of a method.

So, to be precise: spam is actually a function. It is stored as a 
function inside C.__dict__, but when you retrieve it using C().spam 
Python automagically builds a method object wrapping spam and gives you 
that instead. This distinction is important, well, hardly ever, but on 
the rare occasion it is important, you need to know it.

One example of when it may be important: because methods are created as 
needed, you can't assume that identity checks will necessarily pass for 
methods: `inst.spam is inst.spam` may not be true.


Methods can be created outside of classes
-----------------------------------------

The normal way to get a method is by defining it inside a class. In Java, 
I believe that is the *only* way to get methods, but this is Python, and 
we have other options. Methods are first class objects, which means they 
have a type and a constructor, so if you can get hold of their type, you 
can make them by hand.

You can retrieve the method type two ways:

  * call type on a method you made the usual way: type(C().spam);

  * or import types and use types.MethodType.

Confusingly, the method type calls itself <instancemethod> but is 
exported via the types module as MethodType. Oh well.

MethodType requires two arguments, In Python 2, it accepts an optional 
third argument. Since Python 2 is the past, I'll only talk about the 
Python 3 version, where the signature might look something like this:

MethodType(callable, instance) => instancemethod object

This creates a wrapper around callable (usually, but not necessarily a 
function) and binds instance to that wrapper so the first argument 
(conventionally called self) can be automatically provided.

Another way to create methods is by calling the function object __get__ 
method by hand. That's what gets used in the normal C().spam case.

So you can create methods without putting them inside a class. One trick 
is that you can customise behaviour on a per-instance basis by overriding 
the normal method living inside the class with one which lives inside the 
instance:

class C:
    def spam(self):
        return "Spam spam spam, lovely SPAM!!!"

c = C()
c.spam = types.MethodType(lambda self: "Bloody vikings!", c)

Instance c now has a custom method which overrides C.spam.

[Aside #1: I think this was the original point of contention between 
Terry and I. For reasons I still don't understand, it seems to me that he 
denies that the per-instance method spam is actually a method, since it 
doesn't live inside the class.]

[Aside #2: this per-instance method trick may not work for double-
underscore methods like __exit__, since CPython and Jython at least skip 
the instance __dict__ when using dunder methods. In Python 2, it will 
work for classic classes, but they're gone in Python 3. If you go all the 
way back to the initial post in this thread, that was the OP's problem.]


> unbound method:  function that lives in a class

An unbound method is merely a method that doesn't know what instance to 
use, that is, it cannot automatically supply "self".

In Python 2, the easiest way to get an unbound method is by retrieving 
the method from the class instead of an instance:

  C.spam => returns method which is not bound ("unbound method")
  C().spam => returns method which is bound ("bound method")

Since unbound methods are more or less functionally identical to the 
underlying function (the only difference is they do a type-check on the 
argument), Python 3 gets rid of them and and just uses the function 
object itself.

So all the reasons for why methods are not *strictly* functions living 
inside classes apply equally to unbound methods, since unbound methods 
are merely a particular form of method.


> bound method:  callable wrapper around function that has been extracted
> from class that will supply the instance object to the function (note:
> Python does not save these, they are recreated at each lookup)

Bound methods are not *necessarily* extracted from a class, although they 
*usually* are. But wherever it comes from, a bound method knows which 
instance to provide as first argument ("self") to the function; otherwise 
we'd call it an unbound method.


> and here is where I think you two diverge:
> 
> instance method (Steven):  a bound method that has been saved into the
> instance __dict__ (no matter how created)

Certainly not! If I gave that impression, I apologize.

Every thing I have said above refers to *instance* methods, or just 
"methods" for lazy people. Instance methods get their name from the fact 
that they supply the instance as first argument to the function ("self"). 
If they instead supply the class, we call them a class method; if they 
don't supply anything, we call them a static method (much to the 
confusion of Java programmers, who use the same names for different 
things).

You can even create your own method types:

http://code.activestate.com/recipes/577030-dualmethod-descriptor/




-- 
Steven