Unexpected behavior of read only attributes and super

[Steven Bethard]

Samuel M. Smith wrote:
> If you would care to elaborate on the how the lookup differs with  
> method descriptor it would be most appreciated.

For the more authoritative guide, see:
     http://users.rcn.com/python/download/Descriptor.htm

The basic idea is that a descriptor is an object that sits at the class 
level, and redefines how some attribute accesses work.  Consider a 
simple example:

 >>> class D(object):
...     def __get__(self, obj, objtype=None):
...         if obj is None:
...             return 'called from class %r' % objtype
...         else:
...             return 'called from instance %r' % obj
...
 >>> class C(object):
... 	d = D()
...
 >>> C.d
"called from class <class '__main__.C'>"
 >>> C().d
'called from instance <__main__.C object at 0x00E73A30>'

As you can see, instances of the D class, when used as class attributes, 
can tell whether they're being called by the class or the instance. 
This means that descriptors with a __get__ method defined can do just 
about anything on an attribute access.

Note that all functions in Python are descriptors, and they use the 
__get__ method to return either an unbound method or a bound method, 
depending on whether they were called from the type or the instance:

 >>> def f(x):
...     return x*2
...
 >>> class C(object):
...     func = f
...
 >>> f
<function f at 0x00E69C30>
 >>> C.func
<unbound method C.f>
 >>> C().func
<bound method C.f of <__main__.C object at 0x00E73B50>>

> This might help explain why it is that when I define __slots__, the  
> behavior when writing an attribute is different

Yes.  Defining __slots__ basically tells the class to create descriptors 
for each name in the list.  So, for example:

>  >>> class C(dict):
> ...     __slots__ = ['a','b']
> ...

Creates two descriptors that are attributes of class C: one named "a" 
and one named "b".

> Now the behavior is different for class variables and methods when  
> slots defined versus when slots is not defined.
> 
>  >>> c.__iter__ = 4
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> AttributeError: 'C' object attribute '__iter__' is read-only

Here, Python is trying to set the "__iter__" attribute of the object. 
Since you defined __slots__, it tells you that it can't.  So it never 
even looks at the type.

>  >>> super(C,c).__iter__ = 4
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> TypeError: 'super' object has only read-only attributes (assign  to 
> .__iter__)

In this case, you explicitly request the superclass, so you get the same 
error as before because you bypass the __slots__, which are defined for 
instances of C, not for instances of the superclass, dict.

> Then why wasn't __class__ added to c.__dict__ ? Looks like namespace  
> searching to me.

No, as you conclude later, __class__ is special, so you can still assign 
to __class__ even when __slots__ is defined because it's not considered 
a normal attribute.  But note that __class__ is an *instance* attribute, 
not a class attribute, so "c.__class__ = C" changes the class of that 
single instance, and makes no change to the type:

 >>> class C(object):
...     pass
...
 >>> class D(C):
...     pass
...
 >>> c1 = C()
 >>> c2 = C()
 >>> C, c1, c2
(<class '__main__.C'>, <__main__.C object at 0x00E73A30>, <__main__.C 
object at 0x00E73210>)
 >>> c1.__class__ = D
 >>> C, c1, c2
(<class '__main__.C'>, <__main__.D object at 0x00E73A30>, <__main__.C 
object at 0x00E73210>)

So no, even with __class__, you're only assigning to the instance, and 
so Python's not searching any additional namespaces.

> now with slots defined
> 
>  >>> class C(dict):
> ...     __slots__ = ['b']
> ...     a = 0
> ...
>  >>> c = C()
>  >>> c.a
> 0
>  >>> c.a = 4
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> AttributeError: 'C' object attribute 'a' is read-only
>  >>> C.a = 5
>  >>> c.a
> 5
> 
> So the rule is that when __slots__ is defined class variables become  
> read only.

That's not quite right.  As you show above, class variables are still 
modifiable from the class object.  But yes, defining __slots__ means 
that, from an instance, you can only modify the attributes defined in 
__slots__.

> What if the class variable is included in __slots__
> 
>  >>> class C(dict):
> ...     __slots__ = ['b']
> ...     b = 1
> ...
>  >>> c = C()
>  >>> c.b
> 1
>  >>> c.b = 2
> Traceback (most recent call last):
>   File "<stdin>", line 1, in ?
> AttributeError: 'C' object attribute 'b' is read-only
> 
> So even though b is in slots I still can't create an instance  variable 
> by that name and shadow the class variable.

Yes, this behavior is documented:
     http://docs.python.org/ref/slots.html

"""
__slots__ are implemented at the class level by creating descriptors 
(3.3.2) for each variable name. As a result, class attributes cannot be 
used to set default values for instance variables defined by __slots__; 
otherwise, the class attribute would overwrite the descriptor assignment.
"""

The documentation isn't great, I'll agree, but the result is basically 
that if you combine __slots__ with class attributes of the same name, 
you're asking for trouble because your 'b' class attribute and your 'b' 
slot both reside at the class level.  IMO, the manual should probably 
explicitly say that the behavior of such a class is undefined.

> It feels like the implementation of slots is half baked.

I wouldn't go that far.  Once you understand that __slots__ reside as 
descriptors at the class level, you can see why most of the problems 
arise.  That said, I've never had the need to use __slots__ in any real 
world code.

> Because slots break this paradigm then at the very least the error  
> messages should point out that this object
> is using slots "so beware".
> 
> For example I would prefer something like the following
> 
> c.a
> AttributeError: Slot 'a' not yet assigned
> 
> c.c
> AttributeError: No slot named 'c' on instance
> 
> c.c = 4
> AttributeError: No slot named 'c' on instance
> 
> if change rule to not access class from instance when slots define
> c.__iter__ = 4
> AttributeError: No slot named '__iter__' on instance
> 
> or with current behavior
> AttributeError: No slot name '__iter__' class 'C' object attribute  
> '__iter__' is read-only
> 
> super(C,c).__iter__ = 4
> TypeError: 'super' object has only read-only attributes (assign  to 
> .__iter__)

These seem like pretty reasonable requests.  I would suggest adding them 
as a feature request:
     http://sourceforge.net/tracker/?group_id=5470&atid=355470
The one thing I don't know is how hard it is to produce these messages 
-- I don't know the C-level code for __slots__ at all.  Hopefully 
someone else will!

STeVe