can't set attributes of built-in/extension type

[Steve Holden]

Steve Holden wrote:
> Neal Becker wrote:
>> Steve Holden wrote:
>>
>>> Neal Becker wrote:
>>>> Steve Holden wrote:
>>>>
>>>>> Neal Becker wrote:
>>>>>> 7stud wrote:
>>>>>>
>>>>>>> On Feb 21, 11:19 am, Neal Becker <ndbeck... at gmail.com> wrote:
>>>>>>>> I'm working on a simple extension.  Following the classic 'noddy'
>>>>>>>> example.
>>>>>>>>
>>>>>>>> In [15]: cmplx_int32
>>>>>>>> Out[15]: <type 'numpy.cmplx_int32'>
>>>>>>>>
>>>>>>>> Now I want to add an attribute to this type.  More precisely, I want
>>>>>>>> a class attribute.
>>>>>>>>
>>>>>>>> cmplx_int32.test = 0
>>>>>>>> ---------------------------------------------------------------------------
>>>>>>>> TypeError                                 Traceback (most recent call
>>>>>>>> last)
>>>>>>>>
>>>>>>>> /home/nbecker/numpy/<ipython console> in <module>()
>>>>>>>>
>>>>>>>> TypeError: can't set attributes of built-in/extension
>>>>>>>> type 'numpy.cmplx_int32'
>>>>>>>>
>>>>>>>> What am I missing?
>>>>>>> class Dog(object):
>>>>>>>     def __setattr__(self, attr, val):
>>>>>>>         print "TypeError: can't set attributes of built-in/extension"
>>>>>>>         print "type 'Dog.cmplx_int32'"
>>>>>>>
>>>>>>> d = Dog()
>>>>>>> d.test = 0
>>>>>>>
>>>>>>> --output:--
>>>>>>> TypeError: can't set attributes of built-in/extension
>>>>>>> type 'Dog.cmplx_int32'
>>>>>> Not quite, I'm setting a class attribute, not an attribute on an
>>>>>> instance.
>>>>>>
>>>>> Quite. The problem is that extension types' attributes are determined by
>>>>> the layout of the object's slots and forever fixed in the C code that
>>>>> implements them: the slots can't be extended, so there's no way to add
>>>>> attributes. This is an efficiency feature: it would be *extremely* slow
>>>>> to look up the basic types' attributes using late-binding (it would also
>>>>> change the nature of the language somewhat, making it more like Ruby or
>>>>> Self).
>>>>>
>>>>> So the reason you can't do what you want to is the same reason why you
>>>>> can't add attribute to the built-in types (which are, of course, clearly
>>>>> mentioned in the error message).
>>>>>
>>>>>  >>> object.anyoldname = "You lose!"
>>>>> Traceback (most recent call last):
>>>>>    File "<stdin>", line 1, in <module>
>>>>> TypeError: can't set attributes of built-in/extension type 'object'
>>>>>  >>>
>>>>>
>>>>> If you look in typeobject.c you'll find this error message occurs when
>>>>> the object's type isn't a PyHeapTypeObject (in other words, if it's one
>>>>> of the built-in or extension types).
>>>>>
>>>> Thanks, but I'm a bit confused.  After reading in my "Python in a
>>>> Nutshell", I found that if after calling PyReady on my type object, if I
>>>> use PyDict_SetItemString (my_type_obj.tp_dict,)
>>>>
>>>> That seems to work fine (which isn't exactly what it said in the Nutshell
>>>> book, but close).
>>>>
>> I wanted to add an attribute to my type.
>> Specifically, my type object is a static cmplx_int32_scalar_obj.
>>
>> After calling PyType_Ready (&cmplx_int32_scalar_obj), then I did
>> PyDict_SetItemString (cmplx_int32_scalar_obj.tp_dict, "dtype", (PyObject*)d1);
>>
>> Now my type has the property:
>> cmplx_int32.dtype
>> dtype('cmplx_int32')
>>
>> Now, I do see that I still can't set it:
>>
>> cmplx_int32.dtype = 2
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>> TypeError: can't set attributes of built-in/extension type 'numpy.cmplx_int32'
>>
>> In this case, I don't need to.
>>
>> But I still don't know why I can have a python class and set class or instance
>> attributes as I like, but this type acts differently.  What would I need to 
>> do if I did want to allow arbitrary attributes to be set/added to my type?
>>
> I believe it's because PyType_Ready(), among its many other duties,
> calls mro_internal() on the type. It seems obvious that one would want
> to optimize the MRO by not allowing modifications. Yet in C it is
> possible, as you point out, to do so. Hmm ...
> 
> I'll let you know if I come to any conclusion - a query to python-dev
> would probably get an answer, but surely someone on this list knows already?
> 
> [Left this as a draft for a while to mull it over].
> 
> After further consideration I have concluded (without further scrutiny 
> of the source) that it's because the method slots in C-implemented types 
> are pointers to C functions not to Python functions. Would this make sense?
> 
Just to close this one off, Neal wrote to python-dev and got the 
following reply from Guido himself.

> On Sat, Feb 23, 2008 at 4:55 PM, Neal Becker <ndbecker2 at gmail.com> wrote:
>> There is some discussion on this subject, archived here:
>>  http://permalink.gmane.org/gmane.comp.python.general/560661
>>
>>  I wonder if anyone could shed some light on this subject?
>>
>>  (Or, help me understand, what is the difference between a type that I create
>>  using python C api and a python class?)
> 
> This is prohibited intentionally to prevent accidental fatal changes
> to built-in types (fatal to parts of the code that you never though
> of). Also, it is done to prevent the changes to affect different
> interpreters residing in the address space, since built-in types
> (unlike user-defined classes) are shared between all such
> interpreters.
> 
> -- 
> --Guido van Rossum (home page: http://www.python.org/~guido/)

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/