[Python-Dev] PEP 253: Subtyping Built-in Types
Guido van Rossum
guido@digicool.com
Sun, 22 Jul 2001 11:49:55 -0400
> A few people keep asking me for new features on those types, so
> I guess enabling this for Python 2.2 would be a real advantage for
> them.
>
> I still haven't found out how to solve the construction problem
> though (the base type is hard coded into various factory functions
> and methods)... the factory methods could use self.__class__
> to solve this, but the factory functions would need some different
> tweaking.
Using the new "classmethod" feature you can make the factory functions
class methods.
> > Yes, I've worked out a scheme to make this work, but I don't think
> > I've written it down anywhere yet. If your tp_new calls tp_alloc, and
> > your tp_dealloc calls tp_free, then a subtype can override tp_alloc
> > *and* tp_free and the right thing will happen. A subtype can also
> > *extend* tp_new and tp_dealloc. (tp_new and tp_dealloc are sort-of
> > each other's companions, and ditto for tp_alloc and tp_free.)
>
> So I will have to implement tp_free as well ?! Currently I have
> tp_new (which calls tp_alloc), tp_alloc, tp_init for the creation
> procedure and tp_dealloc (which does not call tp_free) for the
> finalization.
Yes, if your tp_new calls tp_alloc, your tp_dealloc should call
tp_free. Otherwise the user can override tp_alloc to use a different
heap, and tp_dealloc would mess up.
> I wonder whether it'd be a good idea to have a tp_del in there
> as well (the __del__ at C level) which is then called instead
> of tp_dealloc if set and which must call tp_dealloc if the
> instance is going to be deleted for good.
I've been thinking about this. I don't think that's quite the right
protocol; I don't want to complicate the DECREF macro any more. I
think that tp_dealloc must call tp_del and then decide whether to
proceed depending on the refcount.
> > > 2. In which order are the allocation/deallocation methods
> > > of subclass and base class called (if at all) and how
> > > does this depend on whether they are implemented or inherited ?
> >
> > Here's the scheme. A subtype's tp_new should call the base type's
> > tp_new, passing the subtype. The base class will call tp_alloc, which
> > is the subtype's version. Similar for deallocation: the subtype's
> > tp_dealloc calls the base type's tp_dealloc which calls tp_free which
> > is the subtype's version.
>
> Like this... ?
>
> subtype basetype
> ----------------------------------------------------
> Creation
>
> tp_new(subtype)
> -> tp_new(subtype) # calls tp_alloc & tp_init
>
> tp_alloc(subtype) <-
> -> tp_alloc(subtype)
Typically, the derved type's tp_alloc shouldn't call the base type's
tp_alloc -- tp_alloc is supposed to allocate memory for the actual
type, zero it, set the type pointer and reference count, and register
it with GC. Any other initializations that can't be left to tp_init
(which is optional) are tp_new's responsibility.
> tp_init(instance) <-
> -> tp_init(instance)
>
> Finalization
>
> (
> tp_delete(instance)
> -> tp_delete(instance) # calls tp_dealloc if
> # the instance should
> # be deleted
> )
> tp_dealloc(instance)
> -> tp_dealloc(instance) # calls tp_free
>
> tp_free(instance) <-
> -> tp_free(instance)
Likewise, tp_free needn't call the base tp_free.
> > > 3. How can I make attributes visible in subclassed types ?
> > >
> > > Even though I found out that I need to use the generic APIs
> > > PyObject_GenericGet|SetAttr() for the tp_get|setattro to
> > > make methods visible, attributes cannot be accessed (and this
> > > even though dir(instance) displays them).
> >
> > Strange. This should work. Probably something's subtly wrong in your
> > setup. Compare your code to xxsubtype.c.
>
> The xxsubtype doesn't define any attributes and neither do lists
> or dictionaries so there seems to be no precedent.
>
> In mxDateTime under Python 2.1, the tp_gettattr slot takes care of
> processing attribute lookup. Now to enable the dynamic goodies in
> Python 2.2, I have to provide the tp_getattro slot (and set it to
> the generic APIs mentioned above).
>
> Since tp_getattro override the tp_getattr slots, I have to rely
> on the generic APIs calling back to the tp_getattr slots to process
> the attributes which are not dynamically set by the user or a
> subclass. However, the new generic lookup APIs do not call the
> tp_getattr slot at all and thus the attributes which were "defined"
> by the tp_getattr in Python 2.1 are no longer visible.
>
> - How do I have to implement attribute lookup in Python 2.2
> for TP_BASETYPEs (methods are now magically handled by the tp_methods
> slot, there doesn't seem to be a corresponding feature for attributes
> though) ?
Ah, now I see the question. There's a tp_members slot, similar to the
tp_methods slot. The tp_members slot is a pointer to a
NULL-terminated array of the same form that you would pass to
PyMember_Get(). If your attributes require custom computation,
there's also a tp_getset slot which points to a NULL-terminated array
of 'struct getsetlist' items, which specify a name, a getter C
function, a setter C function, and a context void *. This means you
have to write a pair of (very simple) functions for each writable
attribute, or a single function per read-only attribute. (The context
pointer gives you a chance to share function implementations, but
I haven't found the need for this yet.)
Examples of all of these can be found in typeobject.c, look for
type_getsets and type_members.
> - Could the generic APIs perhaps fall back to tp_getattr to make
> the transition from classic types to base types a little easier ?
I'd rather not: that would prevent discovery of attributes supported
by the classic tp_getattr. The beauty of the new scheme is that *all*
attributes (methods and data) are listed in the type's __dict__.
--Guido van Rossum (home page: http://www.python.org/~guido/)