[Python-Dev] PEP 253: Subtyping Built-in Types

[M.-A. Lemburg]

Guido van Rossum wrote:
> 
> > A few people keep asking me for new features on those types, so
> > I guess enabling this for Python 2.2 would be a real advantage for
> > them.
> >
> > I still haven't found out how to solve the construction problem
> > though (the base type is hard coded into various factory functions
> > and methods)... the factory methods could use self.__class__
> > to solve this, but the factory functions would need some different
> > tweaking.
> 
> Using the new "classmethod" feature you can make the factory functions
> class methods.

Hmm, I don't like these class methods, but it would probably
help with the problem...

from mx.DateTime import DateTime

dt1 = DateTime(2001,1,16)
dt2 = DateTime.From("16. Januar 2001")

Still looks silly to me... (I don't like these class methods).
 
> > > Yes, I've worked out a scheme to make this work, but I don't think
> > > I've written it down anywhere yet.  If your tp_new calls tp_alloc, and
> > > your tp_dealloc calls tp_free, then a subtype can override tp_alloc
> > > *and* tp_free and the right thing will happen.  A subtype can also
> > > *extend* tp_new and tp_dealloc.  (tp_new and tp_dealloc are sort-of
> > > each other's companions, and ditto for tp_alloc and tp_free.)
> >
> > So I will have to implement tp_free as well ?! Currently I have
> > tp_new (which calls tp_alloc), tp_alloc, tp_init for the creation
> > procedure and tp_dealloc (which does not call tp_free) for the
> > finalization.
> 
> Yes, if your tp_new calls tp_alloc, your tp_dealloc should call
> tp_free.  Otherwise the user can override tp_alloc to use a different
> heap, and tp_dealloc would mess up.

Ok.
 
> > I wonder whether it'd be a good idea to have a tp_del in there
> > as well (the __del__ at C level) which is then called instead
> > of tp_dealloc if set and which must call tp_dealloc if the
> > instance is going to be deleted for good.
> 
> I've been thinking about this.  I don't think that's quite the right
> protocol; I don't want to complicate the DECREF macro any more.  I
> think that tp_dealloc must call tp_del and then decide whether to
> proceed depending on the refcount.

Have you tried to move the decref action into a separate function
(which is only called in case the refcount reaches 0) ? I think
that this could in fact enhance the overall performance since 
the compiler can then decide whether or not to inline the relevant
code.

I wonder what the impact would be...
 
> > > > 2. In which order are the allocation/deallocation methods
> > > > of subclass and base class called (if at all) and how
> > > > does this depend on whether they are implemented or inherited ?
> > >
> > > Here's the scheme.  A subtype's tp_new should call the base type's
> > > tp_new, passing the subtype.  The base class will call tp_alloc, which
> > > is the subtype's version.  Similar for deallocation: the subtype's
> > > tp_dealloc calls the base type's tp_dealloc which calls tp_free which
> > > is the subtype's version.
> >
> > Like this... ?
> >
> >          subtype                  basetype
> > ----------------------------------------------------
> > Creation
> >
> >          tp_new(subtype)
> >                                -> tp_new(subtype)    # calls tp_alloc & tp_init
> >
> >          tp_alloc(subtype)     <-
> >                                -> tp_alloc(subtype)
> 
> Typically, the derved type's tp_alloc shouldn't call the base type's
> tp_alloc -- tp_alloc is supposed to allocate memory for the actual
> type, zero it, set the type pointer and reference count, and register
> it with GC.  Any other initializations that can't be left to tp_init
> (which is optional) are tp_new's responsibility.

Good, so overriding the tp_alloc/free slots is generally not
a wise thing to do, I guess.
 
> >          tp_init(instance)     <-
> >                                -> tp_init(instance)
> >
> > Finalization
> >
> >         (
> >          tp_delete(instance)
> >                                -> tp_delete(instance) # calls tp_dealloc if
> >                                                       # the instance should
> >                                                       # be deleted
> >         )
> >          tp_dealloc(instance)
> >                                -> tp_dealloc(instance) # calls tp_free
> >
> >          tp_free(instance)     <-
> >                                -> tp_free(instance)
> 
> Likewise, tp_free needn't call the base tp_free.
> 
> > > > 3. How can I make attributes visible in subclassed types ?
> > > >
> > > > Even though I found out that I need to use the generic APIs
> > > > PyObject_GenericGet|SetAttr() for the tp_get|setattro to
> > > > make methods visible, attributes cannot be accessed (and this
> > > > even though dir(instance) displays them).
> > >
> > > Strange.  This should work.  Probably something's subtly wrong in your
> > > setup.  Compare your code to xxsubtype.c.
> >
> > The xxsubtype doesn't define any attributes and neither do lists
> > or dictionaries so there seems to be no precedent.
> >
> > In mxDateTime under Python 2.1, the tp_gettattr slot takes care of
> > processing attribute lookup. Now to enable the dynamic goodies in
> > Python 2.2, I have to provide the tp_getattro slot (and set it to
> > the generic APIs mentioned above).
> >
> > Since tp_getattro override the tp_getattr slots, I have to rely
> > on the generic APIs calling back to the tp_getattr slots to process
> > the attributes which are not dynamically set by the user or a
> > subclass. However, the new generic lookup APIs do not call the
> > tp_getattr slot at all and thus the attributes which were "defined"
> > by the tp_getattr in Python 2.1 are no longer visible.
> >
> > - How do I have to implement attribute lookup in Python 2.2
> >   for TP_BASETYPEs (methods are now magically handled by the tp_methods
> >   slot, there doesn't seem to be a corresponding feature for attributes
> >   though) ?
> 
> Ah, now I see the question.  There's a tp_members slot, similar to the
> tp_methods slot.  The tp_members slot is a pointer to a
> NULL-terminated array of the same form that you would pass to
> PyMember_Get().  If your attributes require custom computation,
> there's also a tp_getset slot which points to a NULL-terminated array
> of 'struct getsetlist' items, which specify a name, a getter C
> function, a setter C function, and a context void *.  This means you
> have to write a pair of (very simple) functions for each writable
> attribute, or a single function per read-only attribute.  (The context
> pointer gives you a chance to share function implementations, but
> I haven't found the need for this yet.)
> 
> Examples of all of these can be found in typeobject.c, look for
> type_getsets and type_members.

Thanks. I'll take a look at the implementation ...
 
> > - Could the generic APIs perhaps fall back to tp_getattr to make
> >   the transition from classic types to base types a little easier ?
> 
> I'd rather not: that would prevent discovery of attributes supported
> by the classic tp_getattr.  The beauty of the new scheme is that *all*
> attributes (methods and data) are listed in the type's __dict__.

Uhm, I think you misunderstood me: tp_getattr is not used anymore
once the Python interpreter finds a tp_getattro slot 
implementation, so there's nothing to prevent ;-):

PyObject_GetAttr() does not use tp_getattr if tp_getattro is 
defined, while PyObject_GetAttrString() prefers tp_getattr over
tp_getattro -- something is not symmertic here !

As a result, dir() finds the __members__ attribute which lists
the attributes (it uses PyObject_GetAttrString(), but 
instance.attribute does not work because it uses PyObject_GetAttr().

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/