[Python-Dev] PEP 253: Subtyping Built-in Types

[M.-A. Lemburg]

Guido van Rossum wrote:
> 
> > I've started playing with making mxDateTime types subclassable
> 
> Cool!!!

A few people keep asking me for new features on those types, so
I guess enabling this for Python 2.2 would be a real advantage for 
them.

I still haven't found out how to solve the construction problem
though (the base type is hard coded into various factory functions
and methods)... the factory methods could use self.__class__
to solve this, but the factory functions would need some different
tweaking.
 
> > and have run into a few problems which the PEP does not seem
> > to have answers to:
> >
> > 1. Are tp_new et al. inherited by subclassed types ?
> 
> My apologies that this stuff is so underdocumented -- there's just so
> *much* to be documented...  in typeobject.c, in inherit_slots(),
> there's a call to COPYSLOT(tp_new), so the answer is yes.

Ok.
 
> > This is important when implementing the slot methods, since
> > they may then see types other than the one for which they
> > are defined (e.g. keeping a free list around will only
> > work for the original types, not subclassed ones).
> 
> Yes, I've worked out a scheme to make this work, but I don't think
> I've written it down anywhere yet.  If your tp_new calls tp_alloc, and
> your tp_dealloc calls tp_free, then a subtype can override tp_alloc
> *and* tp_free and the right thing will happen.  A subtype can also
> *extend* tp_new and tp_dealloc.  (tp_new and tp_dealloc are sort-of
> each other's companions, and ditto for tp_alloc and tp_free.)

So I will have to implement tp_free as well ?! Currently I have
tp_new (which calls tp_alloc), tp_alloc, tp_init for the creation
procedure and tp_dealloc (which does not call tp_free) for the
finalization.

I wonder whether it'd be a good idea to have a tp_del in there
as well (the __del__ at C level) which is then called instead
of tp_dealloc if set and which must call tp_dealloc if the
instance is going to be deleted for good.
 
> > 2. In which order are the allocation/deallocation methods
> > of subclass and base class called (if at all) and how
> > does this depend on whether they are implemented or inherited ?
> 
> Here's the scheme.  A subtype's tp_new should call the base type's
> tp_new, passing the subtype.  The base class will call tp_alloc, which
> is the subtype's version.  Similar for deallocation: the subtype's
> tp_dealloc calls the base type's tp_dealloc which calls tp_free which
> is the subtype's version.

Like this... ?

         subtype                  basetype
----------------------------------------------------
Creation

         tp_new(subtype) 
                               -> tp_new(subtype)    # calls tp_alloc & tp_init

         tp_alloc(subtype)     <-
                               -> tp_alloc(subtype)

         tp_init(instance)     <-
                               -> tp_init(instance)

Finalization

        (
         tp_delete(instance)
                               -> tp_delete(instance) # calls tp_dealloc if
                                                      # the instance should
                                                      # be deleted
        )
         tp_dealloc(instance)
                               -> tp_dealloc(instance) # calls tp_free

         tp_free(instance)     <-
                               -> tp_free(instance)

> > 3. How can I make attributes visible in subclassed types ?
> >
> > Even though I found out that I need to use the generic APIs
> > PyObject_GenericGet|SetAttr() for the tp_get|setattro to
> > make methods visible, attributes cannot be accessed (and this
> > even though dir(instance) displays them).
> 
> Strange.  This should work.  Probably something's subtly wrong in your
> setup.  Compare your code to xxsubtype.c.

The xxsubtype doesn't define any attributes and neither do lists
or dictionaries so there seems to be no precedent.

In mxDateTime under Python 2.1, the tp_gettattr slot takes care of
processing attribute lookup. Now to enable the dynamic goodies in
Python 2.2, I have to provide the tp_getattro slot (and set it to
the generic APIs mentioned above). 

Since tp_getattro override the tp_getattr slots, I have to rely 
on the generic APIs calling back to the tp_getattr slots to process 
the attributes which are not dynamically set by the user or a 
subclass. However, the new generic lookup APIs do not call the
tp_getattr slot at all and thus the attributes which were "defined"
by the tp_getattr in Python 2.1 are no longer visible.

- How do I have to implement attribute lookup in Python 2.2
  for TP_BASETYPEs (methods are now magically handled by the tp_methods
  slot, there doesn't seem to be a corresponding feature for attributes
  though) ?

- Could the generic APIs perhaps fall back to tp_getattr to make
  the transition from classic types to base types a little easier ?

Thanks, 
-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
______________________________________________________________________
Consulting & Company:                           http://www.egenix.com/
Python Software:                        http://www.lemburg.com/python/