[Python-3000] A better way to initialize PyTypeObject

[Guido van Rossum]

On 12/2/06, Talin <talin at acm.org> wrote:
> Guido van Rossum wrote:
> > My preference is for a table-based version over a "lots-of-calls"
> > approach. I think the idea of using predefined (integer) constants
> > instead of string constants fits fine into the table-based approach.
> > However I'm not sure that a string-based approach is necessarily bad
> > -- we use that now for everything that's not a special method after
> > all. I do get the point that the string-based approach uses more
> > memory and needs more processing time.
>
> OK. Its easy to implement either way. I coded up a string-based version,
> but if you guys want integer constants that's fine with me; probably
> best to let Fredrik & Larry tweak their patch rather than me continuing
> along this path since their stuff is further along.
>
> The two reasons why I thought strings might be good is:
>
> 1) I wanted to use the PyMethodDef table to store both special and
> regular methods; in other words, instead of having a special table for
> initializing PyTypeObject, I was going to re-use one of the already
> existing tables, and PyMethodDef already identifies methods via string
> (although you can type-pun an integer into the same slot.)

Unfortunately putting the special functions in that table doesn't
really work for many of the methods that have non-standard signatures.

> 2) I wanted regular methods and special methods to be treated uniformly,
> at least at the API level, and let the runtime decide whether or not a
> given method should be "special" (in other words, have a dedicated slot
> in PyTypeObject.)

But that's the thing that doesn't work. If a special function takes or
returns a C int it is impossible to use it as a regular method.

> If regular methods are registered by string, and
> special methods are registered by integer, then you can't really mix
> them; whereas if they are both registered via the same mechanism, then
> you can later go back and decide that a given special method doesn't
> really need to be special and vice versa. So for example, there's no
> dedicated slot for __unicode__, but if in the future performance
> measurement should indicate that creating such a slot would be of
> benefit, you could simply add a new slot for it and all existing
> modules, when recompiled, would automatically use the new slot.

That applies to a very limited set of methods.

> However, all that being said I recognize that string lookups aren't
> cheap. This is especially true in this case where almost all of the
> names in the table start with "__", so you have to look at the 3rd
> character before you detect a mismatch.
>
> What I am aiming at is for PyTypeObject to be as opaque as possible in
> practice, in other words initialization code shouldn't need to know much
> about the ordering or types of fields. This gives the developers of the
> Python runtime a great deal of freedom in their ability to refactor the
> internals of the runtime without breaking existing extension modules.

The ordering argument is also handled by using C99 initializers.

The typing argument seems hard to believe -- if an extension provides
a value for a field, it better have the right type. I don't see how we
can hide the type or protect against the type changing, except in very
limited cases where two types have the same representation.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)