[Python-Dev] Need advice, maybe support

[Christian Tismer]

Guido van Rossum wrote:

>>There is one second thought about this, but I'm not sure
>>whether it is allowed to do so:
>>
>>Assuming that I *would* simply do add a field to PyMethodDef,
>>and take care that all types coming from foreign binaries
>>don't have that special type bit set, could I not simply create
>>a new method table and replace it for that external type
>>by just changing its method table pointer?
> 
> 
> Probably.

Promising! Let's see...

> I just realize that there are two uses of PyMethodDef.
> 
> One is the "classic", where the type's tp_getattr[o] implementation
> calls Py_FindMethod.

Right. This one is under my control, since I have the type
and so I have or don't have the bit.

> The other is the new style where the PyMethodDef
> array is in tp_methods, and is scanned once by PyType_Ready.

Right, again. Now, under the hopeful assumption that every
sensible extension module that has some types to publish also
does this through its module dictionary, I would have the
opportunity to cause PyType_Ready being called early enough
to modify the method table, before any of its methods is used
at all.

> 3rd party modules that have been around for a while are likely to use
> Py_FindMethod.  With Py_FindMethod you don't have a convenient way to
> store the pointer to the converted table, so it may be better to
> simply check your bit in the first array element and then cast to a
> PyMethodDef or a PyMethodDefEx array based on what the bit says (you
> can safely assume that all elements of an array are the same size :-).

Hee hee, yeah. Of course, if there isn't a reliable way to
intercept method table access before the first Py_FindMethod
call, I could of course modify Py_FindMethod. For instance,
a modified, new-style method table might be required to always
start with a dummy entry, where the flags word is completely
-1, to signal having been converted to new-style.

...

>>If that approach is trustworthy, I also could drop
>>the request for these 8 bits.
> 
> Sure.  Ah, a bit in the type would work just as well, and
> Py_FindMethod *does* have access to the type.

You think of the definition in methodobject.c, as it is

"""
/* Find a method in a single method list */

PyObject *
Py_FindMethod(PyMethodDef *methods, PyObject *self, char *name)
"""

, assuming that self always is not NULL, but representing a valid
object with a type, and this type is already referring to the
methods table?
Except for module objects, this seems to be right. I've run
Python against a lot of Python modules, but none seems
to call Py_FindMethod with a self parameter of NULL.

If that is true, then I can patch a small couple of
C functions to check for the new bit, and if it's not
there, re-create the method table in place.
This is music to me ears. But...

Well, there is a drawback:
I *do* need two bits, and I hope you will allow me to add this
second bit, as well.

The one, first bit, tells me if the source has been compiled
with Stackless and its extension stuff. Nullo problemo.
I can then in-place modify the method table in a compatible
way, or leave it as it is, bny default.
But then, this isn't sufficient to set this bit then, like an
"everything is fine, now" relief. This is so, since this is *still*
an old module, and while its type's method tables have been
patched, the type is still not augmented by new slots, like
the new tp_call_nr slots (and maybe a bazillion to come, soon).
The drawback is, that I cannot simply replace the whole type
object, since type objects are not represented as object
pointers (like they are now, most of the time, in the dynamic
heaptype case), but they are constant struct addresses, where
the old C module might be referring to.

So, what I think to need is no longer 9 bits, but two of them:
One that says "everything great from the beginning", and another
one that says "well, ok so far, but this is still an old object".

I do think this is the complete story, now.
Instead of requiring nine bits, I'm asking for two.
But this is just *your options; I also can live with one bit,
but then I have to add a special, invalid method table entry
that just serves for this purpose.
In order to keep my souce code hack to the minimum, I'd really
like to ask for the two bits in the typeobject flags.

Thanks so much for being so supportive -- chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/