help with writing extension
Eric Hagemann
ehagemann at comcast.net
Mon Sep 30 17:28:32 EDT 2002
Alex
Thanks for another excellent answer. I am going to dig into the
Py_TPFLAGS_CHECKTYPES flag as that might just fix my problem. I did not
really want to use the __coerce__function but I did not know how to get
around it. If adding this flag to the definition will skip the __coerce__
function, that should save me a bunch-o-code. BTW I see references to the
CHECKTYPES flag in the Python 2.1 source (Yes - I am still using 2.1) any
reason why I need to move to 2.2 ?
I eventually (after studying the sources) did find the reason for my
"problem".
Summarized for any that might want to know.....
When you do a binary_op between two objects that are not equal in type
the __coerce__ function is called for the first object. If that does not
'coerce' the different objects into compatable ones then the __coerce__
function is called for the second object. If that does not return
sucessfully then you are done for. However if that returns sucess then the
operation is called using the method defined for the first of the two
objects
As it turns out I already skipped throught __coerce__ function by just
incrementing the references on the two objects and returning success. This
worked well when I was doing operations of <obj>+int as I would just pass
both on to my 'add' function and I delt with the integer in my function.
However when working with the arguments in reverse, this method would not
work as after the second coerce function I would still have an int+<obj>
and it would call the 'add' function for the int, which even after two
attempted coercions, would not know what <obj> was and the TypeError was
generated.
What was refreshing was that I could find the root of the problem by
studying the source -- says something for how the code was written, nice --
very nice.
Cheers
"Alex Martelli" <aleax at aleax.it> wrote in message
news:pQYl9.190868$pX1.6828893 at news2.tin.it...
> <posted & mailed>
>
> Eric Hagemann wrote:
>
> > I have been handcrafting an extension module creating a new type
> >
> > I am making use of the numeric methods (+,-,* etc)
> >
> > I solved the problem of the object interating with other types by
working
> > through the __coerce__() function and have no issue with operations like
> > <obj>+2 but I am getting an error on 2-<obj>.
>
> As I understand, __coerce__ is not recommended for new code. You should,
> I believe, add Py_TPFLAGS_CHECKTYPES to your tp_flags slot, and be ready
> in your numeric functions to deal with types as they come. Sometimes
> coercion can be handier, but in general avoiding it is better.
>
>
> > In Beazley's book there are descriptions of functions like __radd__()
when
> > defining a new type in python, but I cannot find reference to these
> > functions in C ?
>
> If you grep Python's C sources (2.2 or better), you'll find some such
> references. Slots nb_add of the PyNumberMethods structure to which
> slot tp_as_number of your PyTypeObject structure point supplies both
> __add__ and __radd__ (when you call PyType_Ready on your type, suitable
> wrappers on that slot are exposed with those two names).
>
>
> > I have checked the doc's (not to say that I did not miss something ;) )
>
> I think you still need to complement existing docs with some little
> study of the Python sources when you want to design rich and complete
> extension types -- and I think that dealing with __radd__ etc in the
> right way is part of a "rich and complete" (rather than just a
> really minimal) extension type. I think this will still be true
> when my "Python in a Nutshell" is finally out, because the chapter
> on extending and embedding with C is limited to 40-50 pages (can't
> be more precise right now because the book's being edited and I'm
> not sure how many of my examples and details the editor will want
> to keep -- or if even more will be added), and doing complete justice
> to the subject would require about 10 times as much space.
>
> But I also think that asking here about such issues is just right!
> Best place to get help -- and this way the help gets archived (thanks
> to google groups and other archivers) and can be pointed to in the
> future (which alas doesn't work for other mailing lists such as
> python-help, nor, of course, for private mail:-).
>
>
> An example can help -- sorry if it's not very concise, but the
> PyTypeObject structure is so large that any examples of its use
> are also large, even though most fields are 0. Say I have a
> directory ~/extype with a setup.py file such as:
>
> from distutils.core import setup, Extension
>
> setup(name = "samp1",
> version = "1.0",
> description = "Sample extension type",
> maintainer = "Alex Martelli",
> maintainer_email = "aleaxit at yahoo.com",
>
> ext_modules = [ Extension('samp1', sources=['samp1.c']) ]
> )
>
> and a samp1.c file such as:
>
> #include "Python.h"
>
> typedef struct {
> PyObject_HEAD
> } samp1;
>
> static void
> samp1_dealloc(PyObject *op)
> {
> op->ob_type->tp_free(op);
> }
>
> static PyObject *
> samp1_amethod(PyObject *self)
> {
> Py_INCREF(Py_None);
> return Py_None;
> }
>
> static PyObject *
> samp1_add(PyObject *self, PyObject *other)
> {
> PyObject* result = PyTuple_New(2);
> if(!result)
> return NULL;
> Py_INCREF(self);
> PyTuple_SET_ITEM(result, 0, self);
> Py_INCREF(other);
> PyTuple_SET_ITEM(result, 1, other);
> return result;
> }
>
> static PyMethodDef samp1_methods[] = {
> {"amethod", (PyCFunction)samp1_amethod, METH_NOARGS},
> {NULL, NULL} /* sentinel */
> };
>
> static PyNumberMethods samp1_as_number = {
> samp1_add,
> };
>
> static PyTypeObject samp1_t = {
> PyObject_HEAD_INIT(0)
> 0,
> "samp1",
> sizeof(samp1),
> 0,
> samp1_dealloc,
> 0,
> 0,
> 0,
> 0, /* tp_compare */
> 0, /* tp_repr */
> &samp1_as_number, /* tp_as_number */
> 0, /* tp_as_sequence */
> 0, /* tp_as_mapping */
> 0, /* tp_hash */
> 0, /* tp_call */
> 0, /* tp_str */
> 0, /* tp_getattro */
> 0, /* tp_setattro */
> 0, /* tp_as_buffer */
> Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE|Py_TPFLAGS_CHECKTYPES,
> "sample extension type",
> 0, /* tp_traverse */
> 0, /* tp_clear */
> 0, /* tp_richcompare */
> 0, /* tp_weaklistoffset */
> 0, /* tp_iter */
> 0, /* tp_iternext */
> samp1_methods, /* tp_methods */
> 0, /* tp_members */
> 0, /* tp_getset */
> 0, /* tp_base */
> 0, /* tp_dict */
> 0, /* tp_descr_get */
> 0, /* tp_descr_set */
> 0, /* tp_dictoffset */
> 0, /* tp_init */
> PyType_GenericAlloc, /* tp_alloc */
> PyType_GenericNew, /* tp_new */
> _PyObject_Del, /* tp_free */
> };
>
> static PyMethodDef no_methods[] = { {0} };
>
> void
> initsamp1(void)
> {
> PyObject* self;
> PyType_Ready(&samp1_t);
> self = Py_InitModule("samp1", no_methods);
> PyObject_SetAttrString(self, "samp1", (PyObject*)&samp1_t);
> }
>
>
> Now of course running python setup.py in this dictionary builds
> and installs extension module samp1. Let's look at it in action:
>
> [alex at lancelot extype]$ python
> Python 2.2.1 (#2, Jul 15 2002, 17:32:26)
> [GCC 2.96 20000731 (Mandrake Linux 8.1 2.96-0.62mdk)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import samp1
> >>> examp = samp1.samp1()
> >>> examp + 'ciao!'
> (<samp1 object at 0x8160f28>, 'ciao!')
> >>> 'salute...' + examp
> ('salute...', <samp1 object at 0x8160f28>)
> >>>
>
> See what's going on? Despite the (traditional) argument names,
> samp1_add is being called with the LEFT operand of + as the first
> argument ('self'), the RIGHT operand as the second argument
> ('other') -- your C code can test the types directly to do the
> right thing in both the __add__ and __radd__ cases...!
>
> Sure, you CAN make do with coercion (so have all Python extension
> types until about last year), but bypassing it makes things a bit
> clearer, I believe. If you do want coercion in some but nor all
> of your numeric-methods you can deal with it yourself -- call your
> own "coercion-like" function from your methods' code as and where
> appropriate. I do think it's simpler than having Python do it for
> you and sometimes having to "bypass" it, so to speak... this way
> you can still supply a function to be called when the user does an
> explicit coerce call, and yet have whatever precise behavior you
> want for all operators that look like numeric operators ("look like",
> because, for example, sequence concatenation isn't really numeric --
> it just LOOKS LIKE it is, and has to live in PyNumberMethods, but
> that's not quite where one might like to place it...).
>
>
> Feel free to keep posting such questions to this list if there is
> anything unclear in the above, and/or if you need any more help!
>
>
> Alex
>
More information about the Python-list
mailing list