[C++-SIG] Extending PyMethodDef to match arguments...

[Skip Montanaro]

-----BEGIN PGP SIGNED MESSAGE-----


I'm working slowly toward wrapping VTK into Python.  As a large, still
evolving, C++ library it represents two problems.  One, it almost certainly
has to be wrapped automatically.  While I could start from either the
existing Tcl or Java wrappers that Ken Martin wrote, both would involve a
reasonable amount of work, not be applicable later to anything besides VTK
and generate huge wrappers.  Two, just because it's a C++ library, it
presents some extra effort when wrapping into Python's C environment.

I'd like to use SWIG to wrap the library, but it doesn't currently handle
overloaded functions.  That's okay.  I figure any work I put into that would
be useful to the larger community and probably wouldn't be that much more
(or different) work than writing a Python wrapper specifically for VTK.

That all as a preface, I started thinking about what I'd like to see in the
Python environment to support wrapping C++ classes.  In a previous effort I
wrapped a fair amount of an early (pre 1.0) version of VTK into Python using
some ad hoc lexing/parsing stuff I wrote and some simple mods to Python
(then 1.0.5 or 1.1 I think) to get it to compile under C++.  This worked
okay, however the generated code was huge because of all the straight-line
argument parsing.  To demonstrate the problem, suppose I have a member
function for the spam class called foo that exists in two versions, one that
takes an int arg and one that takes a char * arg.  After wrapping, I might
call it like:

    a = spam()
    a.foo(4)
    a.foo("seven")

At run-time, a PyMethodDef struct steers the interpreter to a function,
perhaps spam_foo.  Inside spam_foo code is needed that type-checks the
argument list and calls the correct version of the C++ spam::foo.

    int i;
    int result;
    char *s;

    if (PyArg_ParseTuple(args, "i", &i)) {
	self->cpp->foo(i);
	return PyInt_FromLong(result);
    }
    PyErr_Clear();

    if (PyArg_ParseTuple(args, "s", &s)) {
	result = self->cpp->foo(s);
	return PyInt_FromLong(result);
    }

    return NULL;

This sort of code gets big *fast*, especially in classes that do lots of
function overloading.

What I'd like to see is some way to declare an enhanced PyMethodDef struct
that would allow Python's function lookup code to distinguish functions
based upon their complete signature, not just their names.  That way a
single piece of code could compare the name and actual arguments with the
name and formal arguments of the various versions of the overloaded
function, and call a thin veneer function in the module that simply extracts
the arguments and makes the appropriate member function call.

One way to do this might be using C++ name mangling.  Assume we had the
following simple class definition:

    #include <stdlib.h>

    class spam {
    public:
	int foo(int i);
	int foo(char *s);
    };

    int
    spam::foo(int i) {
	return i;
    }

    int
    spam::foo(char *s) {
	return atoi(s);
    }

On my system, when compiled, I get the following two symbols in the
generated .o file:

    _foo__4spamPc
    _foo__4spami

corresponding to the two member functions.  Seems to me that a program like
SWIG could spit out PyMethodDef declarations like:

    static PyMethodDef spam_methods[] = {
	    {"foo__4spamPc",	spam_foo__4spamPc,	1},
	    {"foo__4spami",	spam_foo__4spami,	1},
	    {NULL,		NULL}		/* sentinel */
    };

and with a slightly different Py_InitModule call, set things up to use this
alternative name lookup at runtime.  The only difference at run-time is that
Python has to generate a mangled name and use that when looking for a
matching wrapper function.  This it can do based upon the function called
and the values of the actual arguments.

spam_foo__4spamPc might look like:

    static PyObject *
    spam_foo__4spamPc(PySpamObject *self, PyObject *args) {
	char *s;
	int i;
	PyArg_ParseTuple(args, "s", &s);
	/* no checks necessary, because we know the type checking has
	   already been done! */
	i = self->cpp->foo(s);
	return PyInt_FromLong(i);
	Py_INCREF(Py_None);
	return Py_None;
    }

assuming the wrapper generated a PySpamObject with a spam * member called
cpp.  Thus, most wrapper functions become a few argument declarations, a
single call to PyArg_ParseTuple, a call to a member function, and possibly a
return statement.  (I'm oversimplifying, since with more complex argument
types things get a bit dicier.)

The net effect is to centralize function lookup based upon both names and
argument types, thus drastically reducing the size of class wrappers.

I am only now subscribing to the c++-sig list to see if something like this
is already being done and will begin marching through that group's archives.
I welcome feedback on the idea and pointers to other work in this area.

- -- 
Skip Montanaro     | Musi-Cal Express - get your own private Musi-Cal
skip at calendar.com  | domain name! http://concerts.calendar.com/express.shtml
(518)372-5583      | WebFast - http://www.webfast.com/

-----BEGIN PGP SIGNATURE-----
Version: 2.6.2
Comment: Processed by Mailcrypt 3.4, an Emacs/PGP interface

iQCVAwUBM8hO+R+q0G630cGhAQFaqQP/TkY5jBYLHvcMr11pMGItL6HcWBemmE7H
TndSgWN4BOpk02UyBFCwvyTkY5DmqF7BJkg5Y0aD4YUQ0iZ7rUyNnRiLkvxjny1f
NPdKPhKWOB6un3KfRHg+sMdHTz8rSm4tVgJ3P1gHjwWQGCP9jO0GgVDnBYTcKwCQ
L+TowlTZvjU=
=QEbk
-----END PGP SIGNATURE-----

_______________
C++-SIG - SIG for Development of a C++ Binding to Python

send messages to: c++-sig at python.org
administrivia to: c++-sig-request at python.org
_______________