From gjcarneiro at gmail.com  Mon Dec  8 17:49:00 2008
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Mon, 8 Dec 2008 16:49:00 +0000
Subject: [capi-sig] ANN: PyBindGen 0.10 released
Message-ID: <a467ca4f0812080849s292ad71cle0cad75cfc05529a@mail.gmail.com>

PyBindGen is a Python module that is geared to generating C/C++ code that
binds a C/C++ library for Python. It does so without extensive use of either
C++ templates or C pre-processor macros. It has modular handling of C/C++
types, and can be easily extended with Python plugins. The generated code is
almost as clean as what a human programmer would write.

It can be downloaded from:

        http://code.google.com/p/pybindgen/

Bug reports should be filed here:

        https://bugs.launchpad.net/pybindgen


=== pybindgen 0.10 ===

    - New null_ok, default_value options for pointer-to-class parameters;

    - Thread safety fixes;

    - Map C++ operator() into Python's tp_call (__call__);

    - Initial support for std containers (except mapping containers);

    - Generate __copy__ methods for classes with copy constructor;

    - Add 'wrapper registry' optional feature (off by default): allows
      C++ instances returned by pointer to be consistently wrapped
      always by the same Python wrapper, so that the 'is' operator can
      be used for identity testing;

    - New C type expression parser, is_const not longer needed for
      parameter types, just put const in the type string as
      appropriate;

    - Sort the declarations returned by (py)gccxml.  If generating
      python script files from gccxml scanning, now the output file
      becomes more stable and less prone to move declarations up and
      down with different people scanning and the output file is under
      version control;

    - Map binary comparison operators (< <= == >= >) from C++ to Python;

    - Map some binary numeric operators (+ - * /) from C++ to Python;

    - Allow installation of pybindgen even if no C/C++ compiler is detected;

    - Fix compatibility with older Python versions (tested with 2.3, 2.4,
and 2.5)

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert

From chojrak11 at gmail.com  Mon Dec 22 02:02:33 2008
From: chojrak11 at gmail.com (chojrak11 at gmail.com)
Date: Mon, 22 Dec 2008 02:02:33 +0100
Subject: [capi-sig] Exceptions with additional instance variables
Message-ID: <bfb4c52d0812211702l57c0d003g4b2a694be7d3b534@mail.gmail.com>

Hello,

I'm trying to implement custom exception that have to carry some
useful info by means of instance members, to be used like:

try:
    // some code
except MyException, data:
    // use data.errorcode, data.errorcategory, data.errorlevel,
data.errormessage and some others

The question is - how to implement the instance variables with
PyErr_NewException? Or should I use another approach? And how to set
values of these variables at the moment of exception? Should I use
PyErr_SetString or PyErr_SetObject?

I'm lost.

Thanks in advance.
Kind regards.

From ggpolo at gmail.com  Mon Dec 22 03:29:31 2008
From: ggpolo at gmail.com (Guilherme Polo)
Date: Mon, 22 Dec 2008 00:29:31 -0200
Subject: [capi-sig] Exceptions with additional instance variables
In-Reply-To: <bfb4c52d0812211702l57c0d003g4b2a694be7d3b534@mail.gmail.com>
References: <bfb4c52d0812211702l57c0d003g4b2a694be7d3b534@mail.gmail.com>
Message-ID: <ac2200130812211829o7589249dp818fed9eb1b0642d@mail.gmail.com>

On Sun, Dec 21, 2008 at 11:02 PM,  <chojrak11 at gmail.com> wrote:
> Hello,
>
> I'm trying to implement custom exception that have to carry some
> useful info by means of instance members, to be used like:
>
> try:
>    // some code
> except MyException, data:
>    // use data.errorcode, data.errorcategory, data.errorlevel,
> data.errormessage and some others
>
> The question is - how to implement the instance variables with
> PyErr_NewException?

Using PyErr_NewException is fine. You must understand that an
exception is a class, and thus PyErr_NewException creates one for you
and returns it.

> Or should I use another approach? And how to set
> values of these variables at the moment of exception?

Just like you would do with a class that has __dict__, set some
attributes to what you want. That is, use PyObject_SetAttrString or
something more appropriated for you.

> Should I use
> PyErr_SetString or PyErr_SetObject?

In both cases you are just specifying the error message, so you use
what is better for you to provide the error message.

>
> I'm lost.
>
> Thanks in advance.
> Kind regards.
> _______________________________________________
> capi-sig mailing list
> capi-sig at python.org
> http://mail.python.org/mailman/listinfo/capi-sig
>



-- 
-- Guilherme H. Polo Goncalves

From chojrak11 at gmail.com  Mon Dec 22 13:06:17 2008
From: chojrak11 at gmail.com (chojrak11 at gmail.com)
Date: Mon, 22 Dec 2008 13:06:17 +0100
Subject: [capi-sig] Exceptions with additional instance variables
Message-ID: <bfb4c52d0812220406x5877db66ud10d64b2407415d4@mail.gmail.com>

On Mon, Dec 22, 2008 at 03:29, Guilherme Polo <ggpolo at gmail.com> wrote:
> On Sun, Dec 21, 2008 at 11:02 PM,  <chojrak11 at gmail.com> wrote:
>> Hello,
>>
>> I'm trying to implement custom exception that have to carry some
>> useful info by means of instance members, to be used like:
>>
>> try:
>>    // some code
>> except MyException, data:
>>    // use data.errorcode, data.errorcategory, data.errorlevel,
>> data.errormessage and some others
>>
>> The question is - how to implement the instance variables with
>> PyErr_NewException?
>
> Using PyErr_NewException is fine. You must understand that an
> exception is a class, and thus PyErr_NewException creates one for you
> and returns it.
> Just like you would do with a class that has __dict__, set some
> attributes to what you want. That is, use PyObject_SetAttrString or
> something more appropriated for you.

Ok so I did the following. In init function (forget refcounting and
error checking for a moment ;-)

PyObject *dict = PyDict_New();
PyDict_SetItemString(dict, "errorcode", PyInt_FromLong(0));
static PyObject *myexception =
PyErr_NewException("module.MyException", NULL, dict);
PyModule_AddObject(module, "MyException", myexception);

It worked more or less as expected, the help shown:

 |  ----------------------------------------------------------------------
 |  Data and other attributes defined here:
 |
 |  errorcode = 0
 |
 |  ----------------------------------------------------------------------

Then I did the following when raising the exception:

PyObject_SetAttrString(myexception, "errorcode", PyInt_FromLong(111));
PyErr_SetString(myexception, "Bad thing happened");
return NULL;

and the test code was:
try:
    do_bad_thing();
except MyException, data:

and you surely already guessed it -- data.errorcode was 0.... Not only
that, module.MyException.errorcode was also 0...

What I'm doing wrong? I certainly don't get the idea of exceptions in
Python, especially what is being raised - a class or an instance? If
the latter - how's the class instantiated? If not - what about values
in different threads? The docs are so vague about that...


Thanks again in advance,
Chojrak

From stefan_ml at behnel.de  Mon Dec 22 13:35:27 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 22 Dec 2008 13:35:27 +0100 (CET)
Subject: [capi-sig] Exceptions with additional instance variables
In-Reply-To: <bfb4c52d0812220406x5877db66ud10d64b2407415d4@mail.gmail.com>
References: <bfb4c52d0812220406x5877db66ud10d64b2407415d4@mail.gmail.com>
Message-ID: <42989.213.61.181.86.1229949327.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

chojrak11 at gmail.com wrote:
> Ok so I did the following. In init function (forget refcounting and
> error checking for a moment ;-)
>
> PyObject *dict = PyDict_New();
> PyDict_SetItemString(dict, "errorcode", PyInt_FromLong(0));
> static PyObject *myexception =
> PyErr_NewException("module.MyException", NULL, dict);
> PyModule_AddObject(module, "MyException", myexception);
>
> It worked more or less as expected, the help shown:
>
>  |  ----------------------------------------------------------------------
>  |  Data and other attributes defined here:
>  |
>  |  errorcode = 0
>  |
>  |  ----------------------------------------------------------------------
>
> Then I did the following when raising the exception:
>
> PyObject_SetAttrString(myexception, "errorcode", PyInt_FromLong(111));
> PyErr_SetString(myexception, "Bad thing happened");
> return NULL;
>
> and the test code was:
> try:
>     do_bad_thing();
> except MyException, data:
>
> and you surely already guessed it -- data.errorcode was 0.... Not only
> that, module.MyException.errorcode was also 0...
>
> What I'm doing wrong? I certainly don't get the idea of exceptions in
> Python, especially what is being raised - a class or an instance? If
> the latter - how's the class instantiated? If not - what about values
> in different threads? The docs are so vague about that...

The quick way to fix your problem might be to write the exception code in
Cython like th?s:

    class MyException(Exception):
        def __init__(self, errorcode, message):
            super(MyException, self).__init__(self, message)
            self.errorcode = errorcode

and use that instead. You can also make it an exception class if you like,
by declaring the Exception class as an external type (requires Py2.5+). In
this case, Cython can generate a header file for you that declares your
new extension type.

http://cython.org/

Stefan


From python_capi at behnel.de  Mon Dec 22 17:16:45 2008
From: python_capi at behnel.de (Stefan Behnel)
Date: Mon, 22 Dec 2008 17:16:45 +0100 (CET)
Subject: [capi-sig] [Fwd: Re: Exceptions with additional instance variables]
Message-ID: <46874.213.61.181.86.1229962605.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

[... and to the list ...]

chojrak11 at gmail.com wrote:
> Ok so I did the following. In init function (forget refcounting and
> error checking for a moment ;-)
>
> PyObject *dict = PyDict_New();
> PyDict_SetItemString(dict, "errorcode", PyInt_FromLong(0));
> static PyObject *myexception =
> PyErr_NewException("module.MyException", NULL, dict);
> PyModule_AddObject(module, "MyException", myexception);
>
> It worked more or less as expected, the help shown:
>
>  |  ----------------------------------------------------------------------
>  |  Data and other attributes defined here:
>  |
>  |  errorcode = 0
>  |
>  |  ----------------------------------------------------------------------
>
> Then I did the following when raising the exception:
>
> PyObject_SetAttrString(myexception, "errorcode", PyInt_FromLong(111));
> PyErr_SetString(myexception, "Bad thing happened");
> return NULL;
>
> and the test code was:
> try:
>     do_bad_thing();
> except MyException, data:
>
> and you surely already guessed it -- data.errorcode was 0.... Not only
> that, module.MyException.errorcode was also 0...
>
> What I'm doing wrong? I certainly don't get the idea of exceptions in
> Python, especially what is being raised - a class or an instance? If
> the latter - how's the class instantiated? If not - what about values
> in different threads? The docs are so vague about that...

The quick way to fix your problem might be to write the exception code in
Cython like th?s:

    class MyException(Exception):
        def __init__(self, errorcode, message):
            super(MyException, self).__init__(self, message)
            self.errorcode = errorcode

and use that instead. You can also make it an exception class if you like,
by declaring the Exception class as an external type (requires Py2.5+). In
this case, Cython can generate a header file for you that declares your
new extension type.

http://cython.org/

Stefan


From chojrak11 at gmail.com  Mon Dec 22 20:32:02 2008
From: chojrak11 at gmail.com (chojrak11 at gmail.com)
Date: Mon, 22 Dec 2008 20:32:02 +0100
Subject: [capi-sig] [Fwd: Re: Exceptions with additional instance
	variables]
In-Reply-To: <46874.213.61.181.86.1229962605.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <46874.213.61.181.86.1229962605.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <bfb4c52d0812221132x42b82cecpd44c90ce40cdf52f@mail.gmail.com>

2008/12/22 Stefan Behnel <python_capi at behnel.de>:

> The quick way to fix your problem might be to write the exception code in
> Cython like th?s:
>
>    class MyException(Exception):
>        def __init__(self, errorcode, message):
>            super(MyException, self).__init__(self, message)
>            self.errorcode = errorcode
>
> and use that instead. You can also make it an exception class if you like,
> by declaring the Exception class as an external type (requires Py2.5+). In
> this case, Cython can generate a header file for you that declares your
> new extension type.
>
> http://cython.org/

Hello Stefan,

thanks for the tip, I'll surely look at Cython and how it can help ;-)
Thanks for the pointer and for the example!

Regards,
Chojrak

From chojrak11 at gmail.com  Mon Dec 22 22:24:45 2008
From: chojrak11 at gmail.com (chojrak11 at gmail.com)
Date: Mon, 22 Dec 2008 22:24:45 +0100
Subject: [capi-sig] Exceptions with additional instance variables
In-Reply-To: <bfb4c52d0812221321x1a0c0008p2b05c5aa32a3f141@mail.gmail.com>
References: <bfb4c52d0812220406x5877db66ud10d64b2407415d4@mail.gmail.com>
	<ac2200130812220445v524d8288lf0df0d435433a87e@mail.gmail.com>
	<bfb4c52d0812221321x1a0c0008p2b05c5aa32a3f141@mail.gmail.com>
Message-ID: <bfb4c52d0812221324x538fb12crc92973fcaa417fa6@mail.gmail.com>

2008/12/22 Guilherme Polo <ggpolo at gmail.com>:

> On Mon, Dec 22, 2008 at 10:06 AM,  <chojrak11 at gmail.com> wrote:
>

> #include "Python.h"
>
> static PyObject *MyErr;
>
> static PyMethodDef module_methods[] = {

>        {"raise_test1", (PyCFunction)raise_test1, METH_NOARGS, NULL},
>        {"raise_test2", (PyCFunction)raise_test2, METH_NOARGS, NULL},
>        {"raise_test3", (PyCFunction)raise_test3, METH_NOARGS, NULL},

>        {NULL},
> };
>
> PyMODINIT_FUNC
> initfancy_exc(void)
> {
>        PyObject *m;
>
>        m = Py_InitModule("fancy_exc", module_methods);
>        if (m == NULL)
>                return;
>
>        MyErr = PyErr_NewException("fancy_exc.err", NULL, NULL);
>
>        Py_INCREF(MyErr);
>        if (PyModule_AddObject(m, "err", MyErr) < 0)
>                return;
> }
>

> static PyObject *
> raise_test1(PyObject *self)

> {
>        PyObject_SetAttrString(MyErr, "code", PyInt_FromLong(42));
>        PyObject_SetAttrString(MyErr, "category", PyString_FromString("nice one"));
>        PyErr_SetString(MyErr, "All is good, I hope");
>        return NULL;
> }
>

> static PyObject *
> raise_test2(PyObject *self)

> {
>
>        PyObject *t = PyTuple_New(3);
>        PyTuple_SetItem(t, 0, PyString_FromString("error message"));
>        PyTuple_SetItem(t, 1, PyInt_FromLong(10));
>        PyTuple_SetItem(t, 2, PyString_FromString("category name here"));
>        PyErr_SetObject(MyErr, t);
>        Py_DECREF(t);
>        return NULL;
> }
>
> In this second form you check for the args attribute of the exception.

static PyObject *
raise_test3(PyObject *self) {
PyObject *d = PyDict_New();
       PyDict_SetItemString(d, "category", PyInt_FromLong(111));
       PyDict_SetItemString(d, "message", PyString_FromString("error
message"));
       PyErr_SetObject(MyErr, d);
       Py_DECREF(d);
       return NULL;
}

(Small changes in the above code to be able to call more variants of
raise_test methods simultaneously.)

Yes! I finally understood this (I think...) So to explain things for
people like me:

1) PyErr_NewException creates *the class* in the module, it's a simple
method of creating exception classes, but classes created that way are
limited in features (i.e. cannot be manipulated from the module in all
ways a 'full' type can). Third argument to PyErr_NewException can be
NULL, in which case API will create an empty dictionary. After
creating the class you need to add it to the module with
PyModule_AddObject. Side note: If you want to specify a help for the
class, you do PyObject_SetAttrString on the class with the key
'__doc__'.

2) there's no instantiation anywhere:
   a. PyErr_SetString and PyErr_SetObject set the exception *class*
(exception type) and exception data -- see
http://docs.python.org/c-api/exceptions.html which notes that
exceptions are similar in concept to the global 'errno' variable, so
you just set what type of last error was and what error message (or
other data) you want to associate with it
   b. the "code" and "category" variables from raise_test1() in the
above example inserted with PyObject_SetAttrString() are *class*
variables, not instance variables:

try:
   fancy_exc.raise_test1()
except fancy_exc.err, e:
   print e.code, fancy_exc.err.code
print fancy_exc.err.code

it prints:
42 42
42

   c. the data is still present in the fancy_exc.err class after
exception handling is finished, which is ok for now but may be
problematic in case of multithreaded usage patterns (however I
probably don't understand how multithreading in Python works)

3) alternative to the above is to pass all required data to the
exception with PyErr_SetObject - you can prepare a dictionary or a
tuple earlier, which will be accessible with 'args' member:

try:
   fancy_exc.raise_test2()
except fancy_exc.err, e:
   print e.args[0]

If it's dictionary, the syntax is a bit weird because e.args is always a tuple:

try:
   fancy_exc.raise_test3()
except fancy_exc.err, e:
   print e.args[0]['category']

The 'args' values are unavailable outside of 'except' clause, however
you can still use the 'e' variable which retains the values. So it's
an instance variable.

4) creating the exception class using a new type in C (PyTypeObject
structure) would give the most robust solution because every nuance of
the class can be manipulated, but it's not worth the trouble now. I
can switch to it transparently at a later time. Transparently means
that nothing will need to be updated in Python solutions written by
the module users.

5) most of the projects I've inspected with Google Code Search use the
PyErr_NewException approach.

6) there's the option of using Cython which simplifies creating
extensions and hides many unnecessary internals.

Many thanks Guilherme and Stefan for your help and for the patience.


Kind regards,
Chojrak

From ideasman42 at gmail.com  Wed Dec 24 05:12:22 2008
From: ideasman42 at gmail.com (Campbell Barton)
Date: Tue, 23 Dec 2008 20:12:22 -0800
Subject: [capi-sig] running C method, benchmarks
Message-ID: <7c1ab96d0812232012o6cf38817nbb8162ccd6a990c9@mail.gmail.com>

Discussing if its worth moving Py/C functions from METH_VARARGS to
METHO when they only recieve 1 argument on the PyGame mailing list.
Crossposting here since some of you might be interested in the results.

tested different ways to evaluate args to see how much speed
difference there was
* 10,000,000 tests, python 2.6 on 32bit arch linux
* included a pass and NOARGS metrhod to see the difference in overhead
of the loop and parsing an arg compared to running a method with no
args.

---- output
pass 1.85659885406
METH_NOARGS 3.24079704285
METH_O 3.66321516037
METH_VARARGS 6.09881997108
METH_KEYWORDS 6.037307024
METH_KEYWORDS (as keyword) 10.9263861179

------- Python Script, (used blender module for testing)

import time

from Blender.sys import test_METHO, test_METH_VARARGS,
test_METH_KEYWORDS, test_METH_NOARGS

RUN = 10000000

t = time.time()
for i in xrange(RUN):   pass
print 'pass', time.time()-t

t = time.time()
for i in xrange(RUN):   test_METH_NOARGS()
print 'METH_NOARGS', time.time()-t

t = time.time()
for i in xrange(RUN):   test_METHO(1)
print 'METH_O', time.time()-t

t = time.time()
for i in xrange(RUN):   test_METH_VARARGS(1)
print 'METH_VARARGS', time.time()-t

t = time.time()
for i in xrange(RUN):   test_METH_KEYWORDS(1)
print 'METH_KEYWORDS', time.time()-t

t = time.time()
for i in xrange(RUN):   test_METH_KEYWORDS(val=1)
print 'METH_KEYWORDS (as keyword)', time.time()-t

------------------- C functions

static PyObject *test_METHO( PyObject * self, PyObject * value )
{
       int val = (int)PyInt_AsLong(value);

       if( val==-1 && PyErr_Occurred() ) {
               PyErr_SetString(PyExc_AttributeError, "not an int");
               return NULL;
       }

       Py_RETURN_NONE;
}

static PyObject *test_METH_VARARGS( PyObject * self, PyObject * args )
{
       int val;

       if( !PyArg_ParseTuple( args, "i", &val ) )
               return NULL;

       Py_RETURN_NONE;
}

static PyObject *test_METH_KEYWORDS( PyObject * self, PyObject * args,
PyObject *kwd)
{
       int val;
       static char *kwlist[] = {"val", NULL};

       if( !PyArg_ParseTupleAndKeywords(args, kwd, "i", kwlist, &val) )
               return NULL;

       Py_RETURN_NONE;
}

static PyObject *test_METH_NOARGS( PyObject * self, PyObject * args )
{
       Py_RETURN_NONE;
}

struct PyMethodDef M_sys_methods[] = {
       {"test_METHO", test_METHO, METH_O, ""},
       {"test_METH_KEYWORDS", test_METH_KEYWORDS, METH_KEYWORDS, ""},
       {"test_METH_NOARGS", test_METH_NOARGS, METH_NOARGS, ""},
       {"test_METH_VARARGS", test_METH_VARARGS, METH_VARARGS, ""},
       {NULL, NULL, 0, NULL}
};

-- 
- Campbell

From python_capi at behnel.de  Wed Dec 24 07:46:58 2008
From: python_capi at behnel.de (Stefan Behnel)
Date: Wed, 24 Dec 2008 07:46:58 +0100
Subject: [capi-sig] running C method, benchmarks
In-Reply-To: <7c1ab96d0812232012o6cf38817nbb8162ccd6a990c9@mail.gmail.com>
References: <7c1ab96d0812232012o6cf38817nbb8162ccd6a990c9@mail.gmail.com>
Message-ID: <4951DAE2.2070009@behnel.de>

Hi,

Campbell Barton wrote:
> Discussing if its worth moving Py/C functions from METH_VARARGS to
> METHO when they only recieve 1 argument on the PyGame mailing list.
> 
> tested different ways to evaluate args to see how much speed
> difference there was
> * 10,000,000 tests, python 2.6 on 32bit arch linux
> * included a pass and NOARGS metrhod to see the difference in overhead
> of the loop and parsing an arg compared to running a method with no
> args.
> 
> ---- output
> pass 1.85659885406
> METH_NOARGS 3.24079704285
> METH_O 3.66321516037
> METH_VARARGS 6.09881997108
> METH_KEYWORDS 6.037307024
> METH_KEYWORDS (as keyword) 10.9263861179

I tried doing something similar in Cython, but it's not directly
comparable. Cython uses optimised code instead of a generic call to
ParseTupleAndKeywords and it will always give you a METH_O function when
you only use one argument. Anyway, here are the numbers. I used the latest
Cython developer version with gcc 4.1.3 on Linux.

Benchmarked code:

-----------------
def f0(): pass             # METH_NOARGS
def f1(a): pass            # METH_O
def f1opt(a=1): pass       # METH_VARARGS|METH_KEYWORDS
def f2(a,b): pass          # METH_VARARGS|METH_KEYWORDS
def f2opt(a=1,b=2): pass   # METH_VARARGS|METH_KEYWORDS
-----------------

Benchmarks:

$ python2.5 -m timeit -s '...; from calltest import f0' 'f0()'
10000000 loops, best of 3: 0.126 usec per loop
$ python2.5 -m timeit -s '...; from calltest import f1opt' 'f1opt()'
10000000 loops, best of 3: 0.14 usec per loop
$ python2.5 -m timeit -s '...; from calltest import f2opt' 'f2opt()'
10000000 loops, best of 3: 0.141 usec per loop
$ python2.5 -m timeit -s '...; from calltest import f1' 'f1(1)'
10000000 loops, best of 3: 0.145 usec per loop
$ python2.5 -m timeit -s '...; from calltest import f2' 'f2(1,2)'
1000000 loops, best of 3: 0.225 usec per loop
$ python2.5 -m timeit -s '...; from calltest import f2' 'f2(1,b=2)'
1000000 loops, best of 3: 0.489 usec per loop

I used Python 2.5.1 as the ihooks module in 2.6.1 is still broken, so
pyximport doesn't work (and I was too lazy to build the module by hand).

Note how f2opt is not much slower than f1opt (both METH_KEYWORDS), which in
turn is still faster then the METH_O function f1.

So my suggestion is that the main reasons for your METH_O function being
faster above are a) that you actually /pass/ arguments, i.e. Python's own
argument passing overhead, and b) the use of ParseTupleAndKeywords() in
your other functions above, which is very fast, but also very generic.
Cython's dedicated argument parsing code is a lot faster in most cases.

Could you repeat your benchmarks using timeit on 2.5 as I do above? That
would give us comparable numbers.

Stefan


From python_capi at behnel.de  Wed Dec 24 22:30:53 2008
From: python_capi at behnel.de (Stefan Behnel)
Date: Wed, 24 Dec 2008 22:30:53 +0100
Subject: [capi-sig] running C method, benchmarks
In-Reply-To: <4951DAE2.2070009@behnel.de>
References: <7c1ab96d0812232012o6cf38817nbb8162ccd6a990c9@mail.gmail.com>
	<4951DAE2.2070009@behnel.de>
Message-ID: <4952AA0D.40306@behnel.de>


Stefan Behnel wrote:
> Campbell Barton wrote:
>> Discussing if its worth moving Py/C functions from METH_VARARGS to
>> METHO when they only recieve 1 argument on the PyGame mailing list.
>>
>> tested different ways to evaluate args to see how much speed
>> difference there was
>> * 10,000,000 tests, python 2.6 on 32bit arch linux
>> * included a pass and NOARGS metrhod to see the difference in overhead
>> of the loop and parsing an arg compared to running a method with no
>> args.
>>
>> ---- output
>> pass 1.85659885406
>> METH_NOARGS 3.24079704285
>> METH_O 3.66321516037
>> METH_VARARGS 6.09881997108
>> METH_KEYWORDS 6.037307024
>> METH_KEYWORDS (as keyword) 10.9263861179
> 
> I tried doing something similar in Cython, but it's not directly
> comparable. Cython uses optimised code instead of a generic call to
> ParseTupleAndKeywords and it will always give you a METH_O function when
> you only use one argument. Anyway, here are the numbers. I used the latest
> Cython developer version with gcc 4.1.3 on Linux.
> 
> Benchmarked code:
> 
> -----------------
> def f0(): pass             # METH_NOARGS
> def f1(a): pass            # METH_O
> def f1opt(a=1): pass       # METH_VARARGS|METH_KEYWORDS
> def f2(a,b): pass          # METH_VARARGS|METH_KEYWORDS
> def f2opt(a=1,b=2): pass   # METH_VARARGS|METH_KEYWORDS
> -----------------
> 
> Benchmarks:
> 
> $ python2.5 -m timeit -s '...; from calltest import f0' 'f0()'
> 10000000 loops, best of 3: 0.126 usec per loop
> $ python2.5 -m timeit -s '...; from calltest import f1opt' 'f1opt()'
> 10000000 loops, best of 3: 0.14 usec per loop
> $ python2.5 -m timeit -s '...; from calltest import f2opt' 'f2opt()'
> 10000000 loops, best of 3: 0.141 usec per loop
> $ python2.5 -m timeit -s '...; from calltest import f1' 'f1(1)'
> 10000000 loops, best of 3: 0.145 usec per loop
> $ python2.5 -m timeit -s '...; from calltest import f2' 'f2(1,2)'
> 1000000 loops, best of 3: 0.225 usec per loop
> $ python2.5 -m timeit -s '...; from calltest import f2' 'f2(1,b=2)'
> 1000000 loops, best of 3: 0.489 usec per loop

I noticed that I forgot to run one interesting test, which is calling
f2opt() with a single argument in comparison to calling f1() with one argument:

$ python2.5 -m timeit -s '...; from calltest import f1' 'f1(1)'
10000000 loops, best of 3: 0.145 usec per loop
$ python2.5 -m timeit -s '...; from calltest import f2opt' 'f2opt(1)'
1000000 loops, best of 3: 0.204 usec per loop
$ python2.5 -m timeit -s '...; from calltest import f1opt' 'f1opt(1)'
1000000 loops, best of 3: 0.204 usec per loop

So, yes, this actually is slower than the METH_O function f1(). It's not
66% as in your example, more like 40%, but it definitely is a lot slower.
So I would say that the overhead of calling a METH_VARARGS|METH_KEYWORDS
function versus a METH_O function is somewhere in the order of 40% for the
case that only positional arguments are involved.

Stefan