From alok.jadhav at credit-suisse.com Tue Dec 8 01:09:14 2009 From: alok.jadhav at credit-suisse.com (Jadhav, Alok) Date: Tue, 8 Dec 2009 08:09:14 +0800 Subject: [capi-sig] Python C Api extension (multi threaded environment) Raises exception (Accesss violation error) Message-ID: <8037240182E2574D8E8706A2CB3D99B1084D3116@ESNG17P32002A.csfb.cs-group.com> I am using SWIG to extend Python to C++. But the questions I have consists of basic Python C API related. I hope this list will be able to help me. I am trying to solve a simple problem which many of you must have encountered. I wonder what I am doing different than others that I am facing a problem. Background ----------- A c++ library is wrapped in my C++ Class. This new C++ class is extended in Python using SWIG. I have to call a python function from C++ code ( which is on a new thread spawned from C++ main thread). Issue ----- I am able to call the function. No issues. I could pass basic data types such as int. No issues. But when I try to pass a C++ class object (eg RFAMessageWrapper) to my python function I get an exception. It is an Access violation exception. The memory could not be "read". I am not even able to catch this exception. Some code snippets -------------------- C++ function which calls python function. ... [code] static void PythonCallBack(RFAMessage *msg, void *clientdata) { PyObject *func, *arglist; int blah = 1; PyGILState_STATE state; state = PyGILState_Ensure(); func = (PyObject *) clientdata; // Get Python function RFAMessageWrapper msg2(msg); try { arglist = Py_BuildValue("(O)",msg2); // Build argument list //arglist = Py_BuildValue("(i)", blah); PyEval_CallObject(func,arglist); // Call Python } catch (...) { cout<<"Unknown exception..."< References: <8037240182E2574D8E8706A2CB3D99B1084D3116@ESNG17P32002A.csfb.cs-group.com> Message-ID: Hi, I don't have any experience with SWIG, but I can see two places where things may go wrong: 1) If the call back function was garbage collected before you called it (probably not the case if it is a normal function on the Python side) 2) I assume that RFAMessageWrapper is an extension of a PyObject that swig generated; in this case you should probably build arglist using a pointer to, ie: arglist = Py_BuildValue("(O)",&msg2); -- Ulf Wors?e Mosek ApS On Tue, Dec 8, 2009 at 1:09 AM, Jadhav, Alok wrote: > I am using SWIG to extend Python to C++. ? But the questions I have > consists of basic Python C API related. I hope this list will be able to > help me. > > I am trying to solve a simple problem which many of you must have > encountered. I wonder what I am doing different than others that I am > facing a problem. > > Background > ----------- > > A c++ library is wrapped in my C++ Class. This new C++ class is extended > in Python using SWIG. I have to call a python function from C++ code ( > which is on a new thread spawned from C++ main thread). > > Issue > ----- > I am able to call the function. No issues. I could pass basic data types > such as int. No issues. But when I try to pass a C++ class object (eg > RFAMessageWrapper) to my python function I get an exception. It is an > Access violation exception. The memory could not be "read". ?I am not > even able to catch this exception. > > > > Some code snippets > -------------------- > > C++ function which calls python function. ... > > [code] > static void PythonCallBack(RFAMessage *msg, void *clientdata) > { > ? PyObject *func, *arglist; > ? int blah = 1; > > ? PyGILState_STATE state; > ? state = PyGILState_Ensure(); > ? func = (PyObject *) clientdata; ? ? ? ? ? ? ? // Get Python function > ? RFAMessageWrapper msg2(msg); > ? ? ? ?try { > ? ? ? ? ? ? ? ?arglist = Py_BuildValue("(O)",msg2); ? ? ? ? ? ? // > Build argument list > ? ? ? ? ? ? ? ?//arglist = Py_BuildValue("(i)", blah); > ? ? ? ? ? ? ? ?PyEval_CallObject(func,arglist); ? ? // Call Python > ? ? ? ?} catch (...) { > ? ? ? ? ? ? ? ?cout<<"Unknown exception..."< ? ? ? ?} > ?Py_XDECREF(arglist); ? ? ? ? ? ? ? ? ? ? ? ? ? // Trash arglist > ? PyGILState_Release(state); > } > [/code] > > > Python function call.. > > [code] > def callback_fun(msg): > ? ?try: > ? ? ? ?print "RFAMessage received for service:"+msg.service > ? ?except Exception: > ? ? ? ?print "Exception handled" > [/code] > > Whenever I try to access msg (of type RFAMessageWrapper .. C++ object) I > get exception. msg is not None. I checked that in if condition. even > type(msg) raises exception. > RFAMessageWrapper class has been wrapped properly by SWIG as I could > create an object manually and use it in Python. But only when I pass it > from C++ to python I get this exception. > > > I have spent long time to solve this issue but in vain. I hope I get > some help from this Forum. > > > Regards, > Alok > > =============================================================================== > ?Please access the attached hyperlink for an important electronic communications disclaimer: > ?http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html > ?=============================================================================== > > _______________________________________________ > capi-sig mailing list > capi-sig at python.org > http://mail.python.org/mailman/listinfo/capi-sig > From python_capi at behnel.de Tue Dec 8 09:55:26 2009 From: python_capi at behnel.de (Stefan Behnel) Date: Tue, 08 Dec 2009 09:55:26 +0100 Subject: [capi-sig] Python C Api extension (multi threaded environment) Raises exception (Accesss violation error) In-Reply-To: <8037240182E2574D8E8706A2CB3D99B1084D3116@ESNG17P32002A.csfb.cs-group.com> References: <8037240182E2574D8E8706A2CB3D99B1084D3116@ESNG17P32002A.csfb.cs-group.com> Message-ID: <4B1E147E.8060106@behnel.de> Jadhav, Alok, 08.12.2009 01:09: > I am using SWIG to extend Python to C++. But the questions I have > consists of basic Python C API related. I hope this list will be able to > help me. > > I am trying to solve a simple problem which many of you must have > encountered. I wonder what I am doing different than others that I am > facing a problem. > > Background > ----------- > > A c++ library is wrapped in my C++ Class. This new C++ class is extended > in Python using SWIG. I have to call a python function from C++ code ( > which is on a new thread spawned from C++ main thread). This might help: http://www.linuxjournal.com/article/3641 Stefan From george_edison55 at hotmail.com Wed Dec 9 04:28:17 2009 From: george_edison55 at hotmail.com (Nathan Osman) Date: Tue, 8 Dec 2009 19:28:17 -0800 Subject: [capi-sig] Using the Python/C API with mingw-w64 Message-ID: I am having a little bit of trouble getting the Python headers working with mingw-w64. The app I'm developing is C++ based and is being compiled for the x64 platform. When I include Python headers I get errors saying that standard C functions like round() are being redefined. I think it was in PyMath.h. And yes, I am using the 64 bit headers and libs from python.org - George _________________________________________________________________ Windows Live: Keep your friends up to date with what you do online. http://go.microsoft.com/?linkid=9691815 From gjcarneiro at gmail.com Mon Dec 21 23:53:45 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Mon, 21 Dec 2009 22:53:45 +0000 Subject: [capi-sig] ANN: PyBindGen 0.13 released Message-ID: PyBindGen is a Python module that is geared to generating C/C++ code that binds a C/C++ library for Python. It does so without extensive use of either C++ templates or C pre-processor macros. It has modular handling of C/C++ types, and can be easily extended with Python plugins. The generated code is almost as clean as what a human programmer would write. It can be downloaded from: http://code.google.com/p/pybindgen/ Bug reports should be filed here: https://bugs.launchpad.net/ pybindgen Documentation: http://packages.python.org/PyBindGen/ NEWS: - Lots of small bug fixes - Custodian/ward-style memory management works better now - Add 'reference_existing_object' and 'return_interal_reference' options for pointer/reference return values - New Sphinx based documentation -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert From ColesworthyD at Otologics.com Tue Dec 29 21:38:30 2009 From: ColesworthyD at Otologics.com (Dan Colesworthy) Date: Tue, 29 Dec 2009 13:38:30 -0700 Subject: [capi-sig] Reference counts and dictionary creadtion Message-ID: <86989B801589354BBDE224488335AAF42D2961@xchange2.otologics.com> Hello, I am attempting to write a python extension in C to "glue" the ASIO SDK to python 2.5.4. However I am really confused about reference counts. Below is a code snippet that "seems" to work, however I suspect that some of the object reference counts have been incremented too many times. static PyObject * getAsioInfo(PyObject *self, PyObject *args) { PyObject *dict = Py_None; if (loadAsioDriver (ASIO_DRIVER_NAME)) { if (ASIOInit (&asioDriverInfo.driverInfo) == ASE_OK){ dict = PyDict_New(); PyDict_SetItem(dict, PyString_FromString("asioVersion"), PyInt_FromLong(asioDriverInfo.driverInfo.asioVersion) ); PyDict_SetItem(dict, PyString_FromString("driverName"), PyString_FromString(asioDriverInfo.driverInfo.name) ); return dict; } } return Null; } As nearly as I can tell from the Python source code, PyDict_SetItem will do a Py_INCREF on both the key and value parameters. However, it looks as if PyInt_FromLong will incref the object it is returning - at least if it is a "small value". Am I on the right track here? Or am I better off doing something like: PyObject *str, *lng; str = PyString_FromString("asioVersion"); lng = PyInt_FromLong(asioDriverInfo.driverInfo.asioVersion) ); PyDict_SetItem(dict, str, lng); Py_DECREF(str); Py_DECREF(lng); Thanks in advance for all advice! Dan Colesworthy From python_capi at behnel.de Tue Dec 29 23:25:23 2009 From: python_capi at behnel.de (Stefan Behnel) Date: Tue, 29 Dec 2009 23:25:23 +0100 Subject: [capi-sig] Reference counts and dictionary creadtion In-Reply-To: <86989B801589354BBDE224488335AAF42D2961@xchange2.otologics.com> References: <86989B801589354BBDE224488335AAF42D2961@xchange2.otologics.com> Message-ID: <4B3A81D3.2090103@behnel.de> Dan Colesworthy, 29.12.2009 21:38: > I am attempting to write a python extension in C to "glue" the ASIO SDK > to python 2.5.4. However I am really confused about reference counts. You might want to take a look into Cython, a Python-like language that is well suited for writing very fast C library wrappers. Besides many other features, it will do all the reference counting for you and generates code that works in Python 2 and Python 3. > Below is a code snippet that "seems" to work, however I suspect that > some of the object reference counts have been incremented too many > times. > > > static PyObject * > > getAsioInfo(PyObject *self, PyObject *args) { > > PyObject *dict = Py_None; > > if (loadAsioDriver (ASIO_DRIVER_NAME)) { > > if (ASIOInit (&asioDriverInfo.driverInfo) == ASE_OK){ > > dict = PyDict_New(); > > PyDict_SetItem(dict, > > PyString_FromString("asioVersion"), Yes, this doesn't work. PyDict_SetItem() doesn't steal the reference, so you end up with a dangling reference to the just created string. Same for the other objects created below: > PyInt_FromLong(asioDriverInfo.driverInfo.asioVersion) ); > > PyDict_SetItem(dict, > > PyString_FromString("driverName"), > > > PyString_FromString(asioDriverInfo.driverInfo.name) ); You need to assign each object to a variable, call SetItem() on it, and then Py_DECREF() it afterwards. Note that there is also a special function PyDict_SetItemString() that accepts a char* as key and creates the Python string object internally. Also note that the PyString_*() functions are likely not what you want to use in Python 3. Stefan From mal at egenix.com Wed Dec 30 23:41:15 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 30 Dec 2009 23:41:15 +0100 Subject: [capi-sig] Reference counts and dictionary creadtion In-Reply-To: <86989B801589354BBDE224488335AAF42D2961@xchange2.otologics.com> References: <86989B801589354BBDE224488335AAF42D2961@xchange2.otologics.com> Message-ID: <4B3BD70B.9080300@egenix.com> Dan Colesworthy wrote: > Hello, > > > > I am attempting to write a python extension in C to "glue" the ASIO SDK > to python 2.5.4. However I am really confused about reference counts. > Below is a code snippet that "seems" to work, however I suspect that > some of the object reference counts have been incremented too many > times. > > > > static PyObject * > > getAsioInfo(PyObject *self, PyObject *args) { > > PyObject *dict = Py_None; > > if (loadAsioDriver (ASIO_DRIVER_NAME)) { > > if (ASIOInit (&asioDriverInfo.driverInfo) == ASE_OK){ > > dict = PyDict_New(); > > PyDict_SetItem(dict, > > PyString_FromString("asioVersion"), > > > PyInt_FromLong(asioDriverInfo.driverInfo.asioVersion) ); > > PyDict_SetItem(dict, > > PyString_FromString("driverName"), > > > PyString_FromString(asioDriverInfo.driverInfo.name) ); > > return dict; > > } > > } > > return Null; > > } > > > > As nearly as I can tell from the Python source code, PyDict_SetItem will > do a Py_INCREF on both the key and value parameters. However, it looks > as if PyInt_FromLong will incref the object it is returning - at least > if it is a "small value". Object constructors will always return objects with a refcount of 1. When adding a key, value pair to a dictionary using PyDict_SetItem() the function will take care of adjusting the refcounts. After the call, you still own the objects key and value, so you have to decref them if you no longer need them. > Am I on the right track here? Or am I better > off doing something like: > > > > PyObject *str, *lng; > > > > str = PyString_FromString("asioVersion"); > > lng = PyInt_FromLong(asioDriverInfo.driverInfo.asioVersion) > ); > > PyDict_SetItem(dict, str, lng); > > Py_DECREF(str); > > Py_DECREF(lng); See above. The second approach is likely what you need. BTW: PyDict_SetItemString() is usually a better approach to filling dictionaries that use often used keys. It interns the key string for you, making subsequent lookups faster. OTOH, if you plan to port to Python 3.x, you will likely want to use Unicode objects as keys. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Dec 30 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/