From pycnweb at 126.com Tue Mar 3 03:27:52 2009 From: pycnweb at 126.com (pycnweb) Date: Tue, 3 Mar 2009 10:27:52 +0800 (CST) Subject: [capi-sig] Problem with reference Message-ID: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> Hi All, I am now writing a python extension, but I confuse with the reference. PyTuple_SetItem and PyList_SetItem will steal a reference from item. So PyTuple_SetItem(pTuple, 2, Py_BuildValue("s", "Python")) PyList_SetItem(pList, i, Py_BuildValue("i", i)) will not cause a memory leak. But PyList_Insert(pList, 2, Py_BuildValue("s", "inserted")) PyList_Append(pList, Py_BuildValue("s", "appended")) PyDict_SetItemString(pDict, "second", Py_BuildValue("f", 3.14f)) will cause a memory leak. Am I right? Thanks. From python_capi at behnel.de Tue Mar 3 08:16:39 2009 From: python_capi at behnel.de (Stefan Behnel) Date: Tue, 03 Mar 2009 08:16:39 +0100 Subject: [capi-sig] Problem with reference In-Reply-To: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> References: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> Message-ID: <49ACD957.3020608@behnel.de> pycnweb wrote: > I am now writing a python extension, but I confuse with the reference. If you care more about working code than fighting confusion, Cython might be worth a look. http://cython.org > PyTuple_SetItem and PyList_SetItem will steal a reference from item. So > > PyTuple_SetItem(pTuple, 2, Py_BuildValue("s", "Python")) > PyList_SetItem(pList, i, Py_BuildValue("i", i)) > > will not cause a memory leak. Correct. > But > > PyList_Insert(pList, 2, Py_BuildValue("s", "inserted")) > PyList_Append(pList, Py_BuildValue("s", "appended")) > PyDict_SetItemString(pDict, "second", Py_BuildValue("f", 3.14f)) > > will cause a memory leak. Yes. Stefan From hniksic at xemacs.org Tue Mar 3 08:29:14 2009 From: hniksic at xemacs.org (Hrvoje Niksic) Date: Tue, 03 Mar 2009 08:29:14 +0100 Subject: [capi-sig] Problem with reference In-Reply-To: <49ACD957.3020608@behnel.de> (Stefan Behnel's message of "Tue\, 03 Mar 2009 08\:16\:39 +0100") References: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> <49ACD957.3020608@behnel.de> Message-ID: <877i37uks5.fsf@mulj.homelinux.net> Stefan Behnel writes: > pycnweb wrote: >> I am now writing a python extension, but I confuse with the reference. > > If you care more about working code than fighting confusion, Cython > might be worth a look. Stefan, please stop doing this. This list is about helping people using Python/C, not Cython advocacy. From python_capi at behnel.de Tue Mar 3 10:38:14 2009 From: python_capi at behnel.de (Stefan Behnel) Date: Tue, 3 Mar 2009 10:38:14 +0100 (CET) Subject: [capi-sig] Problem with reference In-Reply-To: <877i37uks5.fsf@mulj.homelinux.net> References: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> <49ACD957.3020608@behnel.de> <877i37uks5.fsf@mulj.homelinux.net> Message-ID: <51518.213.61.181.86.1236073094.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Hrvoje Niksic wrote: > Stefan Behnel writes: >> pycnweb wrote: >>> I am now writing a python extension, but I confuse with the reference. >> >> If you care more about working code than fighting confusion, Cython >> might be worth a look. > > Stefan, please stop doing this. This list is about helping people > using Python/C, not Cython advocacy. Where is the difference? Stefan From mal at egenix.com Tue Mar 3 10:51:47 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 03 Mar 2009 10:51:47 +0100 Subject: [capi-sig] Problem with reference In-Reply-To: <51518.213.61.181.86.1236073094.squirrel@groupware.dvs.informatik.tu-darmstadt.de> References: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> <49ACD957.3020608@behnel.de> <877i37uks5.fsf@mulj.homelinux.net> <51518.213.61.181.86.1236073094.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Message-ID: <49ACFDB3.80002@egenix.com> On 2009-03-03 10:38, Stefan Behnel wrote: > Hrvoje Niksic wrote: >> Stefan Behnel writes: >>> pycnweb wrote: >>>> I am now writing a python extension, but I confuse with the reference. >>> If you care more about working code than fighting confusion, Cython >>> might be worth a look. >> Stefan, please stop doing this. This list is about helping people >> using Python/C, not Cython advocacy. > > Where is the difference? The list would be called cython-sig otherwise :-) There are many other alternatives to approaching Python extension writing in C/C++: * use ctypes * use Boost.Python * use SWIG * use sip * use PyCXX * use PyRex/Cython * use Robin * ... (there are lots more) However, this list is about the layer underneath all of those. It's about the Python C API and there are many good reasons to use it directly rather than using one of the above wrappers or wrapper generators. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 03 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From python_capi at behnel.de Tue Mar 3 12:14:24 2009 From: python_capi at behnel.de (Stefan Behnel) Date: Tue, 3 Mar 2009 12:14:24 +0100 (CET) Subject: [capi-sig] Problem with reference In-Reply-To: <49ACFDB3.80002@egenix.com> References: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> <49ACD957.3020608@behnel.de> <877i37uks5.fsf@mulj.homelinux.net> <51518.213.61.181.86.1236073094.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <49ACFDB3.80002@egenix.com> Message-ID: <55273.213.61.181.86.1236078864.squirrel@groupware.dvs.informatik.tu-darmstadt.de> M.-A. Lemburg wrote: > On 2009-03-03 10:38, Stefan Behnel wrote: >> Hrvoje Niksic wrote: >>> Stefan Behnel writes: >>>> pycnweb wrote: >>>>> I am now writing a python extension, but I confuse with the >>>>> reference. >>>> If you care more about working code than fighting confusion, Cython >>>> might be worth a look. >>> Stefan, please stop doing this. This list is about helping people >>> using Python/C, not Cython advocacy. >> >> Where is the difference? > > The list would be called cython-sig otherwise :-) > > There are many other alternatives to approaching Python extension writing > in C/C++: > > * use ctypes > * use Boost.Python > * use SWIG > * use sip > * use PyCXX > * use PyRex/Cython > * use Robin > * ... > > (there are lots more) I know almost all of these at least by name. However, most of them are wrapper generators, which clearly limits their field of applicability (and rarely leads to good wrappers without major manual interaction). Cython is not a wrapper generator. It's a programming language that sits right between Python and C and generates code for the C-API of Python. That's why its related to this mailing list and why I advocate it to people who are clearly starting to write code against that API. It just makes their life a lot easier without sacrificing anything. It's similar to what the Jython developers say, "we write C so you don't have to". If you ever need to make a direct call to Python's C-API in Cython code, it's one cimport away. In all other cases, it's better to concentrate on real logic and algorithmic optimisations in working code, than on counting references, converting types, building dicts and writing extension class structs. It's pretty close to Python's own spirit in that regard. Plus, no matter how long you grow your code, Cython just doesn't start to get in your way - which has always been the case for any wrapper generator I've ever worked with. I do see ctypes as a pure-Python-level alternative, but it's not related to this list in any obvious way. > However, this list is about the layer underneath all of those. > It's about the Python C API and there are many good reasons > to use it directly rather than using one of the above wrappers > or wrapper generators. I agree regarding the wrapper generators, but I do not agree regarding the need to call Python's C-API. I know that from personal experience as the author of lxml (which is written in Cython). lxml's code uses direct C-API calls in quite a few places, but that's almost purely for historical reasons. Today, most of that has become a premature performance optimisation that only makes the code less obvious. Last year, I started refactoring that back into Python/Cython code, because in 95% of the code, there just isn't any reason left to use direct C-API calls. Plain Cython code is at least as fast and often faster than hand-written code, especially when it comes to interaction with the Python runtime. Call me a burnt child, I hope this makes it understandable that I'm trying to keep new C-API users from investing tons of time into their C code, just to notice that it will finally run, say, 30% slower and be less portable than what they could have had with 10-15% of the code size in Cython. And since the factor of 10 in code size is very real, it leaves you with 80-95% of the plain-C development time to benchmark and optimise your code. Stefan From hniksic at xemacs.org Tue Mar 3 12:40:04 2009 From: hniksic at xemacs.org (Hrvoje Niksic) Date: Tue, 03 Mar 2009 12:40:04 +0100 Subject: [capi-sig] Problem with reference In-Reply-To: <51518.213.61.181.86.1236073094.squirrel@groupware.dvs.informatik.tu-darmstadt.de> (Stefan Behnel's message of "Tue\, 3 Mar 2009 10\:38\:14 +0100 \(CET\)") References: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> <49ACD957.3020608@behnel.de> <877i37uks5.fsf@mulj.homelinux.net> <51518.213.61.181.86.1236073094.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Message-ID: <87myc2vnqj.fsf@mulj.homelinux.net> "Stefan Behnel" writes: > Hrvoje Niksic wrote: >> Stefan Behnel writes: >>> pycnweb wrote: >>>> I am now writing a python extension, but I confuse with the reference. >>> >>> If you care more about working code than fighting confusion, Cython >>> might be worth a look. >> >> Stefan, please stop doing this. This list is about helping people >> using Python/C, not Cython advocacy. > > Where is the difference? The point is that statements like the above don't help someone who *wants* (for whatever reason) to learn Python/C. It's perfectly fine to mention Cython as a viable alternative, but not to imply that Python/C is somehow non-functional or that it doesn't result in working code. It's like posting to a C newsgroup telling everyone to use C++ because it's so much better. It may well be better, but there are good reasons why there are separate C and C++ forums. comp.lang.python is a much more appropriate venue for Cython (or boost.python, or PyCXX) advocacy. From mal at egenix.com Tue Mar 3 12:48:50 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 03 Mar 2009 12:48:50 +0100 Subject: [capi-sig] Problem with reference In-Reply-To: <55273.213.61.181.86.1236078864.squirrel@groupware.dvs.informatik.tu-darmstadt.de> References: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> <49ACD957.3020608@behnel.de> <877i37uks5.fsf@mulj.homelinux.net> <51518.213.61.181.86.1236073094.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <49ACFDB3.80002@egenix.com> <55273.213.61.181.86.1236078864.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Message-ID: <49AD1922.6060500@egenix.com> On 2009-03-03 12:14, Stefan Behnel wrote: > Cython is not a wrapper generator. It's a programming language that sits right > between Python and C and generates code for the C-API of Python. I think that's where the misunderstanding lies: This list is about the Python C-API and not about generators or other helpers that provide ways of avoiding it :-) http://www.python.org/community/sigs/current/capi-sig/ IMHO, the list is about: * discussing ways of improving the Python C API * helping developers that have questions about the Python C API Suggesting to use Cython or any of the other available tools feels a bit like saying "avoid the Python C API wherever you can", which doesn't give the right impression to developers seeking help from this forum. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 03 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From gjcarneiro at gmail.com Tue Mar 3 12:54:28 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Tue, 3 Mar 2009 11:54:28 +0000 Subject: [capi-sig] Problem with reference In-Reply-To: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> References: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> Message-ID: 2009/3/3 pycnweb > Hi All, > > > I am now writing a python extension, but I confuse with the reference. > > PyTuple_SetItem and PyList_SetItem will steal a reference from item. So > > PyTuple_SetItem(pTuple, 2, Py_BuildValue("s", "Python")) > PyList_SetItem(pList, i, Py_BuildValue("i", i)) > > will not cause a memory leak. But > > PyList_Insert(pList, 2, Py_BuildValue("s", "inserted")) > PyList_Append(pList, Py_BuildValue("s", "appended")) > PyDict_SetItemString(pDict, "second", Py_BuildValue("f", 3.14f)) > > will cause a memory leak. > > Am I right? You are right. -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert From gjcarneiro at gmail.com Tue Mar 3 12:56:38 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Tue, 3 Mar 2009 11:56:38 +0000 Subject: [capi-sig] Problem with reference In-Reply-To: <49ACFDB3.80002@egenix.com> References: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> <49ACD957.3020608@behnel.de> <877i37uks5.fsf@mulj.homelinux.net> <51518.213.61.181.86.1236073094.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <49ACFDB3.80002@egenix.com> Message-ID: 2009/3/3 M.-A. Lemburg > On 2009-03-03 10:38, Stefan Behnel wrote: > > Hrvoje Niksic wrote: > >> Stefan Behnel writes: > >>> pycnweb wrote: > >>>> I am now writing a python extension, but I confuse with the reference. > >>> If you care more about working code than fighting confusion, Cython > >>> might be worth a look. > >> Stefan, please stop doing this. This list is about helping people > >> using Python/C, not Cython advocacy. > > > > Where is the difference? > > The list would be called cython-sig otherwise :-) > > There are many other alternatives to approaching Python extension writing > in C/C++: > > * use ctypes > * use Boost.Python > * use SWIG > * use sip > * use PyCXX > * use PyRex/Cython > * use Robin > * ... And, * PyBindGen :-) And Cython *is* a wrapper generator. I clearly remember seeing (ugly) C code generated by Cython, before I decided to write PyBindGen. > > (there are lots more) > > However, this list is about the layer underneath all of those. > It's about the Python C API and there are many good reasons > to use it directly rather than using one of the above wrappers > or wrapper generators. Agreed! > > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Mar 03 2009) > >>> Python/Zope Consulting and Support ... http://www.egenix.com/ > >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ > >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ > ________________________________________________________________________ > > ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: > > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ > _______________________________________________ > capi-sig mailing list > capi-sig at python.org > http://mail.python.org/mailman/listinfo/capi-sig > -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert From python_capi at behnel.de Tue Mar 3 13:07:16 2009 From: python_capi at behnel.de (Stefan Behnel) Date: Tue, 3 Mar 2009 13:07:16 +0100 (CET) Subject: [capi-sig] Problem with reference In-Reply-To: <87myc2vnqj.fsf@mulj.homelinux.net> References: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> <49ACD957.3020608@behnel.de> <877i37uks5.fsf@mulj.homelinux.net> <51518.213.61.181.86.1236073094.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <87myc2vnqj.fsf@mulj.homelinux.net> Message-ID: <44335.213.61.181.86.1236082036.squirrel@groupware.dvs.informatik.tu-darmstadt.de> [and to the list] Hrvoje Niksic wrote: > "Stefan Behnel" writes: >> Hrvoje Niksic wrote: >>> Stefan Behnel writes: >>>> pycnweb wrote: >>>>> I am now writing a python extension, but I confuse with the >>>>> reference. >>>> >>>> If you care more about working code than fighting confusion, Cython >>>> might be worth a look. >>> >>> Stefan, please stop doing this. This list is about helping people >>> using Python/C, not Cython advocacy. >> >> Where is the difference? > > The point is that statements like the above don't help someone who > *wants* (for whatever reason) to learn Python/C. It's perfectly fine > to mention Cython as a viable alternative, but not to imply that > Python/C is somehow non-functional or that it doesn't result in > working code. I take your point. Especially to a someone who has just started to learn the C-API, my above statement might sound misleading. Sorry. Stefan From hniksic at xemacs.org Tue Mar 3 13:15:19 2009 From: hniksic at xemacs.org (Hrvoje Niksic) Date: Tue, 03 Mar 2009 13:15:19 +0100 Subject: [capi-sig] Problem with reference In-Reply-To: <44335.213.61.181.86.1236082036.squirrel@groupware.dvs.informatik.tu-darmstadt.de> (Stefan Behnel's message of "Tue\, 3 Mar 2009 13\:07\:16 +0100 \(CET\)") References: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> <49ACD957.3020608@behnel.de> <877i37uks5.fsf@mulj.homelinux.net> <51518.213.61.181.86.1236073094.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <87myc2vnqj.fsf@mulj.homelinux.net> <44335.213.61.181.86.1236082036.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Message-ID: <87eixevm3s.fsf@mulj.homelinux.net> "Stefan Behnel" writes: > I take your point. Especially to a someone who has just started to > learn the C-API, my above statement might sound misleading. Thank you. Other than the advocacy thing :-), I very much appreciate your contributions to the list. From python_capi at behnel.de Tue Mar 3 15:44:35 2009 From: python_capi at behnel.de (Stefan Behnel) Date: Tue, 3 Mar 2009 15:44:35 +0100 (CET) Subject: [capi-sig] Problem with reference In-Reply-To: References: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> <49ACD957.3020608@behnel.de> <877i37uks5.fsf@mulj.homelinux.net> <51518.213.61.181.86.1236073094.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <49ACFDB3.80002@egenix.com> Message-ID: <35237.213.61.181.86.1236091475.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Gustavo Carneiro wrote: > And Cython *is* a wrapper generator. Well, no, but PyBindGen is, now that you mention it. It looks a bit like a FFI (like ctypes), but it generates the corresponding wrapper code, if I'm not mistaken. Using it for anything but wrapping C/C++ code to make it callable in Python would be futile. Cython is (literally) not a wrapper generator as it will not generate a wrapper for you. You have to write it yourself. Cython is a programming language to write extension modules for CPython, as generic as that. It focuses on Python-like simplicity, C-like speed and easy interfacing with both CPython and C (while better C++ and Fortran support is still being fledged out). It is being used to write wrappers, but it is also being used to write high-performance computational code, with or without interfacing to external C libraries. In the second line, it's also heading to become a fully-compliant Python compiler (a 1.0 goal), but that doesn't belong here (just like the automatic-compile-on-import support for Python files *wink*). > I clearly remember seeing (ugly) C > code generated by Cython, before I decided to write PyBindGen. Let's not start a discussion about taste here. Readable, commented, easy to debug C code is an obvious goal of the Cython project, simply because its developers are those who debug it most of the time. That said, we generally tell our users to look at the highlighted HTML page that Cython generates of their source code, so that they can focus more easily on those parts of the generated C code that are worth optimising. Having to read the entire C code file just to see where things go wrong is not a good idea. Stefan From python_capi at behnel.de Tue Mar 3 15:54:19 2009 From: python_capi at behnel.de (Stefan Behnel) Date: Tue, 3 Mar 2009 15:54:19 +0100 (CET) Subject: [capi-sig] Problem with reference In-Reply-To: <49AD1922.6060500@egenix.com> References: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> <49ACD957.3020608@behnel.de> <877i37uks5.fsf@mulj.homelinux.net> <51518.213.61.181.86.1236073094.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <49ACFDB3.80002@egenix.com> <55273.213.61.181.86.1236078864.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <49AD1922.6060500@egenix.com> Message-ID: <47314.213.61.181.86.1236092059.squirrel@groupware.dvs.informatik.tu-darmstadt.de> M.-A. Lemburg wrote: > On 2009-03-03 12:14, Stefan Behnel wrote: >> Cython is not a wrapper generator. It's a programming language that sits >> right between Python and C and generates code for the C-API of Python. > > I think that's where the misunderstanding lies: This list is about the > Python C-API and not about generators or other helpers that provide > ways of avoiding it :-) > > http://www.python.org/community/sigs/current/capi-sig/ > > IMHO, the list is about: > > * discussing ways of improving the Python C API Fine with me, happy to contribute, but a rare thing. > * helping developers that have questions about the Python C API Most developers who ask on this list do so because they have problems with the C-API and its concepts, with reference counting or because they fail to understand why the code they have written does not do what they want it to do. This is the right list to ask these things, and I'm all for helping them out. > Suggesting to use Cython or any of the other available tools feels > a bit like saying "avoid the Python C API wherever you can", which > doesn't give the right impression to developers seeking help from > this forum. I'm not saying "avoid the C-API wherever you can", because it is not the goal to avoid the C-API. The goal is to get your code working, to keep it maintainable, and to make it fast (in more or less that order). Plus, once you have achieved to build your "first Python extension", you are totally on your own when it comes to supporting different Python versions and C compilers. For example, migrating code from Py2.4 to Py2.5 wasn't trivial, but making it work on both Py2.x and Py3 is actually really hard, been there, done that. There is no win in pushing new developers into all that. That is what the Cython project is all about. Stefan From gjcarneiro at gmail.com Tue Mar 3 16:01:52 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Tue, 3 Mar 2009 15:01:52 +0000 Subject: [capi-sig] Problem with reference In-Reply-To: <35237.213.61.181.86.1236091475.squirrel@groupware.dvs.informatik.tu-darmstadt.de> References: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> <49ACD957.3020608@behnel.de> <877i37uks5.fsf@mulj.homelinux.net> <51518.213.61.181.86.1236073094.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <49ACFDB3.80002@egenix.com> <35237.213.61.181.86.1236091475.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Message-ID: 2009/3/3 Stefan Behnel > Gustavo Carneiro wrote: > > And Cython *is* a wrapper generator. > > Well, no, but PyBindGen is, now that you mention it. It looks a bit like a > FFI (like ctypes), but it generates the corresponding wrapper code, if I'm > not mistaken. Using it for anything but wrapping C/C++ code to make it > callable in Python would be futile. > > Cython is (literally) not a wrapper generator as it will not generate a > wrapper for you. You have to write it yourself. Cython is a programming > language to write extension modules for CPython, as generic as that. It > focuses on Python-like simplicity, C-like speed and easy interfacing with > both CPython and C (while better C++ and Fortran support is still being > fledged out). It is being used to write wrappers, but it is also being > used to write high-performance computational code, with or without > interfacing to external C libraries. In the second line, it's also heading > to become a fully-compliant Python compiler (a 1.0 goal), but that doesn't > belong here (just like the automatic-compile-on-import support for Python > files *wink*). OK, but surely we can say that Cython is a code generator, or compiler. So it's more generic than a wrapper generator. But it can fulfill the role of wrapper generator, even if it means writing more code than other tools for describing the same API. For me, wrapper is any C/C++ code that bindings Python to a C/C++ library. If Cython can generate that code, then Cython can also be considered a wrapper generator, or at least the engine for a wrapper generator. It is more or less at the same level as Boost.Python. You also have to write the wrapper code using Boost.Python yourself... > > > > > I clearly remember seeing (ugly) C > > code generated by Cython, before I decided to write PyBindGen. > > Let's not start a discussion about taste here. Readable, commented, easy > to debug C code is an obvious goal of the Cython project, simply because > its developers are those who debug it most of the time. That said, we > generally tell our users to look at the highlighted HTML page that Cython > generates of their source code, so that they can focus more easily on > those parts of the generated C code that are worth optimising. Having to > read the entire C code file just to see where things go wrong is not a > good idea. OK. I won't comment except to mention that I am glad the Cython project wants to generate readable code. Regards, -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert From python_capi at behnel.de Tue Mar 3 16:22:33 2009 From: python_capi at behnel.de (Stefan Behnel) Date: Tue, 3 Mar 2009 16:22:33 +0100 (CET) Subject: [capi-sig] Problem with reference In-Reply-To: References: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> <49ACD957.3020608@behnel.de> <877i37uks5.fsf@mulj.homelinux.net> <51518.213.61.181.86.1236073094.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <49ACFDB3.80002@egenix.com> <35237.213.61.181.86.1236091475.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Message-ID: <46331.213.61.181.86.1236093753.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Gustavo Carneiro wrote: > 2009/3/3 Stefan Behnel wrote: >> Cython is (literally) not a wrapper generator as it will not generate a >> wrapper for you. You have to write it yourself. Cython is a programming >> language to write extension modules for CPython, as generic as that. > > OK, but surely we can say that Cython is a code generator, or compiler. > [...] > It is more or less at the same level as Boost.Python. > You also have to write the wrapper code using Boost.Python yourself... Except that Boost.Python does not optimise the code for you, simply because it is a library, not a compiler. It doesn't know what called it with what intention. The Cython compiler can extract a lot of information from your code and it *will* use it to make your code run fast. For example, calling a Cython implemented function is very fast compared to normal Python, simply because we can adapt the generated code to the specific signature of a function, instead of relying on generic argument unpacking code. Stefan From khughes at pacific.edu Tue Mar 3 16:28:50 2009 From: khughes at pacific.edu (Ken Hughes) Date: Tue, 03 Mar 2009 07:28:50 -0800 Subject: [capi-sig] (was Re: Problem with reference) In-Reply-To: <46331.213.61.181.86.1236093753.squirrel@groupware.dvs.informatik.tu-darmstadt.de> References: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> <49ACD957.3020608@behnel.de> <877i37uks5.fsf@mulj.homelinux.net> <51518.213.61.181.86.1236073094.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <49ACFDB3.80002@egenix.com> <35237.213.61.181.86.1236091475.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <46331.213.61.181.86.1236093753.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Message-ID: <49AD4CB2.8030705@pacific.edu> 1) Did anyone answer the original poster's question? 2) If this is going to persist as an OT thread, change the subject line? Ken Stefan Behnel wrote: > Gustavo Carneiro wrote: > >> 2009/3/3 Stefan Behnel wrote: >> >>> Cython is (literally) not a wrapper generator as it will not generate a >>> wrapper for you. You have to write it yourself. Cython is a programming >>> language to write extension modules for CPython, as generic as that. >>> >> OK, but surely we can say that Cython is a code generator, or compiler. >> [...] >> It is more or less at the same level as Boost.Python. >> You also have to write the wrapper code using Boost.Python yourself... >> > > Except that Boost.Python does not optimise the code for you, simply > because it is a library, not a compiler. It doesn't know what called it > with what intention. The Cython compiler can extract a lot of information > from your code and it *will* use it to make your code run fast. For > example, calling a Cython implemented function is very fast compared to > normal Python, simply because we can adapt the generated code to the > specific signature of a function, instead of relying on generic argument > unpacking code. > > Stefan > > _______________________________________________ > capi-sig mailing list > capi-sig at python.org > http://mail.python.org/mailman/listinfo/capi-sig > > From python_capi at behnel.de Tue Mar 3 16:43:35 2009 From: python_capi at behnel.de (Stefan Behnel) Date: Tue, 3 Mar 2009 16:43:35 +0100 (CET) Subject: [capi-sig] (was Re: Problem with reference) In-Reply-To: <49AD4CB2.8030705@pacific.edu> References: <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> <49ACD957.3020608@behnel.de> <877i37uks5.fsf@mulj.homelinux.net> <51518.213.61.181.86.1236073094.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <49ACFDB3.80002@egenix.com> <35237.213.61.181.86.1236091475.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <46331.213.61.181.86.1236093753.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <49AD4CB2.8030705@pacific.edu> Message-ID: <36063.213.61.181.86.1236095015.squirrel@groupware.dvs.informatik.tu-darmstadt.de> Ken Hughes wrote: > 1) Did anyone answer the original poster's question? Yes, two people (and one right in the first post, actually). > 2) If this is going to persist as an OT thread, change the subject line? I think "problem with reference" isn't that far from a good subject. ;-) Stefan From pycnweb at 126.com Wed Mar 4 03:55:46 2009 From: pycnweb at 126.com (pycnweb) Date: Wed, 4 Mar 2009 10:55:46 +0800 (CST) Subject: [capi-sig] (was Re: Problem with reference) In-Reply-To: <36063.213.61.181.86.1236095015.squirrel@groupware.dvs.informatik.tu-darmstadt.de> References: <36063.213.61.181.86.1236095015.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <27505012.140851236047272278.JavaMail.coremail@bj126app60.126.com> <49ACD957.3020608@behnel.de> <877i37uks5.fsf@mulj.homelinux.net> <51518.213.61.181.86.1236073094.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <49ACFDB3.80002@egenix.com> <35237.213.61.181.86.1236091475.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <46331.213.61.181.86.1236093753.squirrel@groupware.dvs.informatik.tu-darmstadt.de> <49AD4CB2.8030705@pacific.edu> Message-ID: <27812322.947631236135346159.JavaMail.coremail@bj126app74.126.com> 2009-03-03?23:43:35?"Stefan?Behnel"? wrote? >Ken?Hughes?wrote: >>?1)?Did?anyone?answer?the?original?poster's?question? > >Yes,?two?people?(and?one?right?in?the?first?post,?actually). > > >>?2)?If?this?is?going?to?persist?as?an?OT?thread,?change?the?subject?line? > >I?think?"problem?with?reference"?isn't?that?far?from?a?good?subject.?;-) > >Stefan > >_______________________________________________ >capi-sig?mailing?list >capi-sig at python.org >http://mail.python.org/mailman/listinfo/capi-sig Thanks for everybody's answer. Cython is a good assistant for writing c/c++ extension, but I it is indispensable to master Python/C API also. :) From hniksic at xemacs.org Wed Mar 11 10:22:11 2009 From: hniksic at xemacs.org (Hrvoje Niksic) Date: Wed, 11 Mar 2009 10:22:11 +0100 Subject: [capi-sig] super() in Python/C Message-ID: <8763igxvlo.fsf@busola.homelinux.net> After the initial controversy regarding Python's super(), I feel that it's now fairly well-understood when it should be used and what problems it solves. (Even if not, this list is probably not the right medium for objections to super.) The one question I haven't seen covered is whether and how to use it from C. All of the Python/C code I've seen calls its base class(es) directly, typically by only invoking the method of their superclass in the C layout sense. This means that such code will call into the superclass twice in a diamond inheritance scenario. For low-level classes that are not expected to be multiply inherited this is not a problem. But in some cases we need to convert higher-level classes from Python to C for efficiency, either using Python/C, or a higher-level C++ wrapping. The classes use super() with good reason, and I'd like to avoid breaking them during conversion to C. Is there a documented way to use super() from C, other than instantiating PySuper_Type? Has anyone tried to do that? From mal at egenix.com Wed Mar 11 11:42:51 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 11 Mar 2009 11:42:51 +0100 Subject: [capi-sig] super() in Python/C In-Reply-To: <8763igxvlo.fsf@busola.homelinux.net> References: <8763igxvlo.fsf@busola.homelinux.net> Message-ID: <49B795AB.4030002@egenix.com> On 2009-03-11 10:22, Hrvoje Niksic wrote: > After the initial controversy regarding Python's super(), I feel that > it's now fairly well-understood when it should be used and what > problems it solves. (Even if not, this list is probably not the right > medium for objections to super.) The one question I haven't seen > covered is whether and how to use it from C. > > All of the Python/C code I've seen calls its base class(es) directly, > typically by only invoking the method of their superclass in the C > layout sense. This means that such code will call into the superclass > twice in a diamond inheritance scenario. For low-level classes that > are not expected to be multiply inherited this is not a problem. But > in some cases we need to convert higher-level classes from Python to C > for efficiency, either using Python/C, or a higher-level C++ wrapping. > The classes use super() with good reason, and I'd like to avoid > breaking them during conversion to C. > > Is there a documented way to use super() from C, other than > instantiating PySuper_Type? Has anyone tried to do that? If you look at the C code for the builtins module, you'll find that super() is just an alias for the PySuper_Type, so instantiating this corresponds 1-1 to the Python logic. The complicated lookup code for the base method is not exposed in the typeobject.c file as C API, so there's no other way to access it. Perhaps there should be... most uses of super() will not cache the super-object and only use it once, so apart from the syntactic sugar added by having super() return the super-lookup-object in order to be able to reference the method in a more convenient way, there doesn't appear to be much reason for creating a new object just to call the base method. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Mar 11 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From csaba.balazs at researchstudio.at Mon Mar 30 16:59:36 2009 From: csaba.balazs at researchstudio.at (Csaba Balazs) Date: Mon, 30 Mar 2009 16:59:36 +0200 Subject: [capi-sig] CPython and C++ object GC Message-ID: <20090330165936.oqsvswo044ww4kkk@webmail.researchstudio.at> Hello Everybody, I would like to use a C++ gui library with the following (simplified) interface in Python. #include class Gui; class GuiObject { public: GuiObject(Gui *Gui) {printf("creating GuiObject(gui: %X)\n", Gui);} ~GuiObject() {printf("deleting GuiObject\n");} void Move(int x, int y) {printf("GuiObject move(%d, %d)\n", x, y);}; }; class Gui { public: Gui() {printf("creating Gui\n");} ~Gui() {printf("deleting Gui\n");} GuiObject* AddImage() { GuiObject* ob = new GuiObject(this); return ob; } void Print() {printf("Gui: %X\n", this);} }; int main() { Gui *gui = new Gui(); gui->Print(); GuiObject *obj = gui->AddImage(); obj->Move(50, 50); /*GuiObject *obj2 = new GuiObject(gui); // not allowed delete obj2;*/ delete obj; delete gui; return 0; } I created the Python Gui and GuiObject classes (PyTypeObject), and added it to main module (PyModule_AddObject). It works, but there is a problem at the Gui::AddImage(), with constructs a new GuiObject, which is available in Python layer but finally it is not collected and freed by GC: ... obj = _PyObject_New(&GuiObjectType); PyObject_Init(obj, &GuiObjectType); ... I cannot invoke the GuiObject object constructor directly from Python, because of the implementation of the C++ gui library (in this case it would be collected). I use the embedded CPython as an interpreter, so I cannot add additional external .py file for it. So the following Python code would be the target: gui = GUI(); background = gui.AddImage(); #background = GuiObject(gui); <-- Collected but not allowed background.ImageFile("bg.jpg"); background.Move(0, 0); ... How could I implement the AddImage function in order to be freed the constructed object at the end? Thanks in advance! All-the-best, Csaba