From ndbecker2 at gmail.com  Thu Dec  3 03:41:40 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Wed, 02 Dec 2009 21:41:40 -0500
Subject: [C++-sig] functools.partial vs boost::python
Message-ID: <hf78h5$pou$1@ger.gmane.org>

Has anyone noticed that a function created with boost::python using
args() to give keyword arguments doesn't seem to work with 
functools.partial keyword arguments (but does with positional args)?

For example, I have this function:
  class_<boost_uniform_real_wrap>
    ("uniform_real", "Uniform float distribution", 
bp::init<rng_t&,double,double>(
	(bp::arg ("rng"),
	 bp::arg ("min"),
	 bp::arg ("max"))...

Then:
from functools import partial
f = partial (uniform_real, rng=rng1) << using keyword doesn't work
f (1,2)
ArgumentError: Python argument types in
    uniform_real.__init__(uniform_real, int, int)
did not match C++ signature:
    __init__(_object*, boost::random::mersenne_twister<unsigned int, 
32, 624, 397, 31, 2567483615u, 11, 7, 2636928640u, 15, 4022730752u, 
18, 3346425566u> {lvalue} rng, double min, double max)

But this works:
from functools import partial
f = partial (uniform_real, rng1) << pos arg does work

In [27]: f(1,2)
Out[27]: uniform_real(1,2)



From rachimow at googlemail.com  Thu Dec  3 13:11:41 2009
From: rachimow at googlemail.com (meiko rachimow)
Date: Thu, 3 Dec 2009 13:11:41 +0100
Subject: [C++-sig] boost.python: call python function with kwargs from c++
Message-ID: <6a5b23630912030411m5be57968t9d8a2a7baf43fb9c@mail.gmail.com>

Hi

How can I call a function defined in a python module from c++,
if I have to give **kwargs parameter...

example.py:

def hello(**kwargs):
    for key, value in kwargs.items():
        print key, " - ",  value


c++:
using namespace boost::python;

object fun = import("example").attr("hello")
fun()

int number = 42
std::wstring name = "M?hre"

dict kwargs;
kwargs["Number"] = number;
kwargs["Name"] = name;

fun(...) ???

What should i do ?

Meiko
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091203/f7b0db6e/attachment.htm>

From seefeld at sympatico.ca  Thu Dec  3 14:11:36 2009
From: seefeld at sympatico.ca (Stefan Seefeld)
Date: Thu, 03 Dec 2009 08:11:36 -0500
Subject: [C++-sig] boost.python: call python function with kwargs from
 c++
In-Reply-To: <6a5b23630912030411m5be57968t9d8a2a7baf43fb9c@mail.gmail.com>
References: <6a5b23630912030411m5be57968t9d8a2a7baf43fb9c@mail.gmail.com>
Message-ID: <4B17B908.7000003@sympatico.ca>

On 12/03/2009 07:11 AM, meiko rachimow wrote:
> Hi
>
> How can I call a function defined in a python module from c++,
> if I have to give **kwargs parameter...
>
> example.py:
>
> def hello(**kwargs):
>     for key, value in kwargs.items():
>         print key, " - ",  value
>
>
> c++:
> using namespace boost::python;
>
> object fun = import("example").attr("hello")
> fun()
>
> int number = 42
> std::wstring name = "M?hre"
>
> dict kwargs;
> kwargs["Number"] = number;
> kwargs["Name"] = name;
>
> fun(...) ???
>
> What should i do ?


fun(**kwargs);


     Stefan

-- 

       ...ich hab' noch einen Koffer in Berlin...


From troy at resophonic.com  Thu Dec  3 19:07:55 2009
From: troy at resophonic.com (troy d. straszheim)
Date: Thu, 03 Dec 2009 13:07:55 -0500
Subject: [C++-sig] functools.partial vs boost::python
In-Reply-To: <hf78h5$pou$1@ger.gmane.org>
References: <hf78h5$pou$1@ger.gmane.org>
Message-ID: <4B17FE7B.6000602@resophonic.com>

Neal Becker wrote:
> Has anyone noticed that a function created with boost::python using
> args() to give keyword arguments doesn't seem to work with 
> functools.partial keyword arguments (but does with positional args)?
> 
> For example, I have this function:
>   class_<boost_uniform_real_wrap>
>     ("uniform_real", "Uniform float distribution", 
> bp::init<rng_t&,double,double>(
> 	(bp::arg ("rng"),
> 	 bp::arg ("min"),
> 	 bp::arg ("max"))...
> 
> Then:
> from functools import partial
> f = partial (uniform_real, rng=rng1) << using keyword doesn't work
> f (1,2)
> ArgumentError: Python argument types in
>     uniform_real.__init__(uniform_real, int, int)
> did not match C++ signature:
>     __init__(_object*, boost::random::mersenne_twister<unsigned int, 
> 32, 624, 397, 31, 2567483615u, 11, 7, 2636928640u, 15, 4022730752u, 
> 18, 3346425566u> {lvalue} rng, double min, double max)
> 
> But this works:
> from functools import partial
> f = partial (uniform_real, rng1) << pos arg does work
> 
> In [27]: f(1,2)
> Out[27]: uniform_real(1,2)
> 

That doesn't work for pure python functions either:

 >>> def f(x,y,z): return x*100 + y*10 + z
...
 >>> from functools import partial as p
 >>> p(f,x=1)(2,3)
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: f() got multiple values for keyword argument 'x'
 >>> p(f,x=1)(y=2,z=3)
123
 >>> p(f,1)(2,3)
123

The error message is misleading for sure.  Boost.python is going through 
a list of overloads and trying them in order; if it runs out of 
overloads, it says nothing matched.  I think I can get it to be smarter 
about this, lemme see...

-t




From troy at resophonic.com  Thu Dec  3 22:17:40 2009
From: troy at resophonic.com (troy d. straszheim)
Date: Thu, 03 Dec 2009 16:17:40 -0500
Subject: [C++-sig] boost::python overload resolution redux
In-Reply-To: <4B17FE7B.6000602@resophonic.com>
References: <hf78h5$pou$1@ger.gmane.org> <4B17FE7B.6000602@resophonic.com>
Message-ID: <4B182AF4.5050900@resophonic.com>

troy d. straszheim wrote:
> 
> That doesn't work for pure python functions either:
> 
>  >>> def f(x,y,z): return x*100 + y*10 + z
> ...
>  >>> from functools import partial as p
>  >>> p(f,x=1)(2,3)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: f() got multiple values for keyword argument 'x'
>  >>> p(f,x=1)(y=2,z=3)
> 123
>  >>> p(f,1)(2,3)
> 123
> 
> The error message is misleading for sure.  Boost.python is going through 
> a list of overloads and trying them in order; if it runs out of 
> overloads, it says nothing matched.  I think I can get it to be smarter 
> about this, lemme see...
> 

Trac ticket here:  https://svn.boost.org/trac/boost/ticket/3710

For the record, here's a variation on the same theme.  It doesn't have 
anything to do with functools, it is another symptom of our first-match 
overload resolution algorithm:

int addem(int x, int y, int z) { return x*100 + y*10 + z; }

BOOST_PYTHON_MODULE(functools_playnice_ext)
{
   def("addem", &addem, (arg("x"), arg("y"), arg("z")));
}


 >>> from functools_playnice_ext import addem
 >>> addem(1, 8, 2, x=4)
Traceback (most recent call last):
...
ArgumentError: Python argument types in
     functools_playnice_ext.addem(int, int, int)
did not match C++ signature:
     addem(int x, int y, int z)


I've been thinking about this off and on for a while, to fix it right 
isn't trivial.  I could put in a few checks for cases where a function 
is not overloaded, but that feels like hackery.


-t





From ndbecker2 at gmail.com  Fri Dec  4 02:17:06 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Thu, 03 Dec 2009 20:17:06 -0500
Subject: [C++-sig] boost::python overload resolution redux
References: <hf78h5$pou$1@ger.gmane.org> <4B17FE7B.6000602@resophonic.com>
	<4B182AF4.5050900@resophonic.com>
Message-ID: <hf9nui$1ie$1@ger.gmane.org>

troy d. straszheim wrote:

...
> 
> I've been thinking about this off and on for a while, to fix it right
> isn't trivial.  I could put in a few checks for cases where a function
> is not overloaded, but that feels like hackery.
> 
No ideas, but I appreciate you're looking into this.


From rachimow at googlemail.com  Mon Dec  7 14:35:02 2009
From: rachimow at googlemail.com (meiko rachimow)
Date: Mon, 7 Dec 2009 14:35:02 +0100
Subject: [C++-sig] Cplusplus-sig Digest, Vol 15, Issue 3
In-Reply-To: <mailman.28.1259924404.22852.cplusplus-sig@python.org>
References: <mailman.28.1259924404.22852.cplusplus-sig@python.org>
Message-ID: <6a5b23630912070535pfb41e45wc75a38d41e3a79ac@mail.gmail.com>

when i: "fun(**kwargs)"
I get following runtime error:
TypeError: No to_python (by-value) converter found for C++ type:
boost::python::detail::kwds_proxy

thx again

meiko



> Message: 1
> Date: Thu, 3 Dec 2009 13:11:41 +0100
> From: meiko rachimow <rachimow at googlemail.com>
> To: cplusplus-sig at python.org
> Subject: [C++-sig] boost.python: call python function with kwargs from
>        c++
> Message-ID:
>        <6a5b23630912030411m5be57968t9d8a2a7baf43fb9c at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
>
> Hi
>
> How can I call a function defined in a python module from c++,
> if I have to give **kwargs parameter...
>
> example.py:
>
> def hello(**kwargs):
>    for key, value in kwargs.items():
>        print key, " - ",  value
>
>
> c++:
> using namespace boost::python;
>
> object fun = import("example").attr("hello")
> fun()
>
> int number = 42
> std::wstring name = "M?hre"
>
> dict kwargs;
> kwargs["Number"] = number;
> kwargs["Name"] = name;
>
> fun(...) ???
>
> What should i do ?
>
> Meiko
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> http://mail.python.org/pipermail/cplusplus-sig/attachments/20091203/f7b0db6e/attachment-0001.htm
> >
>
> ------------------------------
>
> Message: 2
> Date: Thu, 03 Dec 2009 08:11:36 -0500
> From: Stefan Seefeld <seefeld at sympatico.ca>
> To: cplusplus-sig at python.org
> Subject: Re: [C++-sig] boost.python: call python function with kwargs
>        from c++
> Message-ID: <4B17B908.7000003 at sympatico.ca>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> On 12/03/2009 07:11 AM, meiko rachimow wrote:
> > Hi
> >
> > How can I call a function defined in a python module from c++,
> > if I have to give **kwargs parameter...
> >
> > example.py:
> >
> > def hello(**kwargs):
> >     for key, value in kwargs.items():
> >         print key, " - ",  value
> >
> >
> > c++:
> > using namespace boost::python;
> >
> > object fun = import("example").attr("hello")
> > fun()
> >
> > int number = 42
> > std::wstring name = "M?hre"
> >
> > dict kwargs;
> > kwargs["Number"] = number;
> > kwargs["Name"] = name;
> >
> > fun(...) ???
> >
> > What should i do ?
>
>
> fun(**kwargs);
>
>
>     Stefan
>
> --
>
>       ...ich hab' noch einen Koffer in Berlin...
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091207/1e70d561/attachment.htm>

From alok.jadhav at credit-suisse.com  Tue Dec  8 01:01:57 2009
From: alok.jadhav at credit-suisse.com (Jadhav, Alok)
Date: Tue, 8 Dec 2009 08:01:57 +0800
Subject: [C++-sig] Python C++ extension (multi threaded) raises access
	violation exception
Message-ID: <8037240182E2574D8E8706A2CB3D99B1084D3115@ESNG17P32002A.csfb.cs-group.com>

I am using SWIG to extend Python to C++.   But the questions I have
consists of basic Python C API related. I hope this list will be able to
help me. 

I am trying to solve a simple problem which many of you must have
encountered. I wonder what I am doing different than others that I am
facing a problem.

Background
-----------

A c++ library is wrapped in my C++ Class. This new C++ class is extended
in Python using SWIG. I have to call a python function from C++ code (
which is on a new thread spawned from C++ main thread).

Issue
-----
I am able to call the function. No issues. I could pass basic data types
such as int. No issues. But when I try to pass a C++ class object (eg
RFAMessageWrapper) to my python function I get an exception. It is an
Access violation exception. The memory could not be "read".  I am not
even able to catch this exception.



Some code snippets
--------------------

C++ function which calls python function. ... 

[code]
static void PythonCallBack(RFAMessage *msg, void *clientdata)
{
   PyObject *func, *arglist;
   int blah = 1;
  
   PyGILState_STATE state; 
   state = PyGILState_Ensure();
   func = (PyObject *) clientdata;               // Get Python function
   RFAMessageWrapper msg2(msg);
	try {
		arglist = Py_BuildValue("(O)",msg2);             //
Build argument list
		//arglist = Py_BuildValue("(i)", blah);
		PyEval_CallObject(func,arglist);     // Call Python
	} catch (...) {
		cout<<"Unknown exception..."<<endl;
	}
  Py_XDECREF(arglist);                           // Trash arglist
   PyGILState_Release(state); 
}
[/code]


Python function call..

[code]
def callback_fun(msg):
    try:
        print "RFAMessage received for service:"+msg.service
    except Exception:
        print "Exception handled"
[/code]

Whenever I try to access msg (of type RFAMessageWrapper .. C++ object) I
get exception. msg is not None. I checked that in if condition. even
type(msg) raises exception.
RFAMessageWrapper class has been wrapped properly by SWIG as I could
create an object manually and use it in Python. But only when I pass it
from C++ to python I get this exception.


I have spent long time to solve this issue but in vain. I hope I get
some help from this Forum.


Regards, 
Alok

=============================================================================== 
 Please access the attached hyperlink for an important electronic communications disclaimer: 
 http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html 
 =============================================================================== 
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091208/0a136644/attachment.htm>

From gjcarneiro at gmail.com  Tue Dec  8 01:18:40 2009
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Tue, 8 Dec 2009 00:18:40 +0000
Subject: [C++-sig] Python C++ extension (multi threaded) raises access
	violation exception
In-Reply-To: <8037240182E2574D8E8706A2CB3D99B1084D3115@ESNG17P32002A.csfb.cs-group.com>
References: <8037240182E2574D8E8706A2CB3D99B1084D3115@ESNG17P32002A.csfb.cs-group.com>
Message-ID: <a467ca4f0912071618o3248605g954bac9912faa040@mail.gmail.com>

2009/12/8 Jadhav, Alok <alok.jadhav at credit-suisse.com>

>  I am using SWIG to extend Python to C++.   But the questions I have
> consists of basic Python C API related. I hope this list will be able to
> help me.
>
> I am trying to solve a simple problem which many of you must have
> encountered. I wonder what I am doing different than others that I am facing
> a problem.
>
> Background
> -----------
>
> A c++ library is wrapped in my C++ Class. This new C++ class is extended in
> Python using SWIG. I have to call a python function from C++ code ( which is
> on a new thread spawned from C++ main thread).
>
> Issue
> -----
> I am able to call the function. No issues. I could pass basic data types
> such as int. No issues. But when I try to pass a C++ class object (eg
> RFAMessageWrapper) to my python function I get an exception. It is an Access
> violation exception. The memory could not be "read".  I am not even able to
> catch this exception.
>
>
> Some code snippets
> --------------------
>
> C++ function which calls python function. ...
>
> [code]
> static void PythonCallBack(RFAMessage *msg, void *clientdata)
> {
>    PyObject *func, *arglist;
>    int blah = 1;
>
>    PyGILState_STATE state;
>    state = PyGILState_Ensure();
>    func = (PyObject *) clientdata;               // Get Python function
>    RFAMessageWrapper msg2(msg);
>         try {
>                 arglist = Py_BuildValue("(O)",msg2);             // Build
> argument list
>
This is bad.   Py_BuildValue("(O)", is expecting to find a PyObject, but msg2
is not a PyObject.

Additionally, PyObjects must be created with PyObject_New, and never stack
allocated like msg2 is.

I cannot offer any solution, as it would be SWIG specific and I am not
familiar with SWIG.


>                 //arglist = Py_BuildValue("(i)", blah);
>                 PyEval_CallObject(func,arglist);     // Call Python
>         } catch (...) {
>                 cout<<"Unknown exception..."<<endl;
>         }
>   Py_XDECREF(arglist);                           // Trash arglist
>    PyGILState_Release(state);
> }
> [/code]
>
> Python function call..
>
> [code]
> def callback_fun(msg):
>     try:
>         print "RFAMessage received for service:"+msg.service
>     except Exception:
>         print "Exception handled"
> [/code]
>
> Whenever I try to access msg (of type RFAMessageWrapper .. C++ object) I
> get exception. msg is not None. I checked that in if condition. even
> type(msg) raises exception.
>
> RFAMessageWrapper class has been wrapped properly by SWIG as I could create
> an object manually and use it in Python. But only when I pass it from C++ to
> python I get this exception.
>
> I have spent long time to solve this issue but in vain. I hope I get some
> help from this Forum.
>
> Regards,
> Alok
>
>
> ==============================================================================
> Please access the attached hyperlink for an important electronic
> communications disclaimer:
> http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
>
> ==============================================================================
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091208/27f3f341/attachment-0001.htm>

From Nikolaus at rath.org  Tue Dec  8 16:55:43 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Tue, 08 Dec 2009 10:55:43 -0500
Subject: [C++-sig] [Py++] Generate Python code for ctypes?
Message-ID: <87y6ldv4sg.fsf@vostro.rath.org>

Hello,

Is it possible to use Py++ to "convert" structures from a C header,  say

,----
| struct flock
|   {
|     short int l_type;   /* Type of lock: F_RDLCK, F_WRLCK, or F_UNLCK.  */
|     short int l_whence; /* Where `l_start' is relative to (like `lseek').  */
| #ifndef __USE_FILE_OFFSET64
|     __off_t l_start;    /* Offset where the lock begins.  */
|     __off_t l_len;      /* Size of the locked area; zero means until EOF.  */
| #else
|     __off64_t l_start;  /* Offset where the lock begins.  */
|     __off64_t l_len;    /* Size of the locked area; zero means until EOF.  */
| #endif
|     __pid_t l_pid;      /* Process holding the lock.  */
|   };
`----

from fcntl.h into the corresponding ctypes Structure,
  
,----
| class flock_t(Structure):
|     _fields_ = [
|                 ('l_type', c_short),
|                 ('l_whence', c_short),
|                 ('l_start', c_uint64),
|                 ('l_len', c_uint64),
|                 ('l_pid', c_int)
|                 ]
`----

while automatically taking into account the #ifdefs and how types like
off_t and pid_t are actually defined on the target system?

The introduction on
http://www.language-binding.net/pyplusplus/pyplusplus.html says that
Py++ can act as a code generator for ctypes, but I could not find any
documentation for the sort of application that I describe here.


Best,


   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From roman.yakovenko at gmail.com  Tue Dec  8 20:42:49 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Tue, 8 Dec 2009 21:42:49 +0200
Subject: [C++-sig] [Py++] Generate Python code for ctypes?
In-Reply-To: <87y6ldv4sg.fsf@vostro.rath.org>
References: <87y6ldv4sg.fsf@vostro.rath.org>
Message-ID: <7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>

On Tue, Dec 8, 2009 at 5:55 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>
> Hello,
>
> Is it possible to use Py++ to "convert" structures from a C header, ?say
>
> ,----
> | struct flock
> | ? {
> | ? ? short int l_type; ? /* Type of lock: F_RDLCK, F_WRLCK, or F_UNLCK. ?*/
> | ? ? short int l_whence; /* Where `l_start' is relative to (like `lseek'). ?*/
> | #ifndef __USE_FILE_OFFSET64
> | ? ? __off_t l_start; ? ?/* Offset where the lock begins. ?*/
> | ? ? __off_t l_len; ? ? ?/* Size of the locked area; zero means until EOF. ?*/
> | #else
> | ? ? __off64_t l_start; ?/* Offset where the lock begins. ?*/
> | ? ? __off64_t l_len; ? ?/* Size of the locked area; zero means until EOF. ?*/
> | #endif
> | ? ? __pid_t l_pid; ? ? ?/* Process holding the lock. ?*/
> | ? };
> `----
>
> from fcntl.h into the corresponding ctypes Structure,
>
> ,----
> | class flock_t(Structure):
> | ? ? _fields_ = [
> | ? ? ? ? ? ? ? ? ('l_type', c_short),
> | ? ? ? ? ? ? ? ? ('l_whence', c_short),
> | ? ? ? ? ? ? ? ? ('l_start', c_uint64),
> | ? ? ? ? ? ? ? ? ('l_len', c_uint64),
> | ? ? ? ? ? ? ? ? ('l_pid', c_int)
> | ? ? ? ? ? ? ? ? ]
> `----
>
> while automatically taking into account the #ifdefs and how types like
> off_t and pid_t are actually defined on the target system?

No, the generated code, will only work on the platform with the same
struct definition.

> The introduction on
> http://www.language-binding.net/pyplusplus/pyplusplus.html says that
> Py++ can act as a code generator for ctypes, but I could not find any
> documentation for the sort of application that I describe here.

Unfortunately I don't have time to write one. I am extremely busy this days.

You can take a look on gmplib example:
http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/examples/gmplib_dev/
It is fully working.

You also can take a look on "wrap_library" "executable":
http://sourceforge.net/projects/pygccxml/files/ctypes%20code%20generator/initial%20release/wrap_library.zip/download

It creates ctypes code, using default settings.

HTH
--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From alok.jadhav at credit-suisse.com  Wed Dec  9 02:13:04 2009
From: alok.jadhav at credit-suisse.com (Jadhav, Alok)
Date: Wed, 9 Dec 2009 09:13:04 +0800
Subject: [C++-sig] Converintg C++ object to python object
In-Reply-To: <87y6ldv4sg.fsf@vostro.rath.org>
References: <87y6ldv4sg.fsf@vostro.rath.org>
Message-ID: <8037240182E2574D8E8706A2CB3D99B1084D312D@ESNG17P32002A.csfb.cs-group.com>

In my C++ callback function below

[code]

static void PythonCallBack(RFAMessage *msg, void *clientdata)
{
   PyObject *func, *arglist;
   PyGILState_STATE state; 
   state = PyGILState_Ensure();
   func = (PyObject *) clientdata;               // Get Python function
   RFAMessageWrapper msg2(msg);
 
   PyObject *argobj = SWIG_NewPointerObj(SWIG_as_voidptr(&msg2),
SWIGTYPE_p_RFAMessageWrapper , SWIG_POINTER_NEW |  0 );
  
   arglist = Py_BuildValue("(O)",argobj);             // Build argument
list
   PyEval_CallObject(func,arglist);     // Call Python
  Py_XDECREF(arglist);                           // Trash arglist
   //SWIG_PYTHON_THREAD_END_BLOCK;
   PyGILState_Release(state); 
}

[/code]

Can you tell me if the usage of  SWIG_NewPointerObj is correct? I want
to convert a C++ class object into python equivalent object. In the
python code, the passed object is identified as SwigPyObject and not
RFAMessageWrapper which I am expecting.

How can I convert RFAMessageWrapper object into its Python equivalent to
that in my Python function I can use the passed object as
RFAMessageWrapper.

Regards, 
Alok

=============================================================================== 
 Please access the attached hyperlink for an important electronic communications disclaimer: 
 http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html 
 =============================================================================== 
 

From alok.jadhav at credit-suisse.com  Wed Dec  9 05:54:53 2009
From: alok.jadhav at credit-suisse.com (Jadhav, Alok)
Date: Wed, 9 Dec 2009 12:54:53 +0800
Subject: [C++-sig] Converintg C++ object to python object
In-Reply-To: <8037240182E2574D8E8706A2CB3D99B1084D312D@ESNG17P32002A.csfb.cs-group.com>
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<8037240182E2574D8E8706A2CB3D99B1084D312D@ESNG17P32002A.csfb.cs-group.com>
Message-ID: <8037240182E2574D8E8706A2CB3D99B1084D3137@ESNG17P32002A.csfb.cs-group.com>


When I pass a C++ object to Python function, I want to be able to use
that object using its Proxy class in the python code. How can I achieve
that?

Regards, Alok

 

-----Original Message-----
From: cplusplus-sig-bounces+alok.jadhav=credit-suisse.com at python.org
[mailto:cplusplus-sig-bounces+alok.jadhav=credit-suisse.com at python.org]
On Behalf Of Jadhav, Alok
Sent: Wednesday, December 09, 2009 10:13 AM
To: Development of Python/C++ integration
Subject: [C++-sig] Converintg C++ object to python object

In my C++ callback function below

[code]

static void PythonCallBack(RFAMessage *msg, void *clientdata) {
   PyObject *func, *arglist;
   PyGILState_STATE state; 
   state = PyGILState_Ensure();
   func = (PyObject *) clientdata;               // Get Python function
   RFAMessageWrapper msg2(msg);
 
   PyObject *argobj = SWIG_NewPointerObj(SWIG_as_voidptr(&msg2),
SWIGTYPE_p_RFAMessageWrapper , SWIG_POINTER_NEW |  0 );
  
   arglist = Py_BuildValue("(O)",argobj);             // Build argument
list
   PyEval_CallObject(func,arglist);     // Call Python
  Py_XDECREF(arglist);                           // Trash arglist
   //SWIG_PYTHON_THREAD_END_BLOCK;
   PyGILState_Release(state);
}

[/code]

Can you tell me if the usage of  SWIG_NewPointerObj is correct? I want
to convert a C++ class object into python equivalent object. In the
python code, the passed object is identified as SwigPyObject and not
RFAMessageWrapper which I am expecting.

How can I convert RFAMessageWrapper object into its Python equivalent to
that in my Python function I can use the passed object as
RFAMessageWrapper.

Regards,
Alok

========================================================================
=======
 Please access the attached hyperlink for an important electronic
communications disclaimer: 
 http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
 
========================================================================
======= 
 
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

=============================================================================== 
 Please access the attached hyperlink for an important electronic communications disclaimer: 
 http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html 
 =============================================================================== 
 

From boost-python at lonely-star.org  Wed Dec  9 13:25:24 2009
From: boost-python at lonely-star.org (Nathan Huesken)
Date: Wed, 9 Dec 2009 13:25:24 +0100
Subject: [C++-sig] extracting a numpy.int32 type
Message-ID: <20091209122524.GB5233@SamZwo.iwr.uni-heidelberg.de>

Hi,

I have something like this in my code:
void function(numeric::array& ids)
{
    ...
    int x=extract<int>(ids[make_tuple(pos)]);
    ...
}

Now I plug in a numpy arra with dtype=np.int32.
I get:

No registered converter was able to produce a C++ rvalue of type int
from this Python object of type numpy.int32

It also happens for numpy.int64. But if I extract i.E. a double from a
numpy.float64 array, it works.

What can I do here?
Thanks!
Nathan

From diepen at astron.nl  Wed Dec  9 16:29:19 2009
From: diepen at astron.nl (Ger van Diepen)
Date: Wed, 09 Dec 2009 16:29:19 +0100
Subject: [C++-sig] extracting a numpy.int32 type
In-Reply-To: <20091209122524.GB5233@SamZwo.iwr.uni-heidelberg.de>
References: <20091209122524.GB5233@SamZwo.iwr.uni-heidelberg.de>
Message-ID: <4B1FD051.6086.00A9.0@astron.nl>

Hi Nathan,

We have converters written at the end of 2006. We never needed to convert them to the more powerful newer versions of Boost-Python as they still build fine. So they might look outdated.

They can convert scalars, sequences, dicts, and numpy arrays of many types, also all numeric numpy types. So it can convert, say, an element of a numpy int32 array to a C++ float scalar. Vice-versa a C++ numeric value is always converted to a regular Python numeric type.
They work well on both 32bit and 64bit platforms.

If you want to see more you can find the code at pyrap.googlecode.com.

Cheers,
Ger



From Nikolaus at rath.org  Wed Dec  9 19:02:27 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Wed, 09 Dec 2009 13:02:27 -0500
Subject: [C++-sig] [Py++] Generate Python code for ctypes?
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
Message-ID: <877hswm3f0.fsf@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
> On Tue, Dec 8, 2009 at 5:55 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>>
>> Hello,
>>
>> Is it possible to use Py++ to "convert" structures from a C header, ?say
>>
>> ,----
>> | struct flock
>> | ? {
>> | ? ? short int l_type; ? /* Type of lock: F_RDLCK, F_WRLCK, or F_UNLCK. ?*/
>> | ? ? short int l_whence; /* Where `l_start' is relative to (like `lseek'). ?*/
>> | #ifndef __USE_FILE_OFFSET64
>> | ? ? __off_t l_start; ? ?/* Offset where the lock begins. ?*/
>> | ? ? __off_t l_len; ? ? ?/* Size of the locked area; zero means until EOF. ?*/
>> | #else
>> | ? ? __off64_t l_start; ?/* Offset where the lock begins. ?*/
>> | ? ? __off64_t l_len; ? ?/* Size of the locked area; zero means until EOF. ?*/
>> | #endif
>> | ? ? __pid_t l_pid; ? ? ?/* Process holding the lock. ?*/
>> | ? };
>> `----
>>
>> from fcntl.h into the corresponding ctypes Structure,
>>
>> ,----
>> | class flock_t(Structure):
>> | ? ? _fields_ = [
>> | ? ? ? ? ? ? ? ? ('l_type', c_short),
>> | ? ? ? ? ? ? ? ? ('l_whence', c_short),
>> | ? ? ? ? ? ? ? ? ('l_start', c_uint64),
>> | ? ? ? ? ? ? ? ? ('l_len', c_uint64),
>> | ? ? ? ? ? ? ? ? ('l_pid', c_int)
>> | ? ? ? ? ? ? ? ? ]
>> `----
>>
>> while automatically taking into account the #ifdefs and how types like
>> off_t and pid_t are actually defined on the target system?
>
> No, the generated code, will only work on the platform with the same
> struct definition.

So as long as I generate the code on the local platform (e.g. during
package installation) I will be fine, right? That's enough for me.

>> The introduction on
>> http://www.language-binding.net/pyplusplus/pyplusplus.html says that
>> Py++ can act as a code generator for ctypes, but I could not find any
>> documentation for the sort of application that I describe here.
>
> Unfortunately I don't have time to write one. I am extremely busy this days.
>
> You can take a look on gmplib example:
> http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/examples/gmplib_dev/
> It is fully working.

This looks really interesting. I will try to modify that for use with my
library. Thanks for the link!


> You also can take a look on "wrap_library" "executable":
> http://sourceforge.net/projects/pygccxml/files/ctypes%20code%20generator/initial%20release/wrap_library.zip/download
>
> It creates ctypes code, using default settings.

This file seems to contain only lots of binary files, but no
documentation or .py files, so I am a bit lost. What exactly is it
supposed to do?


Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From alok.jadhav at credit-suisse.com  Thu Dec 10 02:39:10 2009
From: alok.jadhav at credit-suisse.com (Jadhav, Alok)
Date: Thu, 10 Dec 2009 09:39:10 +0800
Subject: [C++-sig] Converintg C++ object to python object
In-Reply-To: <8037240182E2574D8E8706A2CB3D99B1084D3137@ESNG17P32002A.csfb.cs-group.com>
References: <87y6ldv4sg.fsf@vostro.rath.org><8037240182E2574D8E8706A2CB3D99B1084D312D@ESNG17P32002A.csfb.cs-group.com>
	<8037240182E2574D8E8706A2CB3D99B1084D3137@ESNG17P32002A.csfb.cs-group.com>
Message-ID: <8037240182E2574D8E8706A2CB3D99B1084D3145@ESNG17P32002A.csfb.cs-group.com>

I SOLVED IT ... 
thanks for people who provided me help .. but i believe solved it
myself. .haha .. took me long time.. 

so .. the solution is something like this.. 

in my c function ..which calls python call .. 

[code]
PyObject *argobj = SWIG_NewPointerObj(SWIG_as_voidptr(&msg2),
SWIGTYPE_p_RFAMessageWrapper, SWIG_POINTER_NEW |  0 );
 try {
  arglist = Py_BuildValue("(O)",argobj);             // Build argument
list
  PyEval_CallObject(func,arglist);     // Call Python
 } catch (...) {
  cout<<"Unknown exception..."<<endl;
 }
[\code]

I am using 
PyObject *argobj = SWIG_NewPointerObj(SWIG_as_voidptr(&msg2),
SWIGTYPE_p_RFAMessageWrapper, SWIG_POINTER_NEW |  0 );

This was incorrect... i guess SWIG_POINTER_NEW is used to create pointer
to the actual c++ object where as if I pass 0

PyObject *argobj = SWIG_NewPointerObj(SWIG_as_voidptr(&msg2),
SWIGTYPE_p_RFAMessageWrapper, 0 |  0 );

This actually creates a pointer to the wrapped python object :)

I could access my object in my python code now.


I had to include std_string.i to enable string typemaps :)


Thanks,
Alok 

-----Original Message-----
From: cplusplus-sig-bounces+alok.jadhav=credit-suisse.com at python.org
[mailto:cplusplus-sig-bounces+alok.jadhav=credit-suisse.com at python.org]
On Behalf Of Jadhav, Alok
Sent: Wednesday, December 09, 2009 1:55 PM
To: Development of Python/C++ integration
Subject: Re: [C++-sig] Converintg C++ object to python object


When I pass a C++ object to Python function, I want to be able to use
that object using its Proxy class in the python code. How can I achieve
that?

Regards, Alok

 

-----Original Message-----
From: cplusplus-sig-bounces+alok.jadhav=credit-suisse.com at python.org
[mailto:cplusplus-sig-bounces+alok.jadhav=credit-suisse.com at python.org]
On Behalf Of Jadhav, Alok
Sent: Wednesday, December 09, 2009 10:13 AM
To: Development of Python/C++ integration
Subject: [C++-sig] Converintg C++ object to python object

In my C++ callback function below

[code]

static void PythonCallBack(RFAMessage *msg, void *clientdata) {
   PyObject *func, *arglist;
   PyGILState_STATE state; 
   state = PyGILState_Ensure();
   func = (PyObject *) clientdata;               // Get Python function
   RFAMessageWrapper msg2(msg);
 
   PyObject *argobj = SWIG_NewPointerObj(SWIG_as_voidptr(&msg2),
SWIGTYPE_p_RFAMessageWrapper , SWIG_POINTER_NEW |  0 );
  
   arglist = Py_BuildValue("(O)",argobj);             // Build argument
list
   PyEval_CallObject(func,arglist);     // Call Python
  Py_XDECREF(arglist);                           // Trash arglist
   //SWIG_PYTHON_THREAD_END_BLOCK;
   PyGILState_Release(state);
}

[/code]

Can you tell me if the usage of  SWIG_NewPointerObj is correct? I want
to convert a C++ class object into python equivalent object. In the
python code, the passed object is identified as SwigPyObject and not
RFAMessageWrapper which I am expecting.

How can I convert RFAMessageWrapper object into its Python equivalent to
that in my Python function I can use the passed object as
RFAMessageWrapper.

Regards,
Alok

========================================================================
=======
 Please access the attached hyperlink for an important electronic
communications disclaimer: 
 http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
 
========================================================================
======= 
 
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

========================================================================
=======
 Please access the attached hyperlink for an important electronic
communications disclaimer: 
 http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html
 
========================================================================
======= 
 
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

=============================================================================== 
 Please access the attached hyperlink for an important electronic communications disclaimer: 
 http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html 
 =============================================================================== 
 

From rogeeff at gmail.com  Thu Dec 10 09:25:59 2009
From: rogeeff at gmail.com (Gennadiy Rozental)
Date: Thu, 10 Dec 2009 08:25:59 +0000 (UTC)
Subject: [C++-sig] export class constant
Message-ID: <loom.20091210T092233-383@post.gmane.org>

Hi,

Let's say I have string literal as static C++ class member:

string Foo::abc = "ABC";

I'm exporting the class Foo. I'd like to export Foo::abc as well and be able to
access it on Python side using similar interface: Foo.abc.

Can I do this using Boost.Python?

Gennadiy


From troy at resophonic.com  Thu Dec 10 15:06:40 2009
From: troy at resophonic.com (troy d. straszheim)
Date: Thu, 10 Dec 2009 09:06:40 -0500
Subject: [C++-sig] export class constant
In-Reply-To: <loom.20091210T092233-383@post.gmane.org>
References: <loom.20091210T092233-383@post.gmane.org>
Message-ID: <4B210070.6010608@resophonic.com>

Gennadiy Rozental wrote:
> Hi,
> 
> Let's say I have string literal as static C++ class member:
> 
> string Foo::abc = "ABC";
> 
> I'm exporting the class Foo. I'd like to export Foo::abc as well and be able to
> access it on Python side using similar interface: Foo.abc.
> 
> Can I do this using Boost.Python?
> 

Sure, you just pass a pointer/ref to the static to def_readonly().  See the data_members test.

-t

 

From boost-python at lonely-star.org  Thu Dec 10 16:17:03 2009
From: boost-python at lonely-star.org (Nathan Huesken)
Date: Thu, 10 Dec 2009 16:17:03 +0100
Subject: [C++-sig] extracting a numpy.int32 type
In-Reply-To: <4B1FD051.6086.00A9.0@astron.nl>
References: <20091209122524.GB5233@SamZwo.iwr.uni-heidelberg.de>
	<4B1FD051.6086.00A9.0@astron.nl>
Message-ID: <20091210151703.GB22600@SamZwo.urz.uni-heidelberg.de>

Hi,

Thanks for the reply.

But still something, I would like to understand:

If I want to convert numpy.float64 -> double, it works without problems.
If I want to convert numpy.int32 -> int, I seem to need to register a
new converte ...

Why is this?

Thanks!
Nathan

On Wed, Dec 09, 2009 at 04:29:19PM +0100, Ger van Diepen wrote:
> Hi Nathan,
> 
> We have converters written at the end of 2006. We never needed to convert them to the more powerful newer versions of Boost-Python as they still build fine. So they might look outdated.
> 
> They can convert scalars, sequences, dicts, and numpy arrays of many types, also all numeric numpy types. So it can convert, say, an element of a numpy int32 array to a C++ float scalar. Vice-versa a C++ numeric value is always converted to a regular Python numeric type.
> They work well on both 32bit and 64bit platforms.
> 
> If you want to see more you can find the code at pyrap.googlecode.com.
> 
> Cheers,
> Ger
> 
> 

From zookog at gmail.com  Thu Dec 10 19:15:10 2009
From: zookog at gmail.com (Zooko O'Whielacronx)
Date: Thu, 10 Dec 2009 11:15:10 -0700
Subject: [C++-sig] errors with multiple loading cryptopp as shared lib
	on Linux
In-Reply-To: <9e7f6054-c20b-400e-a336-26e96ed4c0e7@c34g2000yqn.googlegroups.com>
References: <9e7f6054-c20b-400e-a336-26e96ed4c0e7@c34g2000yqn.googlegroups.com>
Message-ID: <cd6401a0912101015y3cbe0792v986e32945f2df55a@mail.gmail.com>

[cross-posted to the Crypto++ users mailing list and the Python C++-sig.]

David wrote on the Crypto++ users mailing list:

> We experience errors when trying to use cryptopp as shared lib from more than one other shared libs.

I struggled with this for a long time for the pycryptopp project and
finally gave up and just refactored pycryptopp so that there is only
one sharedlib that links to libcryptopp.so.

It kind of sucks because this means that if some *other* library gets
loaded into the same process as pycryptopp and that library links to
libcryptopp.so then it will crash the process.  But, I couldn't figure
out any other way to solve it.  Here is a letter I wrote at the time
outlining four possible solutions.  I tried them all and ended up
using the one listed as "#2":

http://mail.python.org/pipermail/cplusplus-sig/2009-May/014531.html

I still kind of hope that some Crypto++ or Python hackers will figure
out how to implement the one listed as "#4", which is to mark the
appropriate symbols in the Crypto++ API as being satisfied by a
different DSO (mark them as 'U' -- undefined symbols) so that the
loader will do the right thing. See the end of that letter referenced
above for details. It kind of seems like gcc cannot be configured to
do the right thing here, although MSVC can.

Here is an earlier letter I wrote explaining how there are two
different problems, one that happens if you set RTLD_GLOBAL and one
that happens if you don't set RTLD_GLOBAL:

http://groups.google.com/group/cryptopp-users/msg/7a0dacd0a4be5e2d

Here is the ticket that we used to track this issue in pycryptopp
until it was solved by the "there can be only one user of
libcryptopp.so" solution:

http://allmydata.org/trac/pycryptopp/ticket/9

Regards,

Zooko

From diepen at astron.nl  Fri Dec 11 07:43:05 2009
From: diepen at astron.nl (Ger van Diepen)
Date: Fri, 11 Dec 2009 07:43:05 +0100
Subject: [C++-sig] extracting a numpy.int32 type
In-Reply-To: <20091210151703.GB22600@SamZwo.urz.uni-heidelberg.de>
References: <20091209122524.GB5233@SamZwo.iwr.uni-heidelberg.de>
	<4B1FD051.6086.00A9.0@astron.nl>
	<20091210151703.GB22600@SamZwo.urz.uni-heidelberg.de>
Message-ID: <4B21F7F8.6086.00A9.0@astron.nl>

We noticed things went fine as long as the sizes of the types at the Python and C++ side matched.
On a 32bit system the behaviour differs from a 64bit machine. Because it fails for you, I guess you are working on a 64bit machine. It will be fine on a 32bit system.
You'll also notice it always fail if you you a short at the C++ side.

Here you have the answer I got from Andreas Klockner some time ago.

Cheers,
Ger

On Donnerstag 17 April 2008, Ger van Diepen wrote:
>   ArgumentError: Python argument types in
>     Table.getcell(table, str, numpy.int32)
                                ^^^^^^
>   did not match C++ signature:
>     getcell(casa::TableProxy {lvalue}, std::string columnname, int rownr)

This is numpy's fault, as it returns what they term an 'array scalar'. This is 
because Python's "int" on a 64-bit machine is 64 bits:

>>> 2**39
549755813888
>>> 2**66
73786976294838206464L

(see when the "L" shows up)

But numpy wants to return the exact data type you have in your array, so it 
gives you something that's not an int. As BP doesn't know about numpy by 
default (and can't assume it's there), there's no support. You would need to 
install custom (from-/to-python) rvalue converters for numpy's array scalars. 
I have something in the works that might help you--watch for an announcement 
in the next few days. But for now the explicit conversion hack is probably 
your easiest route to success.

Andreas


>>> On Thu, Dec 10, 2009 at  4:17 PM, in message
<20091210151703.GB22600 at SamZwo.urz.uni-heidelberg.de>, Nathan Huesken
<boost-python at lonely-star.org> wrote: 
> Hi,
> 
> Thanks for the reply.
> 
> But still something, I would like to understand:
> 
> If I want to convert numpy.float64 -> double, it works without problems.
> If I want to convert numpy.int32 -> int, I seem to need to register a
> new converte ...
> 
> Why is this?
> 
> Thanks!
> Nathan
> 
> On Wed, Dec 09, 2009 at 04:29:19PM +0100, Ger van Diepen wrote:
>> Hi Nathan,
>> 
>> We have converters written at the end of 2006. We never needed to convert 
> them to the more powerful newer versions of Boost-Python as they still build 
> fine. So they might look outdated.
>> 
>> They can convert scalars, sequences, dicts, and numpy arrays of many types, 
> also all numeric numpy types. So it can convert, say, an element of a numpy 
> int32 array to a C++ float scalar. Vice-versa a C++ numeric value is always 
> converted to a regular Python numeric type.
>> They work well on both 32bit and 64bit platforms.
>> 
>> If you want to see more you can find the code at pyrap.googlecode.com.
>> 
>> Cheers,
>> Ger
>> 
>> 
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig


From Nikolaus at rath.org  Sun Dec 13 03:54:54 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Sat, 12 Dec 2009 21:54:54 -0500
Subject: [C++-sig] [Py++] Generate Python code for ctypes?
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
Message-ID: <87y6l7wpkx.fsf@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
> You can take a look on gmplib example:
> http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/examples/gmplib_dev/
> It is fully working.

I downloaded Pyplusplus 1.0 from
http://sourceforge.net/projects/pygccxml/files/, but when I try to run
the code, I get

$ python generate_code.py 
/home/nikratio/.local/lib/python2.6/site-packages/pygccxml/parser/declarations_cache.py:8: DeprecationWarning: the md5 module is deprecated; use hashlib instead
  import md5
Traceback (most recent call last):
  File "generate_code.py", line 9, in <module>
    from pyplusplus.module_builder import ctypes_module_builder_t
ImportError: cannot import name ctypes_module_builder_t


Do I need the CVS version?


Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From roman.yakovenko at gmail.com  Sun Dec 13 07:33:59 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Sun, 13 Dec 2009 08:33:59 +0200
Subject: [C++-sig] [Py++] Generate Python code for ctypes?
In-Reply-To: <87y6l7wpkx.fsf@vostro.rath.org>
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
Message-ID: <7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>

On Sun, Dec 13, 2009 at 4:54 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> Roman Yakovenko <roman.yakovenko at gmail.com> writes:
>> You can take a look on gmplib example:
>> http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/examples/gmplib_dev/
>> It is fully working.
>
> I downloaded Pyplusplus 1.0 from
> http://sourceforge.net/projects/pygccxml/files/, but when I try to run
> the code, I get
>
> $ python generate_code.py
> /home/nikratio/.local/lib/python2.6/site-packages/pygccxml/parser/declarations_cache.py:8: DeprecationWarning: the md5 module is deprecated; use hashlib instead
> ?import md5
> Traceback (most recent call last):
> ?File "generate_code.py", line 9, in <module>
> ? ?from pyplusplus.module_builder import ctypes_module_builder_t
> ImportError: cannot import name ctypes_module_builder_t
>
>
> Do I need the CVS version?

Yes. The version, which contains ctypes code generator, was not released yet.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From Nikolaus at rath.org  Sun Dec 13 19:24:21 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Sun, 13 Dec 2009 13:24:21 -0500
Subject: [C++-sig] [Py++] Generate Python code for ctypes?
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
Message-ID: <877hsq91gq.fsf@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
> On Sun, Dec 13, 2009 at 4:54 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>> Roman Yakovenko <roman.yakovenko at gmail.com> writes:
>>> You can take a look on gmplib example:
>>> http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/examples/gmplib_dev/
>>> It is fully working.
>>
>> Do I need the CVS version?
>
> Yes. The version, which contains ctypes code generator, was not released yet.

Alright, now it seems to work great already! But I have another question
right away:

The generated code contains

libfuse_lib = ctypes.CDLL( r"/usr/lib/libfuse.so" )

I'd rather have the library looked up dynamically, as in

libfuse = CDLL(find_library("fuse"))

but it seems that I have to hardcode the complete path in
project_env.py, otherwise I get:

Traceback (most recent call last):
  File "generate_code.py", line 15, in <module>
    mb = ctypes_module_builder_t( [project_env.gmp.header_file], project_env.gmp.symbols_file, gccxml_cfg )
  File "/home/nikratio/.local/lib/python2.6/site-packages/pyplusplus/module_builder/ctypes_builder.py", line 42, in __init__
    self.__blob2decl = binary_parsers.merge_information( self.global_ns, exported_symbols_file )
  File "/home/nikratio/.local/lib/python2.6/site-packages/pygccxml/binary_parsers/parsers.py", line 303, in merge_information
    symbols = parser.parse()
  File "/home/nikratio/.local/lib/python2.6/site-packages/pygccxml/binary_parsers/parsers.py", line 95, in parse
    self.__loaded_symbols = self.load_symbols()
  File "/home/nikratio/.local/lib/python2.6/site-packages/pygccxml/binary_parsers/parsers.py", line 256, in load_symbols
    mangled_smbls = self.__extract_symbols( [part % tmpl_args for part in self.cmd_mangled] )
  File "/home/nikratio/.local/lib/python2.6/site-packages/pygccxml/binary_parsers/parsers.py", line 246, in __extract_symbols
    output = self.__execute_nm( cmd )
  File "/home/nikratio/.local/lib/python2.6/site-packages/pygccxml/binary_parsers/parsers.py", line 242, in __execute_nm
    raise RuntimeError( os.linesep.join(msg) )
RuntimeError: Unable to extract public\exported symbols from 'libfuse.so.2' file.
The command line, which was used to extract symbols, is "['nm', '--extern-only', '--dynamic', '--defined-only', '--numeric-sort', 'libfuse.so.2']"

Is there a way to accomplish this?


Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From Nikolaus at rath.org  Sun Dec 13 19:36:14 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Sun, 13 Dec 2009 13:36:14 -0500
Subject: [C++-sig] [Py++] Missing struct (was: [Py++] Generate Python code
	for ctypes?)
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
Message-ID: <878wd6pvq9.fsf_-_@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
>>> You can take a look on gmplib example:
>>> http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/examples/gmplib_dev/
>>> It is fully working.

There seems to be a problem with the generated code:

,----
| $ python
| Python 2.6.4 (r264:75706, Nov  2 2009, 14:38:03) 
| [GCC 4.4.1] on linux2
| Type "help", "copyright", "credits" or "license" for more information.
| >>> import generated
| Traceback (most recent call last):
|   File "<stdin>", line 1, in <module>
|   File "generated.py", line 227, in <module>
|     ("init", ctypes.POINTER( ctypes.CFUNCTYPE( None, ctypes.c_void_p, ctypes.POINTER( fuse_conn_info ) ) )),
| NameError: name 'fuse_conn_info' is not defined
`----

But struct fuse_conn_info is defined in fuse_common.h, which is included
in fuse_lowlevel.h, which is included in myheader.h, which has been used
to generate the code:

,----
| $ cat project_env.py
| import os
| import sys
| 
| lib = '/usr/lib/libfuse.so'
| 
| class gmp:
|     header_file = './myheader.h'
|     symbols_file = lib
|     shared_library_file = lib
|     generated_code_dir = 'generated'
`----

,----
| $ cat myheader.h 
| #define __builtin_va_arg_pack_len int
| #define _FILE_OFFSET_BITS 64
| #define FUSE_USE_VERSION 26
| #include <fuse/fuse_lowlevel.h>
`----

(the first define was necessary to prevent some gccxml errors).

Why is the fuse_file_info struct not translated into ctypes code?


Thanks,


   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From roman.yakovenko at gmail.com  Sun Dec 13 21:51:14 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Sun, 13 Dec 2009 22:51:14 +0200
Subject: [C++-sig] [Py++] Generate Python code for ctypes?
In-Reply-To: <877hsq91gq.fsf@vostro.rath.org>
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<877hsq91gq.fsf@vostro.rath.org>
Message-ID: <7465b6170912131251x65404c67o79e56d2a3c0fcc4f@mail.gmail.com>

On Sun, Dec 13, 2009 at 8:24 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> Roman Yakovenko <roman.yakovenko at gmail.com> writes:
>> On Sun, Dec 13, 2009 at 4:54 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>>> Roman Yakovenko <roman.yakovenko at gmail.com> writes:
>>>> You can take a look on gmplib example:
>>>> http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/examples/gmplib_dev/
>>>> It is fully working.
>>>
>>> Do I need the CVS version?
>>
>> Yes. The version, which contains ctypes code generator, was not released yet.
>
> Alright, now it seems to work great already! But I have another question
> right away:
>
> The generated code contains
>
> libfuse_lib = ctypes.CDLL( r"/usr/lib/libfuse.so" )
>
> I'd rather have the library looked up dynamically, as in
>
> libfuse = CDLL(find_library("fuse"))
>
> but it seems that I have to hardcode the complete path in
> project_env.py, otherwise I get:
>
> Traceback (most recent call last):
> ?File "generate_code.py", line 15, in <module>
> ? ?mb = ctypes_module_builder_t( [project_env.gmp.header_file], project_env.gmp.symbols_file, gccxml_cfg )
> ?File "/home/nikratio/.local/lib/python2.6/site-packages/pyplusplus/module_builder/ctypes_builder.py", line 42, in __init__
> ? ?self.__blob2decl = binary_parsers.merge_information( self.global_ns, exported_symbols_file )
> ?File "/home/nikratio/.local/lib/python2.6/site-packages/pygccxml/binary_parsers/parsers.py", line 303, in merge_information
> ? ?symbols = parser.parse()
> ?File "/home/nikratio/.local/lib/python2.6/site-packages/pygccxml/binary_parsers/parsers.py", line 95, in parse
> ? ?self.__loaded_symbols = self.load_symbols()
> ?File "/home/nikratio/.local/lib/python2.6/site-packages/pygccxml/binary_parsers/parsers.py", line 256, in load_symbols
> ? ?mangled_smbls = self.__extract_symbols( [part % tmpl_args for part in self.cmd_mangled] )
> ?File "/home/nikratio/.local/lib/python2.6/site-packages/pygccxml/binary_parsers/parsers.py", line 246, in __extract_symbols
> ? ?output = self.__execute_nm( cmd )
> ?File "/home/nikratio/.local/lib/python2.6/site-packages/pygccxml/binary_parsers/parsers.py", line 242, in __execute_nm
> ? ?raise RuntimeError( os.linesep.join(msg) )
> RuntimeError: Unable to extract public\exported symbols from 'libfuse.so.2' file.
> The command line, which was used to extract symbols, is "['nm', '--extern-only', '--dynamic', '--defined-only', '--numeric-sort', 'libfuse.so.2']"
>
> Is there a way to accomplish this?

The functionality you ask doesn't exist right now. If I remember
right, ctypes module has "look for a library" functionality. I will
take a look how to integrate it with Py++.

Your ideas ( and patches ) are welcome.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From roman.yakovenko at gmail.com  Sun Dec 13 21:54:07 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Sun, 13 Dec 2009 22:54:07 +0200
Subject: [C++-sig] [Py++] Missing struct (was: [Py++] Generate Python
	code for ctypes?)
In-Reply-To: <878wd6pvq9.fsf_-_@vostro.rath.org>
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<878wd6pvq9.fsf_-_@vostro.rath.org>
Message-ID: <7465b6170912131254x8f50918u3ca36e842b562eb6@mail.gmail.com>

On Sun, Dec 13, 2009 at 8:36 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> Roman Yakovenko <roman.yakovenko at gmail.com> writes:
>>>> You can take a look on gmplib example:
>>>> http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/examples/gmplib_dev/
>>>> It is fully working.
>
> There seems to be a problem with the generated code:
>
> ,----
> | $ python
> | Python 2.6.4 (r264:75706, Nov ?2 2009, 14:38:03)
> | [GCC 4.4.1] on linux2
> | Type "help", "copyright", "credits" or "license" for more information.
> | >>> import generated
> | Traceback (most recent call last):
> | ? File "<stdin>", line 1, in <module>
> | ? File "generated.py", line 227, in <module>
> | ? ? ("init", ctypes.POINTER( ctypes.CFUNCTYPE( None, ctypes.c_void_p, ctypes.POINTER( fuse_conn_info ) ) )),
> | NameError: name 'fuse_conn_info' is not defined
> `----
>
> But struct fuse_conn_info is defined in fuse_common.h, which is included
> in fuse_lowlevel.h, which is included in myheader.h, which has been used
> to generate the code:
>
> ,----
> | $ cat project_env.py
> | import os
> | import sys
> |
> | lib = '/usr/lib/libfuse.so'
> |
> | class gmp:
> | ? ? header_file = './myheader.h'
> | ? ? symbols_file = lib
> | ? ? shared_library_file = lib
> | ? ? generated_code_dir = 'generated'
> `----
>
> ,----
> | $ cat myheader.h
> | #define __builtin_va_arg_pack_len int
> | #define _FILE_OFFSET_BITS 64
> | #define FUSE_USE_VERSION 26
> | #include <fuse/fuse_lowlevel.h>
> `----
>
> (the first define was necessary to prevent some gccxml errors).
>
> Why is the fuse_file_info struct not translated into ctypes code?

This is because Py++ tries to make some intelligent guess what to
export. By default it exports all declarations found in the files in
the same directory.

I suggest you to use include/exclude functionality to  create the
desired declarations set.


HTH

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From Nikolaus at rath.org  Mon Dec 14 01:06:29 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Sun, 13 Dec 2009 19:06:29 -0500
Subject: [C++-sig] [Py++] Missing struct
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<878wd6pvq9.fsf_-_@vostro.rath.org>
	<7465b6170912131254x8f50918u3ca36e842b562eb6@mail.gmail.com>
Message-ID: <87y6l6wha2.fsf@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
>> Why is the fuse_file_info struct not translated into ctypes code?
>
> This is because Py++ tries to make some intelligent guess what to
> export. By default it exports all declarations found in the files in
> the same directory.
>
> I suggest you to use include/exclude functionality to  create the
> desired declarations set.

After looking into the API documentation and the example in
generate_code.py, I understand that I need to do something like this:

mb = ctypes_module_builder_t( [header_file], symbols_file, gccxml_cfg )
select_fn = lambda f: (check if f is the struct that I want)
mb.calldefs( select_fn ).include()

But I could not figure out what kind of argument the select_fn function
gets. Can you help me out again?


Thanks,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From Nikolaus at rath.org  Mon Dec 14 00:55:28 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Sun, 13 Dec 2009 18:55:28 -0500
Subject: [C++-sig] [Py++] Generate Python code for ctypes?
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<877hsq91gq.fsf@vostro.rath.org>
	<7465b6170912131251x65404c67o79e56d2a3c0fcc4f@mail.gmail.com>
Message-ID: <871viyxwcv.fsf@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
>> The generated code contains
>>
>> libfuse_lib = ctypes.CDLL( r"/usr/lib/libfuse.so" )
>>
>> I'd rather have the library looked up dynamically, as in
>>
>> libfuse = CDLL(find_library("fuse"))
>>
>> Is there a way to accomplish this?
>
> The functionality you ask doesn't exist right now. If I remember
> right, ctypes module has "look for a library" functionality. I will
> take a look how to integrate it with Py++.
>
> Your ideas ( and patches ) are welcome.

I looked into the code a bit. It seems that for the symbols file, it is
not possible to get rid of the absolute path, because it is passed to an
external command (and the output of ctypes.find_library is only useful to
ctypes.CDLL).

For the shared_lib file, however, it seems alright to always use
CDLL(find_library()) instead of CDLL with an absolute path.

I tried to create a patch, but when I tried to find out where the
'CDLL(...)' code is actually written, I got completely lost in the
source...


What I don't quite understand is what the symbols file is actually used
for. The code seems to determine the symbols available in the shared
library, but what is it doing with that information? Certainly all the
exported symbols should already be available from parsing the header
file... Is it possible to omit the symbols file and generate code based
only on the headers?


Best,


   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From roman.yakovenko at gmail.com  Mon Dec 14 06:27:42 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Mon, 14 Dec 2009 07:27:42 +0200
Subject: [C++-sig] [Py++] Missing struct
In-Reply-To: <87y6l6wha2.fsf@vostro.rath.org>
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<878wd6pvq9.fsf_-_@vostro.rath.org>
	<7465b6170912131254x8f50918u3ca36e842b562eb6@mail.gmail.com>
	<87y6l6wha2.fsf@vostro.rath.org>
Message-ID: <7465b6170912132127y7453ca4cvea48d4e1ee467360@mail.gmail.com>

On Mon, Dec 14, 2009 at 2:06 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> mb = ctypes_module_builder_t( [header_file], symbols_file, gccxml_cfg )
> select_fn = lambda f: (check if f is the struct that I want)
> mb.calldefs( select_fn ).include()
>
> But I could not figure out what kind of argument the select_fn function
> gets. Can you help me out again?

http://language-binding.net/pygccxml/query_interface.html

HTH


-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From Nikolaus at rath.org  Mon Dec 14 14:14:53 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Mon, 14 Dec 2009 08:14:53 -0500
Subject: [C++-sig] [Py++] Missing struct
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<878wd6pvq9.fsf_-_@vostro.rath.org>
	<7465b6170912131254x8f50918u3ca36e842b562eb6@mail.gmail.com>
Message-ID: <873a3dvgs2.fsf@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
>> Why is the fuse_file_info struct not translated into ctypes code?
>
> This is because Py++ tries to make some intelligent guess what to
> export. By default it exports all declarations found in the files in
> the same directory.

Hmm. Is there any actual use case for that? Just exporting the
declarations in one file without taking into account their dependencies
doesn't make much sense to me.


Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From roman.yakovenko at gmail.com  Mon Dec 14 14:40:44 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Mon, 14 Dec 2009 15:40:44 +0200
Subject: [C++-sig] [Py++] Missing struct
In-Reply-To: <873a3dvgs2.fsf@vostro.rath.org>
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<878wd6pvq9.fsf_-_@vostro.rath.org>
	<7465b6170912131254x8f50918u3ca36e842b562eb6@mail.gmail.com>
	<873a3dvgs2.fsf@vostro.rath.org>
Message-ID: <7465b6170912140540h2b3e6df5sa9963499463d695d@mail.gmail.com>

On Mon, Dec 14, 2009 at 3:14 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> Roman Yakovenko <roman.yakovenko at gmail.com> writes:
>>> Why is the fuse_file_info struct not translated into ctypes code?
>>
>> This is because Py++ tries to make some intelligent guess what to
>> export. By default it exports all declarations found in the files in
>> the same directory.
>
> Hmm. Is there any actual use case for that? Just exporting the
> declarations in one file without taking into account their dependencies
> doesn't make much sense to me.

//pseudo code
#include "stdio.h"

void do_smth( int i ){
    printf( i );
}

Do you still think Py++ needs to export every declaration that comes
from "stdio.h" header?

Py++ assumes, that files, that belongs to the same directory as the
file you pass, contain "interface", why all others contain
implementation.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From Nikolaus at rath.org  Mon Dec 14 15:00:56 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Mon, 14 Dec 2009 09:00:56 -0500
Subject: [C++-sig] [Py++] Include additional code with ctypes code generator
Message-ID: <87r5qxu02v.fsf@vostro.rath.org>

Hello,

Is there a way to add additional code to the output generated by the
ctypes code generator? Specifically, I would like to add some special
comments to prevent the autogenerated files being checked by PyLink and
PyDev.

http://language-binding.net/pyplusplus/documentation/inserting_code.html
talks about a add_registration_code method, but it doesn't seem to exist
for the ctypes generator:

mb = ctypes_module_builder_t( [header_file], symbols_file, gccxml_config=gccxml_cfg )
mb.build_code_creator(shared_library_file)
mb.add_registration_code( '#@PyDevCodeAnalysisIgnore', False )

yields

AttributeError: 'ctypes_module_builder_t' object has no attribute
'add_registration_code'


Best,


   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From Nikolaus at rath.org  Mon Dec 14 14:37:12 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Mon, 14 Dec 2009 08:37:12 -0500
Subject: [C++-sig] [Py++] Missing struct
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<878wd6pvq9.fsf_-_@vostro.rath.org>
	<7465b6170912131254x8f50918u3ca36e842b562eb6@mail.gmail.com>
	<87y6l6wha2.fsf@vostro.rath.org>
	<7465b6170912132127y7453ca4cvea48d4e1ee467360@mail.gmail.com>
Message-ID: <87ws0pu16f.fsf@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
> On Mon, Dec 14, 2009 at 2:06 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>> mb = ctypes_module_builder_t( [header_file], symbols_file, gccxml_cfg )
>> select_fn = lambda f: (check if f is the struct that I want)
>> mb.calldefs( select_fn ).include()
>>
>> But I could not figure out what kind of argument the select_fn function
>> gets. Can you help me out again?
>
> http://language-binding.net/pygccxml/query_interface.html

Thanks. The following now includes the missing struct:

mb.global_ns.class_('fuse_conn_info').include()

but I am actually not really interested in that struct itself, but in
the struct fuse_lowlevel_ops whose definitions happens to include a
reference to fuse_conn_info:

# from the generated code
fuse_lowlevel_ops._fields_ = [ #class fuse_lowlevel_ops
    ("init", ctypes.POINTER( ctypes.CFUNCTYPE( None, ctypes.c_void_p, ctypes.POINTER( fuse_conn_info ) ) )),
    ("destroy", ctypes.POINTER( ctypes.CFUNCTYPE( None, ctypes.c_void_p ) )),
# [...]

So I tried to include the class that I'm interested in instead:

mb.global_ns.class_('fuse_lowlevel_ops').include()

but in that case fuse_conn_info is no longer exported. Does that mean
that I have to manually trace and include all dependencies? I can't
quite believe that...


Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From Nikolaus at rath.org  Mon Dec 14 18:22:03 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Mon, 14 Dec 2009 12:22:03 -0500
Subject: [C++-sig] [Py++] Missing struct
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<878wd6pvq9.fsf_-_@vostro.rath.org>
	<7465b6170912131254x8f50918u3ca36e842b562eb6@mail.gmail.com>
	<873a3dvgs2.fsf@vostro.rath.org>
	<7465b6170912140540h2b3e6df5sa9963499463d695d@mail.gmail.com>
Message-ID: <877hspxyh0.fsf@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
> On Mon, Dec 14, 2009 at 3:14 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>> Roman Yakovenko <roman.yakovenko at gmail.com> writes:
>>>> Why is the fuse_file_info struct not translated into ctypes code?
>>>
>>> This is because Py++ tries to make some intelligent guess what to
>>> export. By default it exports all declarations found in the files in
>>> the same directory.
>>
>> Hmm. Is there any actual use case for that? Just exporting the
>> declarations in one file without taking into account their dependencies
>> doesn't make much sense to me.
>
> //pseudo code
> #include "stdio.h"
>
> void do_smth( int i ){
>     printf( i );
> }
>
> Do you still think Py++ needs to export every declaration that comes
> from "stdio.h" header?

No, but that's not what I'm saying. The only dependency here is printf,
so Py++ should export do_smth, because it is from the main file, and
printf from stdio.h, because do_smth depends on it.

> Py++ assumes, that files, that belongs to the same directory as the
> file you pass, contain "interface", why all others contain
> implementation.

In my opinion that is not a particularly good assumption. Why not work
without it and instead export all the dependencies of the file I'm
passing?


Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From alexey.akimov85 at gmail.com  Mon Dec 14 19:25:18 2009
From: alexey.akimov85 at gmail.com (Alexey Akimov)
Date: Mon, 14 Dec 2009 12:25:18 -0600
Subject: [C++-sig] what do I need to correctly port boost python extensions
Message-ID: <382d227f0912141025y6c896d6cq73097248f4aea1c@mail.gmail.com>

Hello to everyone,

I get confused trying to port my boost python extension module. So I've made
a shared object file test.so which actually perfectly works in the original
development directory. However, when I copy this file to another Linux
machine (I also copy some extra libraries such as
libboost_python-gcc344-1_39.so.1.39.0 and make sure that both .so files are
in PATH and LD_LIBRARY_PATH) I recieve "Import Error: undefined symbol:
/home/username/bin/libboost_python-gcc34-1_39.so.1.39.0:
PyUnicodeUCS4_FromEncodedObject". I aslo istalled the same version (compared
to one which was used for creation of the module) of Python to the Linux
machine where I try to use my extension.
Can anyone explain me where could the problem be. Also I would clarify what
is the general way to export just created modules? As I understand the order
is following: 1) create .so object (look for it in
/bin/compiler-name/release-or-debug/module-name.so directory) 2) copy this
file as well as other necessary library to some /bin or /lib directory (for
convenience) - and make sure they are in PATH and LD_LIBRARY_PATH variables.
3) import the module in python interpreter.
Do I need to do something else (define some other variables or make some
other changes) in order to make my module work?

Thanks a lot

Alexey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091214/c1b4e98f/attachment-0001.htm>

From Nikolaus at rath.org  Mon Dec 14 19:47:34 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Mon, 14 Dec 2009 13:47:34 -0500
Subject: [C++-sig] what do I need to correctly port boost python
	extensions
References: <382d227f0912141025y6c896d6cq73097248f4aea1c@mail.gmail.com>
Message-ID: <874ontxuih.fsf@vostro.rath.org>

Alexey Akimov <alexey.akimov85 at gmail.com> writes:
> I get confused trying to port my boost python extension module. So I've made
> a shared object file test.so which actually perfectly works in the original
> development directory. However, when I copy this file to another Linux
> machine (I also copy some extra libraries such as

I am not very familiar with Py++ yet, but .so libraries are not supposed
to be copied around from machine to machine unless your runtime
environment is exactly the same.

You should probably compile test.so (and any other required libraries)
on the target machine (i.e., copy the sources and not the binaries from
the development system).


Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From roman.yakovenko at gmail.com  Mon Dec 14 20:31:52 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Mon, 14 Dec 2009 21:31:52 +0200
Subject: [C++-sig] [Py++] Missing struct
In-Reply-To: <877hspxyh0.fsf@vostro.rath.org>
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<878wd6pvq9.fsf_-_@vostro.rath.org>
	<7465b6170912131254x8f50918u3ca36e842b562eb6@mail.gmail.com>
	<873a3dvgs2.fsf@vostro.rath.org>
	<7465b6170912140540h2b3e6df5sa9963499463d695d@mail.gmail.com>
	<877hspxyh0.fsf@vostro.rath.org>
Message-ID: <7465b6170912141131p507f9f3t5d20b1a4d207900b@mail.gmail.com>

On Mon, Dec 14, 2009 at 7:22 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> No, but that's not what I'm saying. The only dependency here is printf,
> so Py++ should export do_smth, because it is from the main file, and
> printf from stdio.h, because do_smth depends on it.

Unfortunately, GCCXML doesn't provide information about function
implementation and arbitrary C/C++ expression. So Py++ can't see this
dependency.

>> Py++ assumes, that files, that belongs to the same directory as the
>> file you pass, contain "interface", why all others contain
>> implementation.
>
> In my opinion that is not a particularly good assumption. Why not work
> without it and instead export all the dependencies of the file I'm
> passing?

Because what you think is good for you is actually bad for me( and
other users who already use Py++ ).

The assumption has a simple background( at least in my case ): in most
cases the project dependencies are bigger( more code ) than the
project itself. So it should be much easier to define what to export,
then the opposite. You can argue with this, but this is the current
default behavior and it is not going to change.

I understand that your use case is different than my - fine. I believe
Py++ has all functionality you need, to define what declarations
should be exported. Consider to use "reverse"logic:

mb = ctypes_module_builder_t( ... )
mb.global_ns.exclude()
#and now define what you want to include

May be you should take a look on
"module_builder/ctypes_decl_dependencies.py" module. It contains
"find_out_dependencies" functionality. As input it takes a set of
declarations you want to expose, the output is a set of all classes,
that need to be exposed.

HTH

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From roman.yakovenko at gmail.com  Mon Dec 14 20:33:19 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Mon, 14 Dec 2009 21:33:19 +0200
Subject: [C++-sig] [Py++] Include additional code with ctypes code
	generator
In-Reply-To: <87r5qxu02v.fsf@vostro.rath.org>
References: <87r5qxu02v.fsf@vostro.rath.org>
Message-ID: <7465b6170912141133w46fdd115mb56f0845a3c3ef3e@mail.gmail.com>

On Mon, Dec 14, 2009 at 4:00 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> Hello,
>
> Is there a way to add additional code to the output generated by the
> ctypes code generator? Specifically, I would like to add some special
> comments to prevent the autogenerated files being checked by PyLink and
> PyDev.
>
> http://language-binding.net/pyplusplus/documentation/inserting_code.html
> talks about a add_registration_code method, but it doesn't seem to exist
> for the ctypes generator:
>
> mb = ctypes_module_builder_t( [header_file], symbols_file, gccxml_config=gccxml_cfg )
> mb.build_code_creator(shared_library_file)
> mb.add_registration_code( '#@PyDevCodeAnalysisIgnore', False )
>
> yields
>
> AttributeError: 'ctypes_module_builder_t' object has no attribute
> 'add_registration_code'
>

Right now - there is no such way. I will try to add this functionality
today or tomorrow.


-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From roman.yakovenko at gmail.com  Mon Dec 14 20:44:52 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Mon, 14 Dec 2009 21:44:52 +0200
Subject: [C++-sig] [Py++] Generate Python code for ctypes?
In-Reply-To: <871viyxwcv.fsf@vostro.rath.org>
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<877hsq91gq.fsf@vostro.rath.org>
	<7465b6170912131251x65404c67o79e56d2a3c0fcc4f@mail.gmail.com>
	<871viyxwcv.fsf@vostro.rath.org>
Message-ID: <7465b6170912141144t52bcd685q456177a7dd14cc5f@mail.gmail.com>

On Mon, Dec 14, 2009 at 1:55 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> I tried to create a patch, but when I tried to find out where the
> 'CDLL(...)' code is actually written, I got completely lost in the
> source...

I suggest you to read the following document:
http://language-binding.net/pyplusplus/documentation/architecture.html#code-generation-engine
I hope it will clarify few things.

>
> What I don't quite understand is what the symbols file is actually used
> for. The code seems to determine the symbols available in the shared
> library, but what is it doing with that information?

Py++ uses information extracted from shared library
* to define the exact set of declarations which needs to be exported
* to extract the function calling convention
* to extract "mangled" function and/or variable name - later it will
be used to provide access to that symbol.

For example, you can use C++ compiler to compile "C" code and still
get access to the functionality.

> Certainly all the
> exported symbols should already be available from parsing the header
> file... Is it possible to omit the symbols file and generate code based
> only on the headers?

I don't think so( it didn't worked in my use case ) but you can try.
Start to comment out code in ctypes_builder.py and post the result :-)

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From roman.yakovenko at gmail.com  Mon Dec 14 20:54:04 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Mon, 14 Dec 2009 21:54:04 +0200
Subject: [C++-sig] what do I need to correctly port boost python
	extensions
In-Reply-To: <382d227f0912141025y6c896d6cq73097248f4aea1c@mail.gmail.com>
References: <382d227f0912141025y6c896d6cq73097248f4aea1c@mail.gmail.com>
Message-ID: <7465b6170912141154t7602b0f2kef0ebe0297c76037@mail.gmail.com>

On Mon, Dec 14, 2009 at 8:25 PM, Alexey Akimov
<alexey.akimov85 at gmail.com> wrote:
> Hello to everyone,
>
> I get confused trying to port my boost python extension module. So I've made
> a shared object file test.so which actually perfectly works in the original
> development directory. However, when I copy this file to another Linux
> machine (I also copy some extra libraries such as
> libboost_python-gcc344-1_39.so.1.39.0 and make sure that both .so files are
> in PATH and LD_LIBRARY_PATH) I recieve "Import Error: undefined symbol:
> /home/username/bin/libboost_python-gcc34-1_39.so.1.39.0:
> PyUnicodeUCS4_FromEncodedObject". I aslo istalled the same version (compared
> to one which was used for creation of the module) of Python to the Linux
> machine where I try to use my extension.
> Can anyone explain me where could the problem be.

In my opinion you compiled the module against Python version/build
which is differ from the "run" environment.

> Also I would clarify what
> is the general way to export just created modules? As I understand the order
> is following: 1) create .so object (look for it in
> /bin/compiler-name/release-or-debug/module-name.so directory) 2) copy this
> file as well as other necessary library to some /bin or /lib directory (for
> convenience) - and make sure they are in PATH and LD_LIBRARY_PATH variables.

You can use "ldd" command to find out dependencies. In the case, you
have few interpreters it also can help you to see what python*.so will
be actually used.

> 3) import the module in python interpreter.
>
> Do I need to do something else (define some other variables or make some
> other changes) in order to make my module work?

In case you have few modules, sometimes you need to tweak "dlopen" flags.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From alexey.akimov85 at gmail.com  Mon Dec 14 21:12:39 2009
From: alexey.akimov85 at gmail.com (Alexey Akimov)
Date: Mon, 14 Dec 2009 14:12:39 -0600
Subject: [C++-sig] what do I need to correctly port boost python
	extensions
In-Reply-To: <874ontxuih.fsf@vostro.rath.org>
References: <382d227f0912141025y6c896d6cq73097248f4aea1c@mail.gmail.com>
	<874ontxuih.fsf@vostro.rath.org>
Message-ID: <382d227f0912141212n3487526dt793decf72b701f1e@mail.gmail.com>

Thanks, Nikolaus
However I am still wondering if there is a way to avoid rebuilding the
extension when one travel from machine to machine. Of course that should
definitely work, but it seems unpractical for many purposes or at least
inconvenient. Suppose I want to run some calculations on a cluster using the
extension. That means I need the library to be copied on every machine of
that cluster (may be I am wrong with this but this is most straightforward
way to clone the process). Also this makes lot of potential troubles for
other users who would want to work with such extension - do they need to
rebuild extension module every time? I hope that there may exist some fancy
way to avoid rebuilding the module (also to do this the user would need
boost python installed, what may require some extra-work for that user). So
if anyone knows if this is still possible - please, let me know.

Alexey

2009/12/14 Nikolaus Rath <Nikolaus at rath.org>

> Alexey Akimov <alexey.akimov85 at gmail.com> writes:
> > I get confused trying to port my boost python extension module. So I've
> made
> > a shared object file test.so which actually perfectly works in the
> original
> > development directory. However, when I copy this file to another Linux
> > machine (I also copy some extra libraries such as
>
> I am not very familiar with Py++ yet, but .so libraries are not supposed
> to be copied around from machine to machine unless your runtime
> environment is exactly the same.
>
> You should probably compile test.so (and any other required libraries)
> on the target machine (i.e., copy the sources and not the binaries from
> the development system).
>
>
> Best,
>
>   -Nikolaus
>
> --
>  ?Time flies like an arrow, fruit flies like a Banana.?
>
>  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091214/b39f8ec9/attachment-0001.htm>

From roman.yakovenko at gmail.com  Mon Dec 14 21:14:01 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Mon, 14 Dec 2009 22:14:01 +0200
Subject: [C++-sig] [Py++] Include additional code with ctypes code
	generator
In-Reply-To: <7465b6170912141133w46fdd115mb56f0845a3c3ef3e@mail.gmail.com>
References: <87r5qxu02v.fsf@vostro.rath.org>
	<7465b6170912141133w46fdd115mb56f0845a3c3ef3e@mail.gmail.com>
Message-ID: <7465b6170912141214p7eff159at85d85a19efbb1977@mail.gmail.com>

On Mon, Dec 14, 2009 at 9:33 PM, Roman Yakovenko
<roman.yakovenko at gmail.com> wrote:
> On Mon, Dec 14, 2009 at 4:00 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>> Hello,
>>
>> Is there a way to add additional code to the output generated by the
>> ctypes code generator? Specifically, I would like to add some special
>> comments to prevent the autogenerated files being checked by PyLink and
>> PyDev.
>>
>
> Right now - there is no such way. I will try to add this functionality
> today or tomorrow.

Nikolaus, can you take a (dummy) generated code and show me where you
need to add the code?

Thanks

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From Nikolaus at rath.org  Mon Dec 14 20:38:12 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Mon, 14 Dec 2009 14:38:12 -0500
Subject: [C++-sig] [Py++] Missing struct
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<878wd6pvq9.fsf_-_@vostro.rath.org>
	<7465b6170912131254x8f50918u3ca36e842b562eb6@mail.gmail.com>
	<873a3dvgs2.fsf@vostro.rath.org>
	<7465b6170912140540h2b3e6df5sa9963499463d695d@mail.gmail.com>
	<877hspxyh0.fsf@vostro.rath.org>
Message-ID: <871vixfisb.fsf@vostro.rath.org>

Nikolaus Rath <Nikolaus at rath.org> writes:
> Roman Yakovenko <roman.yakovenko at gmail.com> writes:
>> On Mon, Dec 14, 2009 at 3:14 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>>> Roman Yakovenko <roman.yakovenko at gmail.com> writes:
>>>>> Why is the fuse_file_info struct not translated into ctypes code?
>>>>
>>>> This is because Py++ tries to make some intelligent guess what to
>>>> export. By default it exports all declarations found in the files in
>>>> the same directory.
>>>
>>> Hmm. Is there any actual use case for that? Just exporting the
>>> declarations in one file without taking into account their dependencies
>>> doesn't make much sense to me.
>>
>> //pseudo code
>> #include "stdio.h"
>>
>> void do_smth( int i ){
>>     printf( i );
>> }
>>
>> Do you still think Py++ needs to export every declaration that comes
>> from "stdio.h" header?
>
> No, but that's not what I'm saying. The only dependency here is printf,
> so Py++ should export do_smth, because it is from the main file, and
> printf from stdio.h, because do_smth depends on it.

I have to correct myself. printf() is actually only important for the
implementation, so as far as ctypes is concerned, the above code does
not have any dependencies at all and I would not expect Py++ to export
any stdio.h symbols.

In the following case however, I would expect Py++ to export struct stat
from stat.h, since the code depends on it and I don't really see why
anyone would want to export only do_smth (since the generated code won't
even load into Python due to a missing symbol):

#include <sys/stat.h>

void do_smth(struct stat* foo) {
   printf(foo->st_ino);
}


I hope that clarifies what I mean.

Best,


   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From Nikolaus at rath.org  Mon Dec 14 23:09:31 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Mon, 14 Dec 2009 17:09:31 -0500
Subject: [C++-sig] what do I need to correctly port boost python
	extensions
References: <382d227f0912141025y6c896d6cq73097248f4aea1c@mail.gmail.com>
	<874ontxuih.fsf@vostro.rath.org>
	<382d227f0912141212n3487526dt793decf72b701f1e@mail.gmail.com>
Message-ID: <87tyvt438k.fsf@vostro.rath.org>

Alexey Akimov <alexey.akimov85 at gmail.com> writes:
> However I am still wondering if there is a way to avoid rebuilding the
> extension when one travel from machine to machine.

The only way is to have exactly the same runtime environment (i.e.,
having installed the same distributions with the same version on the
same architecture). Otherwise you have to recompile.

> Suppose I want to run some calculations on a cluster using the
> extension. That means I need the library to be copied on every machine of
> that cluster (may be I am wrong with this but this is most straightforward
> way to clone the process).

Well, in a cluster you might be able to configure your systems
sufficiently homogenous that it works.

> Also this makes lot of potential troubles for
> other users who would want to work with such extension - do they need to
> rebuild extension module every time?

No, only once when they install it on their machine.

> I hope that there may exist some fancy way to avoid rebuilding the
> module (also to do this the user would need boost python installed,
> what may require some extra-work for that user).

No, there is no way to avoid that. Say hello to the realities of
software distribution :-).


   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From Nikolaus at rath.org  Mon Dec 14 23:11:08 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Mon, 14 Dec 2009 17:11:08 -0500
Subject: [C++-sig] [Py++] Include additional code with ctypes code
	generator
References: <87r5qxu02v.fsf@vostro.rath.org>
	<7465b6170912141133w46fdd115mb56f0845a3c3ef3e@mail.gmail.com>
	<7465b6170912141214p7eff159at85d85a19efbb1977@mail.gmail.com>
Message-ID: <87r5qx435v.fsf@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
> On Mon, Dec 14, 2009 at 9:33 PM, Roman Yakovenko
> <roman.yakovenko at gmail.com> wrote:
>> On Mon, Dec 14, 2009 at 4:00 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>>> Hello,
>>>
>>> Is there a way to add additional code to the output generated by the
>>> ctypes code generator? Specifically, I would like to add some special
>>> comments to prevent the autogenerated files being checked by PyLink and
>>> PyDev.
>>>
>>
>> Right now - there is no such way. I will try to add this functionality
>> today or tomorrow.
>
> Nikolaus, can you take a (dummy) generated code and show me where you
> need to add the code?

At the very beginning would be best, right below '# This file has been
generated by Py++'.


Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From Nikolaus at rath.org  Mon Dec 14 23:05:08 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Mon, 14 Dec 2009 17:05:08 -0500
Subject: [C++-sig] [Py++] Missing struct
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<878wd6pvq9.fsf_-_@vostro.rath.org>
	<7465b6170912131254x8f50918u3ca36e842b562eb6@mail.gmail.com>
	<873a3dvgs2.fsf@vostro.rath.org>
	<7465b6170912140540h2b3e6df5sa9963499463d695d@mail.gmail.com>
	<877hspxyh0.fsf@vostro.rath.org>
	<7465b6170912141131p507f9f3t5d20b1a4d207900b@mail.gmail.com>
Message-ID: <87ws0p43fv.fsf@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
> On Mon, Dec 14, 2009 at 7:22 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>> No, but that's not what I'm saying. The only dependency here is printf,
>> so Py++ should export do_smth, because it is from the main file, and
>> printf from stdio.h, because do_smth depends on it.
>
> Unfortunately, GCCXML doesn't provide information about function
> implementation and arbitrary C/C++ expression. So Py++ can't see this
> dependency.

Yes, I read the example wrong. See other mail.

>>> Py++ assumes, that files, that belongs to the same directory as the
>>> file you pass, contain "interface", why all others contain
>>> implementation.
>>
>> In my opinion that is not a particularly good assumption. Why not work
>> without it and instead export all the dependencies of the file I'm
>> passing?
>
> Because what you think is good for you is actually bad for me( and
> other users who already use Py++ ).
>
> The assumption has a simple background( at least in my case ): in most
> cases the project dependencies are bigger( more code ) than the
> project itself. So it should be much easier to define what to export,
> then the opposite. You can argue with this, but this is the current
> default behavior and it is not going to change.


No, I think either I don't understand what you are saying or the other
way around. And I'm not arguing about the current implementation being
good or bad (or at least I'm trying not to), but I'm trying to
understand where we don't understand each other :-).

I totally agree with you that it is better to define what to export,
rather than what not to export (actually I'm already calling
global_ns.exclude() first thing in my code).

But what I don't understand his this: If exporting only one symbol "a"
produces python code that cannot be imported (because it depends on the
definition of another symbol 'b' that was not explicitly requested),
what is the point of generating the code in the first place? Under which
circumstances do you want to generate code that cannot be imported? 

> I understand that your use case is different than my - fine. I believe
> Py++ has all functionality you need, to define what declarations
> should be exported. Consider to use "reverse"logic:
>
> mb = ctypes_module_builder_t( ... )
> mb.global_ns.exclude()
> #and now define what you want to include
>
> May be you should take a look on
> "module_builder/ctypes_decl_dependencies.py" module. It contains
> "find_out_dependencies" functionality. As input it takes a set of
> declarations you want to expose, the output is a set of all classes,
> that need to be exposed.

Actually I already did, but I failed to understand the interface once
again. What exactly is a "set of declarations"? A list of the symbol
names? A list of element as they are returned by e.g.
global_ns.class_('mysymbol')?

Can you tell me how to instruct Py++ to include the struct
'fuse_lowlevel_ops' and everything it that it depends on?


Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From Nikolaus at rath.org  Mon Dec 14 23:40:44 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Mon, 14 Dec 2009 17:40:44 -0500
Subject: [C++-sig] [Py++] Missing struct
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<878wd6pvq9.fsf_-_@vostro.rath.org>
	<7465b6170912131254x8f50918u3ca36e842b562eb6@mail.gmail.com>
	<873a3dvgs2.fsf@vostro.rath.org>
	<7465b6170912140540h2b3e6df5sa9963499463d695d@mail.gmail.com>
	<877hspxyh0.fsf@vostro.rath.org>
	<7465b6170912141131p507f9f3t5d20b1a4d207900b@mail.gmail.com>
Message-ID: <87ljh541sj.fsf@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
> May be you should take a look on
> "module_builder/ctypes_decl_dependencies.py" module. It contains
> "find_out_dependencies" functionality. As input it takes a set of
> declarations you want to expose, the output is a set of all classes,
> that need to be exposed.

Update: I found an usage example in ctypes_builder.py and came up with
the following code:

included_decls = set([mb.global_ns.class_('fuse_lowlevel_ops')]) 
to_be_included = ctypes_decls_dependencies.find_out_dependencies(included_decls)
to_be_included.update(included_decls)
map( lambda d: d.include(), to_be_included )

but the resulted generated code still contains only the single struct:

fuse_lowlevel_ops._fields_ = [ #class fuse_lowlevel_ops
    ("init", ctypes.POINTER( ctypes.CFUNCTYPE( None, ctypes.c_void_p, ctypes.POINTER( fuse_conn_info ) ) )),
    ("destroy", ctypes.POINTER( ctypes.CFUNCTYPE( None, ctypes.c_void_p ) )),
    ("lookup", ctypes.POINTER( ctypes.CFUNCTYPE( None, ctypes.POINTER(
    fuse_req ), ctypes.c_ulong, ctypes.c_char_p ) )),

although there are clearly dependencies on fuse_conn_info and fuse_req.



Thank you for taking so much time on my flood of questions!

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From Nikolaus at rath.org  Mon Dec 14 23:35:53 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Mon, 14 Dec 2009 17:35:53 -0500
Subject: [C++-sig] [Py++] Generate Python code for ctypes?
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<877hsq91gq.fsf@vostro.rath.org>
	<7465b6170912131251x65404c67o79e56d2a3c0fcc4f@mail.gmail.com>
	<871viyxwcv.fsf@vostro.rath.org>
	<7465b6170912141144t52bcd685q456177a7dd14cc5f@mail.gmail.com>
Message-ID: <87ocm1420m.fsf@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
>> Certainly all the exported symbols should already be available from
>> parsing the header file... Is it possible to omit the symbols file
>> and generate code based only on the headers?
>
> I don't think so( it didn't worked in my use case ) but you can try.
> Start to comment out code in ctypes_builder.py and post the result :-)

I looked into the xml file created by gccxml and (at least in the case
of C code) it seems to contain all the information that's needed.

I modified parsers.py to return an empty dict if no symbols_file is
provided:

def merge_information( global_ns, fname, runs_under_unittest=False ):
    """high level function - select the appropriate binary file parser and integrates
    the information from the file to the declarations tree. """
    if fname is None:
        return dict()
        

this seems to work partially. I can still export all the struct's, but I
no longer get any function exports. I can mark them as to be exported
without any error, but they don't show up in the generated code.

Any idea?

Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From roman.yakovenko at gmail.com  Tue Dec 15 06:28:12 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Tue, 15 Dec 2009 07:28:12 +0200
Subject: [C++-sig] [Py++] Missing struct
In-Reply-To: <87ws0p43fv.fsf@vostro.rath.org>
References: <87y6ldv4sg.fsf@vostro.rath.org> <87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<878wd6pvq9.fsf_-_@vostro.rath.org>
	<7465b6170912131254x8f50918u3ca36e842b562eb6@mail.gmail.com>
	<873a3dvgs2.fsf@vostro.rath.org>
	<7465b6170912140540h2b3e6df5sa9963499463d695d@mail.gmail.com>
	<877hspxyh0.fsf@vostro.rath.org>
	<7465b6170912141131p507f9f3t5d20b1a4d207900b@mail.gmail.com>
	<87ws0p43fv.fsf@vostro.rath.org>
Message-ID: <7465b6170912142128g38abe9c7g9cdad725e520324b@mail.gmail.com>

On Tue, Dec 15, 2009 at 12:05 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> But what I don't understand his this: If exporting only one symbol "a"
> produces python code that cannot be imported (because it depends on the
> definition of another symbol 'b' that was not explicitly requested),
> what is the point of generating the code in the first place? Under which
> circumstances do you want to generate code that cannot be imported?

Consider use case, where your code is split between different modules:

module Base:
    struct B;

module Derived1:
    d( B* )

module Derived2:
    d2( B* )

In this case, when you exposed Derived* you don't want to expose struct B.

>> I understand that your use case is different than my - fine. I believe
>> Py++ has all functionality you need, to define what declarations
>> should be exported. Consider to use "reverse"logic:
>>
>> mb = ctypes_module_builder_t( ... )
>> mb.global_ns.exclude()
>> #and now define what you want to include
>>
>> May be you should take a look on
>> "module_builder/ctypes_decl_dependencies.py" module. It contains
>> "find_out_dependencies" functionality. As input it takes a set of
>> declarations you want to expose, the output is a set of all classes,
>> that need to be exposed.
>
> Actually I already did, but I failed to understand the interface once
> again. What exactly is a "set of declarations"? A list of the symbol
> names? A list of element as they are returned by e.g.
> global_ns.class_('mysymbol')?
>
> Can you tell me how to instruct Py++ to include the struct
> 'fuse_lowlevel_ops' and everything it that it depends on?

mb = ctypes_module_builder_t( ... )
mb.global_ns.exclude()

fuse_lowlevel_ops = mb.class_( ' 'fuse_lowlevel_ops' )

for d in find_out_dependencies( [fuse_lowlevel_ops] ):
    d.include()

I can't test the code right now, but it should look very similar to this.

HTH

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From roman.yakovenko at gmail.com  Tue Dec 15 06:31:08 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Tue, 15 Dec 2009 07:31:08 +0200
Subject: [C++-sig] [Py++] Missing struct
In-Reply-To: <87ljh541sj.fsf@vostro.rath.org>
References: <87y6ldv4sg.fsf@vostro.rath.org> <87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<878wd6pvq9.fsf_-_@vostro.rath.org>
	<7465b6170912131254x8f50918u3ca36e842b562eb6@mail.gmail.com>
	<873a3dvgs2.fsf@vostro.rath.org>
	<7465b6170912140540h2b3e6df5sa9963499463d695d@mail.gmail.com>
	<877hspxyh0.fsf@vostro.rath.org>
	<7465b6170912141131p507f9f3t5d20b1a4d207900b@mail.gmail.com>
	<87ljh541sj.fsf@vostro.rath.org>
Message-ID: <7465b6170912142131r4735d692v7b505af2951e2497@mail.gmail.com>

On Tue, Dec 15, 2009 at 12:40 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> Roman Yakovenko <roman.yakovenko at gmail.com> writes:
>> May be you should take a look on
>> "module_builder/ctypes_decl_dependencies.py" module. It contains
>> "find_out_dependencies" functionality. As input it takes a set of
>> declarations you want to expose, the output is a set of all classes,
>> that need to be exposed.
>
> Update: I found an usage example in ctypes_builder.py and came up with
> the following code:
>
> included_decls = set([mb.global_ns.class_('fuse_lowlevel_ops')])
> to_be_included = ctypes_decls_dependencies.find_out_dependencies(included_decls)
> to_be_included.update(included_decls)
> map( lambda d: d.include(), to_be_included )
>
> but the resulted generated code still contains only the single struct:
>
> fuse_lowlevel_ops._fields_ = [ #class fuse_lowlevel_ops
> ? ?("init", ctypes.POINTER( ctypes.CFUNCTYPE( None, ctypes.c_void_p, ctypes.POINTER( fuse_conn_info ) ) )),
> ? ?("destroy", ctypes.POINTER( ctypes.CFUNCTYPE( None, ctypes.c_void_p ) )),
> ? ?("lookup", ctypes.POINTER( ctypes.CFUNCTYPE( None, ctypes.POINTER(
> ? ?fuse_req ), ctypes.c_ulong, ctypes.c_char_p ) )),
>
> although there are clearly dependencies on fuse_conn_info and fuse_req.

Can you create a small example, which reproduce the problem and send it to me?

Thanks

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From roman.yakovenko at gmail.com  Tue Dec 15 06:33:44 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Tue, 15 Dec 2009 07:33:44 +0200
Subject: [C++-sig] [Py++] Generate Python code for ctypes?
In-Reply-To: <87ocm1420m.fsf@vostro.rath.org>
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<877hsq91gq.fsf@vostro.rath.org>
	<7465b6170912131251x65404c67o79e56d2a3c0fcc4f@mail.gmail.com>
	<871viyxwcv.fsf@vostro.rath.org>
	<7465b6170912141144t52bcd685q456177a7dd14cc5f@mail.gmail.com>
	<87ocm1420m.fsf@vostro.rath.org>
Message-ID: <7465b6170912142133o2ed52a26x25510cfe33476ff2@mail.gmail.com>

On Tue, Dec 15, 2009 at 12:35 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> Roman Yakovenko <roman.yakovenko at gmail.com> writes:
>>> Certainly all the exported symbols should already be available from
>>> parsing the header file... Is it possible to omit the symbols file
>>> and generate code based only on the headers?
>>
>> I don't think so( it didn't worked in my use case ) but you can try.
>> Start to comment out code in ctypes_builder.py and post the result :-)
>
> I looked into the xml file created by gccxml and (at least in the case
> of C code) it seems to contain all the information that's needed.
>
> I modified parsers.py to return an empty dict if no symbols_file is
> provided:
>
> def merge_information( global_ns, fname, runs_under_unittest=False ):
> ? ?"""high level function - select the appropriate binary file parser and integrates
> ? ?the information from the file to the declarations tree. """
> ? ?if fname is None:
> ? ? ? ?return dict()
>
>
> this seems to work partially. I can still export all the struct's, but I
> no longer get any function exports. I can mark them as to be exported
> without any error, but they don't show up in the generated code.
>
> Any idea?

Yes. Py++ thinks that those functions are "private" and doesn't export
them. You should dig father ( creators_factory/ctypes_creator.py ).

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From romainchanu at gmail.com  Tue Dec 15 08:36:12 2009
From: romainchanu at gmail.com (Romain CHANU)
Date: Tue, 15 Dec 2009 15:36:12 +0800
Subject: [C++-sig] Boost Python - How to export
	boost::shared_ptr<std::string>
Message-ID: <101fdd5e0912142336p51c01505sa355bb226635e114@mail.gmail.com>

Hi,

I want to export an attribute which is a boost::shared_ptr<std::string> and
I got the following error:
" TypeError: No Python class registered for C++ class class
boost::shared_ptr<class std::basic_string<char,struct
std::char_traits<char>,class std::allocator<char> > >  "
I found an old post on a similar issue :
http://old.nabble.com/Boost.python---How-to-export-shared_ptr%3Cvector%3Cshared_ptr%3Cstring%3E-%3E-%3E--tt22389131.html#a22389131but
no one has replied :-\

Can anyone tell me how can I export this type?

Thank you!

Romain
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091215/025d8e3e/attachment.htm>

From roman.yakovenko at gmail.com  Tue Dec 15 09:01:01 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Tue, 15 Dec 2009 10:01:01 +0200
Subject: [C++-sig] Boost Python - How to export
	boost::shared_ptr<std::string>
In-Reply-To: <101fdd5e0912142336p51c01505sa355bb226635e114@mail.gmail.com>
References: <101fdd5e0912142336p51c01505sa355bb226635e114@mail.gmail.com>
Message-ID: <7465b6170912150001q6307f3a1l4f3e89bcc9a82de8@mail.gmail.com>

On Tue, Dec 15, 2009 at 9:36 AM, Romain CHANU <romainchanu at gmail.com> wrote:
> Hi,
>
> I want to export an attribute which is a boost::shared_ptr<std::string> and
> I got the following error:
> " TypeError: No Python class registered for C++ class class
> boost::shared_ptr<class std::basic_string<char,struct
> std::char_traits<char>,class std::allocator<char> > >? "
> I found an old post on a similar issue :
> http://old.nabble.com/Boost.python---How-to-export-shared_ptr%3Cvector%3Cshared_ptr%3Cstring%3E-%3E-%3E--tt22389131.html#a22389131
> but no one has replied :-\
>
> Can anyone tell me how can I export this type?

If I were you, I would not export it. In any case Python strings are
immutable and you always returns a copy of the string.

Instead of exposing that attribute directly create a get/set functions
which use "raw" "std::string" in their interface and expose them.

May be "custom converter" could help you (
http://www.boost.org/doc/libs/1_41_0/libs/python/doc/v2/faq.html#custom_string
), but I never tried it on such use case.



-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From romainchanu at gmail.com  Tue Dec 15 10:14:44 2009
From: romainchanu at gmail.com (Romain CHANU)
Date: Tue, 15 Dec 2009 17:14:44 +0800
Subject: [C++-sig] Boost Python - How to export
	boost::shared_ptr<std::string>
In-Reply-To: <7465b6170912150001q6307f3a1l4f3e89bcc9a82de8@mail.gmail.com>
References: <101fdd5e0912142336p51c01505sa355bb226635e114@mail.gmail.com>
	<7465b6170912150001q6307f3a1l4f3e89bcc9a82de8@mail.gmail.com>
Message-ID: <101fdd5e0912150114p60f8540elc8f993f6d30a55d8@mail.gmail.com>

I have seen this page before but I have also never tried as I was looking
for a fast solution.

I was thinking to use boost::shared_ptr because I want to do something like:

// Python code (mystring is boost::shared_ptr<std::string>)

if object.mystring:

     print 'my string: ', object.mystring

Knowing that:

- If boost::shared_ptr attribute is not initialized, its value is 0)
- I use add_property


Any workaround?

Cheers,


2009/12/15 Roman Yakovenko <roman.yakovenko at gmail.com>

> On Tue, Dec 15, 2009 at 9:36 AM, Romain CHANU <romainchanu at gmail.com>
> wrote:
> > Hi,
> >
> > I want to export an attribute which is a boost::shared_ptr<std::string>
> and
> > I got the following error:
> > " TypeError: No Python class registered for C++ class class
> > boost::shared_ptr<class std::basic_string<char,struct
> > std::char_traits<char>,class std::allocator<char> > >  "
> > I found an old post on a similar issue :
> >
> http://old.nabble.com/Boost.python---How-to-export-shared_ptr%3Cvector%3Cshared_ptr%3Cstring%3E-%3E-%3E--tt22389131.html#a22389131
> > but no one has replied :-\
> >
> > Can anyone tell me how can I export this type?
>
> If I were you, I would not export it. In any case Python strings are
> immutable and you always returns a copy of the string.
>
> Instead of exposing that attribute directly create a get/set functions
> which use "raw" "std::string" in their interface and expose them.
>
> May be "custom converter" could help you (
>
> http://www.boost.org/doc/libs/1_41_0/libs/python/doc/v2/faq.html#custom_string
> ), but I never tried it on such use case.
>
>
>
> --
> Roman Yakovenko
> C++ Python language binding
> http://www.language-binding.net/
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091215/40c7252c/attachment.htm>

From Nikolaus at rath.org  Tue Dec 15 17:33:36 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Tue, 15 Dec 2009 11:33:36 -0500
Subject: [C++-sig] [Py++] Missing struct
References: <87y6ldv4sg.fsf@vostro.rath.org> <87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<878wd6pvq9.fsf_-_@vostro.rath.org>
	<7465b6170912131254x8f50918u3ca36e842b562eb6@mail.gmail.com>
	<873a3dvgs2.fsf@vostro.rath.org>
	<7465b6170912140540h2b3e6df5sa9963499463d695d@mail.gmail.com>
	<877hspxyh0.fsf@vostro.rath.org>
	<7465b6170912141131p507f9f3t5d20b1a4d207900b@mail.gmail.com>
	<87ws0p43fv.fsf@vostro.rath.org>
	<7465b6170912142128g38abe9c7g9cdad725e520324b@mail.gmail.com>
Message-ID: <87fx7cnqn3.fsf@inspiron.ap.columbia.edu>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
> On Tue, Dec 15, 2009 at 12:05 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>> But what I don't understand his this: If exporting only one symbol "a"
>> produces python code that cannot be imported (because it depends on the
>> definition of another symbol 'b' that was not explicitly requested),
>> what is the point of generating the code in the first place? Under which
>> circumstances do you want to generate code that cannot be imported?
>
> Consider use case, where your code is split between different modules:
>
> module Base:
>     struct B;
>
> module Derived1:
>     d( B* )
>
> module Derived2:
>     d2( B* )
>
> In this case, when you exposed Derived* you don't want to expose struct B.

With modules you mean files in this context, right? I guess I finally
understand. So you would have 'struct B' in one file and something that
uses struct B in a different file. Then both files are translated
to ctypes separately, but as long as they are both imported, all the
required symbols are defined. Sorry that it took me so long ;-).

I'm still curious though: how do you usually handle dependencies on
system headers? If you use the dependency tracing code, it won't be able
to tell which dependencies are from system files (and should be
included) and which dependencies are from project headers (and are going
to be exported separately).


Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From roman.yakovenko at gmail.com  Tue Dec 15 21:54:28 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Tue, 15 Dec 2009 22:54:28 +0200
Subject: [C++-sig] [Py++] Missing struct
In-Reply-To: <87ljh4nrmb.fsf@inspiron.ap.columbia.edu>
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<878wd6pvq9.fsf_-_@vostro.rath.org>
	<7465b6170912131254x8f50918u3ca36e842b562eb6@mail.gmail.com>
	<873a3dvgs2.fsf@vostro.rath.org>
	<7465b6170912140540h2b3e6df5sa9963499463d695d@mail.gmail.com>
	<877hspxyh0.fsf@vostro.rath.org>
	<7465b6170912141131p507f9f3t5d20b1a4d207900b@mail.gmail.com>
	<87ljh541sj.fsf@vostro.rath.org>
	<7465b6170912142131r4735d692v7b505af2951e2497@mail.gmail.com>
	<87ljh4nrmb.fsf@inspiron.ap.columbia.edu>
Message-ID: <7465b6170912151254h508bee25r5f29e65a4cf96421@mail.gmail.com>

On Tue, Dec 15, 2009 at 6:12 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>> Can you create a small example, which reproduce the problem and send it to me?
>
> The attached file should reproduce the problem, but it needs the fuse
> headers to be installed (Debian/Ubuntu: libfuse-dev). Let me know if
> you don't have them/don't want to install them, then I'll come up with
> my own header instead.

Okey. That helped. The problem was in pygccxml - it didn't calculate
right a struct dependencies in case of variables. To be more specific,
fuse structs contains variables, which are pointers to functions and
pygccxml failed in this case.

I committed the fix. You will need to update pygccxml and Py++ projects.

I also attached the generated file.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fuse_ctypes.py
Type: text/x-python
Size: 18319 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091215/04bf01d1/attachment-0001.py>

From roman.yakovenko at gmail.com  Tue Dec 15 21:59:02 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Tue, 15 Dec 2009 22:59:02 +0200
Subject: [C++-sig] [Py++] Missing struct
In-Reply-To: <87fx7cnqn3.fsf@inspiron.ap.columbia.edu>
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<878wd6pvq9.fsf_-_@vostro.rath.org>
	<7465b6170912131254x8f50918u3ca36e842b562eb6@mail.gmail.com>
	<873a3dvgs2.fsf@vostro.rath.org>
	<7465b6170912140540h2b3e6df5sa9963499463d695d@mail.gmail.com>
	<877hspxyh0.fsf@vostro.rath.org>
	<7465b6170912141131p507f9f3t5d20b1a4d207900b@mail.gmail.com>
	<87ws0p43fv.fsf@vostro.rath.org>
	<7465b6170912142128g38abe9c7g9cdad725e520324b@mail.gmail.com>
	<87fx7cnqn3.fsf@inspiron.ap.columbia.edu>
Message-ID: <7465b6170912151259u45dce2a5n71dc3e4f8c227e5f@mail.gmail.com>

On Tue, Dec 15, 2009 at 6:33 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> I'm still curious though: how do you usually handle dependencies on
> system headers?

I don't. I prefer interface, that doesn't expose system ( low-level )
functionality

> If you use the dependency tracing code, it won't be able
> to tell which dependencies are from system files (and should be
> included) and which dependencies are from project headers (and are going
> to be exported separately).

The dependency functionality was created for a "command line utility".
So by default, in most cases it will do exactly what is needed. I
prefer to explicitly select the desired declarations.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From Nikolaus at rath.org  Wed Dec 16 03:39:41 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Tue, 15 Dec 2009 21:39:41 -0500
Subject: [C++-sig] [Py++] Generate Python code for ctypes?
References: <87y6ldv4sg.fsf@vostro.rath.org>
	<7465b6170912081142v125cc08eqe3ec718236cc8d81@mail.gmail.com>
	<87y6l7wpkx.fsf@vostro.rath.org>
	<7465b6170912122233n4bec93d6kfe43e3c2442c74c2@mail.gmail.com>
	<877hsq91gq.fsf@vostro.rath.org>
	<7465b6170912131251x65404c67o79e56d2a3c0fcc4f@mail.gmail.com>
	<871viyxwcv.fsf@vostro.rath.org>
	<7465b6170912141144t52bcd685q456177a7dd14cc5f@mail.gmail.com>
	<87ocm1420m.fsf@vostro.rath.org>
	<7465b6170912142133o2ed52a26x25510cfe33476ff2@mail.gmail.com>
Message-ID: <87ws0nmyky.fsf@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
> On Tue, Dec 15, 2009 at 12:35 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>> Roman Yakovenko <roman.yakovenko at gmail.com> writes:
>>>> Certainly all the exported symbols should already be available from
>>>> parsing the header file... Is it possible to omit the symbols file
>>>> and generate code based only on the headers?
>>>
>>> I don't think so( it didn't worked in my use case ) but you can try.
>>> Start to comment out code in ctypes_builder.py and post the result :-)
>>
>> I looked into the xml file created by gccxml and (at least in the case
>> of C code) it seems to contain all the information that's needed.
>>
>> I modified parsers.py to return an empty dict if no symbols_file is
>> provided:
>>
>> def merge_information( global_ns, fname, runs_under_unittest=False ):
>> ? ?"""high level function - select the appropriate binary file parser and integrates
>> ? ?the information from the file to the declarations tree. """
>> ? ?if fname is None:
>> ? ? ? ?return dict()
>>
>>
>> this seems to work partially. I can still export all the struct's, but I
>> no longer get any function exports. I can mark them as to be exported
>> without any error, but they don't show up in the generated code.
>>
>> Any idea?
>
> Yes. Py++ thinks that those functions are "private" and doesn't export
> them. You should dig father ( creators_factory/ctypes_creator.py ).

Ah, I think I'm beginning to understand the complexity. I have modified
the code so that it assumes that every defined function is also
exported:

def merge_information( global_ns, fname, runs_under_unittest=False ):
    """high level function - select the appropriate binary file parser and integrates
    the information from the file to the declarations tree. """
    
    # If there is no shared library available for parsing, we just assume
    # that all functions have been exported.
    if fname is None:
        return dict([ [f.name, f] for f in global_ns.calldefs() ])

but this actually includes lots of irrelevant stuff coming from other
included headers. You are right, parsing the actual shared library is
probably a good thing(tm).

I will concentrate on finding a good way to determine the path of a
shared library instead. At least under linux, it seems to be a good way
to parse the 'ldconfig -p' output...

Best,


   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From troy at resophonic.com  Thu Dec 17 17:08:06 2009
From: troy at resophonic.com (Troy D. Straszheim)
Date: Thu, 17 Dec 2009 11:08:06 -0500
Subject: [C++-sig] Implementation of proper overload resolution
Message-ID: <87aaxha8ih.fsf@resophonic.com>


Here's what I've got on overloading.  This turned out to be a lot more
work, and this mail much longer, than I'd hoped.  Would appreciate a
readthrough and comments/questions, I've done about all I can do.  First
a review of the problems, then a walkthrough of an implementation I have
that fixes them.

1.a)  Problem: Ambiguous calls from python to overloaded functions

Boost.python's 'overloading' is and has always been broken w.r.t. is
ambiguity.  The library currently has no notion, given some tuple of
arguments, of one function being a better match than another.  It knows
only "this function matches" or not.  What the user observes is the
following: when a call to a c++ function is made from python e.g.

>>> m.f(1)

boost.python looks through the overloads for f, in the reverse order of
registration, and calls the first one that is callable.  For instance if
the registrations are

void f_int(int);
void f_bool(bool);

BOOST_PYTHON_MODULE(m)
{
  def("f", f_int);
  def("f", f_bool);
}

then f_bool will execute this call of f(1), since the python 'int'
converts to c++ bool.  If the overloads were registered in the reverse
order, f_int would be executed.

In this case the c++ overloads in question are potentially
distinguishable from one another, as python has distinct types bool and
int.  So these aren't "necessarily" ambiguous.

1.b) Problem: "Necessarily" ambiguous overloads

There are some overload sets that *are* necessarily ambiguous.  This
set, for instance,

  void f_double(double);
  void f_float(float);
  
  BOOST_PYTHON_MODULE(m)
  {
    def("f", f_double);
    def("f", f_float);
  }

can never be unambiguous, since python has no 'float' type.  I.e.

  >>> f(x)

will call f_float for any value of x where the call succeeds at all.

1.c)  Problem: Multiple values for keyword argument

At the same place in the boost.python code where this 'first-match'
overload resolution is done, the user error 'multiple values for keyword
argument' is not checked.  Neal Becker recently pointed this out.  With
boost.python it looks like this:

  int addem(int x, int y, int z) { return x*100 + y*10 + z; }
  
  BOOST_PYTHON_MODULE(M)
  {
    def("addem", &addem, (arg("x"), arg("y"), arg("z")));
  }
  
  >>> from M import addem
  >>> addem(1, 8, 2, x=4)
  Traceback (most recent call last):
  ...
  ArgumentError: Python argument types in
      M.addem(int, int, int)
  did not match C++ signature:
      addem(int x, int y, int z)
  
That error message is very confusing...  f(int,int,int) doesn't match
f(int,int,int)?  The pure python version reports something more
sensible:

  >>> def g(x,y,z, **kwargs):
  ...     print x,y,z,kwargs  
  ... 
  >>> g(1,2,3,x=4)
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
  TypeError: g() got multiple values for keyword argument 'x'
  

2.) An implemented solution

I've got something implemented.  Here's what it does.

2.a) Solution: multiple values for keyword

The easiest case to catch is the last [1.c].  It is also orthogonal to
the others.  If a registered function has keyword arguments, check for
multiple keyword values, and raise an error if so:

  >>> from M import addem
  >>> addem(1,2,3,x=1)
  Boost.Python.TypeError: M.addem() got multiple values for keyword argument 'x'

2.b) Solution: "necessarily" ambiguous overloads

The next easiest thing to catch is case [1.b], "necessarily" ambiguous
registrations.  It proceeds as follows: at import time, as each new
overload V for function F is registered, compare V to the existing
overloads EO for F.  If V has the same signature as something in EO,
raise an AmbiguousOverload exception.  For instance, if you load the
module from [1.b] above, you get:

  >>> import m
  Traceback (most recent call last):
  ...
  AmbiguousOverload: Boost.Python function m.f
  has ambiguous overloads.  C++ signatures
    f(float) -> None
  and
    f(double) -> None
  are indistinguishable to python.

Again this is because c++ float and c++ double both map to python
'float'.   This one is "easy" as it happens only once (not at every
function call) and doesn't take up any space.

2.c) Solution: ambiguous calls

The hard case is [1.a]:

  void f_bool(bool);
  void f_int(int);
  
  BOOST_PYTHON_MODULE(m)
  {
     def(f, f_bool);
     def(f, f_int);
  }
   
For module 'm' above, a call to f(True) or f(1) should succeed and call
the corresponding function.  Passing a float, however, is ambiguous:

  >>> f(True)  # ok
  >>> f(1)     # ok
  >>> f(1.0)
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
  Boost.Python.AmbiguousCall: Ambiguous call to Boost.Python function m.f
  C++ signatures:
      f(int)
      f(bool)

So the implementation has some how 'scored' each possible overload and
in each case either used the best one or reported ambiguity if multiple
overloads are tied for 'best'.

The scoring works as follows.

In the guts of boost.python we recieve a tuple of arguments of type
PyObject*.  We also have a list of overloads, each with an mpl vector
representing the signature of the associated C++ function.  This is
unchanged from the released version.

What has been done until now is to attempt to call each overload in
order and return the result of the first that succeeds.  

In this new implmentation, for each c++ argument of type T_n, we ask a
class overload_score<T_n> to score the conversion of the type
(PyTypeObject*) of this PyObject* to T_n.  Example:

   template <>
   struct overload_score<int>
   {
     boost::optional<unsigned> operator()(PyObject* type)
     {
       if(PyInt_CheckExact(type))
           return boost::optional<unsigned>(0); // int == perfect
       else if (PyBool_Check(type))
           return boost::optional<unsigned>(1); // bool == okay 
       else if (PyFloat_CheckExact(type))
         return boost::optional<unsigned>(1);   // float == okay
       else if(arg_from_python<int>(type).convertible())
         return boost::optional<unsigned>(1);   // fallback
       else
         return boost::optional<unsigned>();    // unsuitable
     }
   };

The "score" type is optional<unsigned>.  A valueless,
(default-constructed) optional<unsigned> means 'unsuitable'.
optional<unsigned>(0) is a perfect match, and optional<unsigned>(value)
for value > 0 is a workable but less than perfect match.

These per-argument scores are added together for all arguments: this is
the overload's total score.  If any argument is 'unsuitable', the total
score is 'unsuitable'.

If there is a tie for the best (lowest) score, the call is ambiguous and
a Boost.Python.AmbiguousCall exception is raised.  If there is only one
function in first place, call it.

3.)  Implications

This breaks a few corner cases that I've found in the tests.

3.a) implied init<> registration

I have found a few instances like this one:

  struct X 
  { 
    X();
    X(int);
  };

  class_<X>("X")       // #1
    .def(init<>())     // #2
    .def(init<int>()) 
    ;

Here, #1 causes a default constructor to be registered, as does #2.
This will cause a throw at load time as in [2.b].  It is simple to fix.

3.b)  raw_constructor

The test suite for raw_constructor depends on our first_match overload
resolution.  The fix is to move a few things around to get the same
effect without relying on this behavior.   

The "before" code looks like this:

  class Foo
  {
   public:
      Foo(tuple args, dict kw)
        : args(args), kw(kw) {}
      
      tuple args;
      dict kw;
  };
  
  object init_foo(tuple args, dict kw)
  {
      tuple rest(args.slice(1,_));
      return args[0].attr("__init__")(rest, kw);
  }
  
  BOOST_PYTHON_MODULE(raw_ctor_ext)
  {
      // using no_init postpones defining __init__ function until after
      // raw_function for proper overload resolution order, since later
      // defs get higher priority.
      class_<Foo>("Foo", no_init) 
          .def("__init__", raw_function(&init_foo))
          .def(init<tuple, dict>())
          .def_readwrite("args", &Foo::args)
          .def_readwrite("kw", &Foo::kw)
          ;
  }
  
The "after" code does the registration piecemeal: Maybe there is a
better way to do this.  To my mind this is a little better because it
makes explicit what is happening rather than relying on some subtle
property of overload resolution:

  class Foo
  {
   public:
    Foo(tuple args, dict kw)
      : args(args), kw(kw) {}
      
      tuple args;
      dict kw;
  };
  
  // our custom factory function
  object init_foo(tuple args, dict kw)  
  {
      tuple rest(args.slice(1,_));
      return args[0].attr("__real_init__")(rest, kw);
  }
  
  BOOST_PYTHON_MODULE(raw_ctor_ext)
  {
    // to get the desired effect we register
    // the actual constructor and our 'raw' constructor,
    // and then rename them
    class_<Foo> c("Foo", init<tuple, dict>()); 
    c
      .def("__tmp_init__", raw_function(&init_foo))
      .def_readwrite("args", &Foo::args)
      .def_readwrite("kw", &Foo::kw)
      ;
  
    //
    //  __init__ => __real_init__
    //  __tmp_init__ => __init__
    //
    object real_constructor = getattr(c, "__init__");
    object raw_constructor = getattr(c, "__tmp_init__");
  
    setattr(c, "__init__", raw_constructor);
    delattr(c, "__tmp_init__");
    setattr(c, "__real_init__", real_constructor);
  
  }

And that basically covers it.  Looking forward to comments/feedback.

The code is here:

  http://gitorious.org/~straszheim/boost/straszheims-python/trees/master

Note that the headers are in subdirectory 'include'...  If there is
enough response to this mail it I'll send out instructions on how to get
a working build going.  

- t

From blake at hailmail.net  Thu Dec 17 20:11:50 2009
From: blake at hailmail.net (Dane Springmeyer)
Date: Thu, 17 Dec 2009 11:11:50 -0800
Subject: [C++-sig] Implementation of proper overload resolution
In-Reply-To: <87aaxha8ih.fsf@resophonic.com>
References: <87aaxha8ih.fsf@resophonic.com>
Message-ID: <0EFC2868-BA76-4E73-8DF2-075C2D73ABD9@hailmail.net>

Troy,

Really __impressive__ write-up and work on this. I'd definitely be  
interested in build instructions and your work getting into future  
releases.

Cheers,

Dane


On Dec 17, 2009, at 8:08 AM, Troy D. Straszheim wrote:

>
> Here's what I've got on overloading.  This turned out to be a lot more
> work, and this mail much longer, than I'd hoped.  Would appreciate a
> readthrough and comments/questions, I've done about all I can do.   
> First
> a review of the problems, then a walkthrough of an implementation I  
> have
> that fixes them.
>
> 1.a)  Problem: Ambiguous calls from python to overloaded functions
>
> Boost.python's 'overloading' is and has always been broken w.r.t. is
> ambiguity.  The library currently has no notion, given some tuple of
> arguments, of one function being a better match than another.  It  
> knows
> only "this function matches" or not.  What the user observes is the
> following: when a call to a c++ function is made from python e.g.
>
>>>> m.f(1)
>
> boost.python looks through the overloads for f, in the reverse order  
> of
> registration, and calls the first one that is callable.  For  
> instance if
> the registrations are
>
> void f_int(int);
> void f_bool(bool);
>
> BOOST_PYTHON_MODULE(m)
> {
>  def("f", f_int);
>  def("f", f_bool);
> }
>
> then f_bool will execute this call of f(1), since the python 'int'
> converts to c++ bool.  If the overloads were registered in the reverse
> order, f_int would be executed.
>
> In this case the c++ overloads in question are potentially
> distinguishable from one another, as python has distinct types bool  
> and
> int.  So these aren't "necessarily" ambiguous.
>
> 1.b) Problem: "Necessarily" ambiguous overloads
>
> There are some overload sets that *are* necessarily ambiguous.  This
> set, for instance,
>
>  void f_double(double);
>  void f_float(float);
>
>  BOOST_PYTHON_MODULE(m)
>  {
>    def("f", f_double);
>    def("f", f_float);
>  }
>
> can never be unambiguous, since python has no 'float' type.  I.e.
>
>>>> f(x)
>
> will call f_float for any value of x where the call succeeds at all.
>
> 1.c)  Problem: Multiple values for keyword argument
>
> At the same place in the boost.python code where this 'first-match'
> overload resolution is done, the user error 'multiple values for  
> keyword
> argument' is not checked.  Neal Becker recently pointed this out.   
> With
> boost.python it looks like this:
>
>  int addem(int x, int y, int z) { return x*100 + y*10 + z; }
>
>  BOOST_PYTHON_MODULE(M)
>  {
>    def("addem", &addem, (arg("x"), arg("y"), arg("z")));
>  }
>
>>>> from M import addem
>>>> addem(1, 8, 2, x=4)
>  Traceback (most recent call last):
>  ...
>  ArgumentError: Python argument types in
>      M.addem(int, int, int)
>  did not match C++ signature:
>      addem(int x, int y, int z)
>
> That error message is very confusing...  f(int,int,int) doesn't match
> f(int,int,int)?  The pure python version reports something more
> sensible:
>
>>>> def g(x,y,z, **kwargs):
>  ...     print x,y,z,kwargs
>  ...
>>>> g(1,2,3,x=4)
>  Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>  TypeError: g() got multiple values for keyword argument 'x'
>
>
> 2.) An implemented solution
>
> I've got something implemented.  Here's what it does.
>
> 2.a) Solution: multiple values for keyword
>
> The easiest case to catch is the last [1.c].  It is also orthogonal to
> the others.  If a registered function has keyword arguments, check for
> multiple keyword values, and raise an error if so:
>
>>>> from M import addem
>>>> addem(1,2,3,x=1)
>  Boost.Python.TypeError: M.addem() got multiple values for keyword  
> argument 'x'
>
> 2.b) Solution: "necessarily" ambiguous overloads
>
> The next easiest thing to catch is case [1.b], "necessarily" ambiguous
> registrations.  It proceeds as follows: at import time, as each new
> overload V for function F is registered, compare V to the existing
> overloads EO for F.  If V has the same signature as something in EO,
> raise an AmbiguousOverload exception.  For instance, if you load the
> module from [1.b] above, you get:
>
>>>> import m
>  Traceback (most recent call last):
>  ...
>  AmbiguousOverload: Boost.Python function m.f
>  has ambiguous overloads.  C++ signatures
>    f(float) -> None
>  and
>    f(double) -> None
>  are indistinguishable to python.
>
> Again this is because c++ float and c++ double both map to python
> 'float'.   This one is "easy" as it happens only once (not at every
> function call) and doesn't take up any space.
>
> 2.c) Solution: ambiguous calls
>
> The hard case is [1.a]:
>
>  void f_bool(bool);
>  void f_int(int);
>
>  BOOST_PYTHON_MODULE(m)
>  {
>     def(f, f_bool);
>     def(f, f_int);
>  }
>
> For module 'm' above, a call to f(True) or f(1) should succeed and  
> call
> the corresponding function.  Passing a float, however, is ambiguous:
>
>>>> f(True)  # ok
>>>> f(1)     # ok
>>>> f(1.0)
>  Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>  Boost.Python.AmbiguousCall: Ambiguous call to Boost.Python function  
> m.f
>  C++ signatures:
>      f(int)
>      f(bool)
>
> So the implementation has some how 'scored' each possible overload and
> in each case either used the best one or reported ambiguity if  
> multiple
> overloads are tied for 'best'.
>
> The scoring works as follows.
>
> In the guts of boost.python we recieve a tuple of arguments of type
> PyObject*.  We also have a list of overloads, each with an mpl vector
> representing the signature of the associated C++ function.  This is
> unchanged from the released version.
>
> What has been done until now is to attempt to call each overload in
> order and return the result of the first that succeeds.
>
> In this new implmentation, for each c++ argument of type T_n, we ask a
> class overload_score<T_n> to score the conversion of the type
> (PyTypeObject*) of this PyObject* to T_n.  Example:
>
>   template <>
>   struct overload_score<int>
>   {
>     boost::optional<unsigned> operator()(PyObject* type)
>     {
>       if(PyInt_CheckExact(type))
>           return boost::optional<unsigned>(0); // int == perfect
>       else if (PyBool_Check(type))
>           return boost::optional<unsigned>(1); // bool == okay
>       else if (PyFloat_CheckExact(type))
>         return boost::optional<unsigned>(1);   // float == okay
>       else if(arg_from_python<int>(type).convertible())
>         return boost::optional<unsigned>(1);   // fallback
>       else
>         return boost::optional<unsigned>();    // unsuitable
>     }
>   };
>
> The "score" type is optional<unsigned>.  A valueless,
> (default-constructed) optional<unsigned> means 'unsuitable'.
> optional<unsigned>(0) is a perfect match, and  
> optional<unsigned>(value)
> for value > 0 is a workable but less than perfect match.
>
> These per-argument scores are added together for all arguments: this  
> is
> the overload's total score.  If any argument is 'unsuitable', the  
> total
> score is 'unsuitable'.
>
> If there is a tie for the best (lowest) score, the call is ambiguous  
> and
> a Boost.Python.AmbiguousCall exception is raised.  If there is only  
> one
> function in first place, call it.
>
> 3.)  Implications
>
> This breaks a few corner cases that I've found in the tests.
>
> 3.a) implied init<> registration
>
> I have found a few instances like this one:
>
>  struct X
>  {
>    X();
>    X(int);
>  };
>
>  class_<X>("X")       // #1
>    .def(init<>())     // #2
>    .def(init<int>())
>    ;
>
> Here, #1 causes a default constructor to be registered, as does #2.
> This will cause a throw at load time as in [2.b].  It is simple to  
> fix.
>
> 3.b)  raw_constructor
>
> The test suite for raw_constructor depends on our first_match overload
> resolution.  The fix is to move a few things around to get the same
> effect without relying on this behavior.
>
> The "before" code looks like this:
>
>  class Foo
>  {
>   public:
>      Foo(tuple args, dict kw)
>        : args(args), kw(kw) {}
>
>      tuple args;
>      dict kw;
>  };
>
>  object init_foo(tuple args, dict kw)
>  {
>      tuple rest(args.slice(1,_));
>      return args[0].attr("__init__")(rest, kw);
>  }
>
>  BOOST_PYTHON_MODULE(raw_ctor_ext)
>  {
>      // using no_init postpones defining __init__ function until after
>      // raw_function for proper overload resolution order, since later
>      // defs get higher priority.
>      class_<Foo>("Foo", no_init)
>          .def("__init__", raw_function(&init_foo))
>          .def(init<tuple, dict>())
>          .def_readwrite("args", &Foo::args)
>          .def_readwrite("kw", &Foo::kw)
>          ;
>  }
>
> The "after" code does the registration piecemeal: Maybe there is a
> better way to do this.  To my mind this is a little better because it
> makes explicit what is happening rather than relying on some subtle
> property of overload resolution:
>
>  class Foo
>  {
>   public:
>    Foo(tuple args, dict kw)
>      : args(args), kw(kw) {}
>
>      tuple args;
>      dict kw;
>  };
>
>  // our custom factory function
>  object init_foo(tuple args, dict kw)
>  {
>      tuple rest(args.slice(1,_));
>      return args[0].attr("__real_init__")(rest, kw);
>  }
>
>  BOOST_PYTHON_MODULE(raw_ctor_ext)
>  {
>    // to get the desired effect we register
>    // the actual constructor and our 'raw' constructor,
>    // and then rename them
>    class_<Foo> c("Foo", init<tuple, dict>());
>    c
>      .def("__tmp_init__", raw_function(&init_foo))
>      .def_readwrite("args", &Foo::args)
>      .def_readwrite("kw", &Foo::kw)
>      ;
>
>    //
>    //  __init__ => __real_init__
>    //  __tmp_init__ => __init__
>    //
>    object real_constructor = getattr(c, "__init__");
>    object raw_constructor = getattr(c, "__tmp_init__");
>
>    setattr(c, "__init__", raw_constructor);
>    delattr(c, "__tmp_init__");
>    setattr(c, "__real_init__", real_constructor);
>
>  }
>
> And that basically covers it.  Looking forward to comments/feedback.
>
> The code is here:
>
>  http://gitorious.org/~straszheim/boost/straszheims-python/trees/ 
> master
>
> Note that the headers are in subdirectory 'include'...  If there is
> enough response to this mail it I'll send out instructions on how to  
> get
> a working build going.
>
> - t
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig


From blake at hailmail.net  Thu Dec 17 20:31:02 2009
From: blake at hailmail.net (Dane Springmeyer)
Date: Thu, 17 Dec 2009 11:31:02 -0800
Subject: [C++-sig] Implementation of proper overload resolution
In-Reply-To: <0EFC2868-BA76-4E73-8DF2-075C2D73ABD9@hailmail.net>
References: <87aaxha8ih.fsf@resophonic.com>
	<0EFC2868-BA76-4E73-8DF2-075C2D73ABD9@hailmail.net>
Message-ID: <9D77FF39-BA7A-476F-9CBF-48009A1D7919@hailmail.net>

Troy,

Incidentally, do you know the proper way  (or is there a proper way?)  
to support None type keyword arguments?

Cheers,

Dane


On Dec 17, 2009, at 11:11 AM, Dane Springmeyer wrote:

> Troy,
>
> Really __impressive__ write-up and work on this. I'd definitely be  
> interested in build instructions and your work getting into future  
> releases.
>
> Cheers,
>
> Dane
>
>
> On Dec 17, 2009, at 8:08 AM, Troy D. Straszheim wrote:
>
>>
>> Here's what I've got on overloading.  This turned out to be a lot  
>> more
>> work, and this mail much longer, than I'd hoped.  Would appreciate a
>> readthrough and comments/questions, I've done about all I can do.   
>> First
>> a review of the problems, then a walkthrough of an implementation I  
>> have
>> that fixes them.
>>
>> 1.a)  Problem: Ambiguous calls from python to overloaded functions
>>
>> Boost.python's 'overloading' is and has always been broken w.r.t. is
>> ambiguity.  The library currently has no notion, given some tuple of
>> arguments, of one function being a better match than another.  It  
>> knows
>> only "this function matches" or not.  What the user observes is the
>> following: when a call to a c++ function is made from python e.g.
>>
>>>>> m.f(1)
>>
>> boost.python looks through the overloads for f, in the reverse  
>> order of
>> registration, and calls the first one that is callable.  For  
>> instance if
>> the registrations are
>>
>> void f_int(int);
>> void f_bool(bool);
>>
>> BOOST_PYTHON_MODULE(m)
>> {
>> def("f", f_int);
>> def("f", f_bool);
>> }
>>
>> then f_bool will execute this call of f(1), since the python 'int'
>> converts to c++ bool.  If the overloads were registered in the  
>> reverse
>> order, f_int would be executed.
>>
>> In this case the c++ overloads in question are potentially
>> distinguishable from one another, as python has distinct types bool  
>> and
>> int.  So these aren't "necessarily" ambiguous.
>>
>> 1.b) Problem: "Necessarily" ambiguous overloads
>>
>> There are some overload sets that *are* necessarily ambiguous.  This
>> set, for instance,
>>
>> void f_double(double);
>> void f_float(float);
>>
>> BOOST_PYTHON_MODULE(m)
>> {
>>   def("f", f_double);
>>   def("f", f_float);
>> }
>>
>> can never be unambiguous, since python has no 'float' type.  I.e.
>>
>>>>> f(x)
>>
>> will call f_float for any value of x where the call succeeds at all.
>>
>> 1.c)  Problem: Multiple values for keyword argument
>>
>> At the same place in the boost.python code where this 'first-match'
>> overload resolution is done, the user error 'multiple values for  
>> keyword
>> argument' is not checked.  Neal Becker recently pointed this out.   
>> With
>> boost.python it looks like this:
>>
>> int addem(int x, int y, int z) { return x*100 + y*10 + z; }
>>
>> BOOST_PYTHON_MODULE(M)
>> {
>>   def("addem", &addem, (arg("x"), arg("y"), arg("z")));
>> }
>>
>>>>> from M import addem
>>>>> addem(1, 8, 2, x=4)
>> Traceback (most recent call last):
>> ...
>> ArgumentError: Python argument types in
>>     M.addem(int, int, int)
>> did not match C++ signature:
>>     addem(int x, int y, int z)
>>
>> That error message is very confusing...  f(int,int,int) doesn't match
>> f(int,int,int)?  The pure python version reports something more
>> sensible:
>>
>>>>> def g(x,y,z, **kwargs):
>> ...     print x,y,z,kwargs
>> ...
>>>>> g(1,2,3,x=4)
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>> TypeError: g() got multiple values for keyword argument 'x'
>>
>>
>> 2.) An implemented solution
>>
>> I've got something implemented.  Here's what it does.
>>
>> 2.a) Solution: multiple values for keyword
>>
>> The easiest case to catch is the last [1.c].  It is also orthogonal  
>> to
>> the others.  If a registered function has keyword arguments, check  
>> for
>> multiple keyword values, and raise an error if so:
>>
>>>>> from M import addem
>>>>> addem(1,2,3,x=1)
>> Boost.Python.TypeError: M.addem() got multiple values for keyword  
>> argument 'x'
>>
>> 2.b) Solution: "necessarily" ambiguous overloads
>>
>> The next easiest thing to catch is case [1.b], "necessarily"  
>> ambiguous
>> registrations.  It proceeds as follows: at import time, as each new
>> overload V for function F is registered, compare V to the existing
>> overloads EO for F.  If V has the same signature as something in EO,
>> raise an AmbiguousOverload exception.  For instance, if you load the
>> module from [1.b] above, you get:
>>
>>>>> import m
>> Traceback (most recent call last):
>> ...
>> AmbiguousOverload: Boost.Python function m.f
>> has ambiguous overloads.  C++ signatures
>>   f(float) -> None
>> and
>>   f(double) -> None
>> are indistinguishable to python.
>>
>> Again this is because c++ float and c++ double both map to python
>> 'float'.   This one is "easy" as it happens only once (not at every
>> function call) and doesn't take up any space.
>>
>> 2.c) Solution: ambiguous calls
>>
>> The hard case is [1.a]:
>>
>> void f_bool(bool);
>> void f_int(int);
>>
>> BOOST_PYTHON_MODULE(m)
>> {
>>    def(f, f_bool);
>>    def(f, f_int);
>> }
>>
>> For module 'm' above, a call to f(True) or f(1) should succeed and  
>> call
>> the corresponding function.  Passing a float, however, is ambiguous:
>>
>>>>> f(True)  # ok
>>>>> f(1)     # ok
>>>>> f(1.0)
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>> Boost.Python.AmbiguousCall: Ambiguous call to Boost.Python function  
>> m.f
>> C++ signatures:
>>     f(int)
>>     f(bool)
>>
>> So the implementation has some how 'scored' each possible overload  
>> and
>> in each case either used the best one or reported ambiguity if  
>> multiple
>> overloads are tied for 'best'.
>>
>> The scoring works as follows.
>>
>> In the guts of boost.python we recieve a tuple of arguments of type
>> PyObject*.  We also have a list of overloads, each with an mpl vector
>> representing the signature of the associated C++ function.  This is
>> unchanged from the released version.
>>
>> What has been done until now is to attempt to call each overload in
>> order and return the result of the first that succeeds.
>>
>> In this new implmentation, for each c++ argument of type T_n, we  
>> ask a
>> class overload_score<T_n> to score the conversion of the type
>> (PyTypeObject*) of this PyObject* to T_n.  Example:
>>
>>  template <>
>>  struct overload_score<int>
>>  {
>>    boost::optional<unsigned> operator()(PyObject* type)
>>    {
>>      if(PyInt_CheckExact(type))
>>          return boost::optional<unsigned>(0); // int == perfect
>>      else if (PyBool_Check(type))
>>          return boost::optional<unsigned>(1); // bool == okay
>>      else if (PyFloat_CheckExact(type))
>>        return boost::optional<unsigned>(1);   // float == okay
>>      else if(arg_from_python<int>(type).convertible())
>>        return boost::optional<unsigned>(1);   // fallback
>>      else
>>        return boost::optional<unsigned>();    // unsuitable
>>    }
>>  };
>>
>> The "score" type is optional<unsigned>.  A valueless,
>> (default-constructed) optional<unsigned> means 'unsuitable'.
>> optional<unsigned>(0) is a perfect match, and  
>> optional<unsigned>(value)
>> for value > 0 is a workable but less than perfect match.
>>
>> These per-argument scores are added together for all arguments:  
>> this is
>> the overload's total score.  If any argument is 'unsuitable', the  
>> total
>> score is 'unsuitable'.
>>
>> If there is a tie for the best (lowest) score, the call is  
>> ambiguous and
>> a Boost.Python.AmbiguousCall exception is raised.  If there is only  
>> one
>> function in first place, call it.
>>
>> 3.)  Implications
>>
>> This breaks a few corner cases that I've found in the tests.
>>
>> 3.a) implied init<> registration
>>
>> I have found a few instances like this one:
>>
>> struct X
>> {
>>   X();
>>   X(int);
>> };
>>
>> class_<X>("X")       // #1
>>   .def(init<>())     // #2
>>   .def(init<int>())
>>   ;
>>
>> Here, #1 causes a default constructor to be registered, as does #2.
>> This will cause a throw at load time as in [2.b].  It is simple to  
>> fix.
>>
>> 3.b)  raw_constructor
>>
>> The test suite for raw_constructor depends on our first_match  
>> overload
>> resolution.  The fix is to move a few things around to get the same
>> effect without relying on this behavior.
>>
>> The "before" code looks like this:
>>
>> class Foo
>> {
>>  public:
>>     Foo(tuple args, dict kw)
>>       : args(args), kw(kw) {}
>>
>>     tuple args;
>>     dict kw;
>> };
>>
>> object init_foo(tuple args, dict kw)
>> {
>>     tuple rest(args.slice(1,_));
>>     return args[0].attr("__init__")(rest, kw);
>> }
>>
>> BOOST_PYTHON_MODULE(raw_ctor_ext)
>> {
>>     // using no_init postpones defining __init__ function until after
>>     // raw_function for proper overload resolution order, since later
>>     // defs get higher priority.
>>     class_<Foo>("Foo", no_init)
>>         .def("__init__", raw_function(&init_foo))
>>         .def(init<tuple, dict>())
>>         .def_readwrite("args", &Foo::args)
>>         .def_readwrite("kw", &Foo::kw)
>>         ;
>> }
>>
>> The "after" code does the registration piecemeal: Maybe there is a
>> better way to do this.  To my mind this is a little better because it
>> makes explicit what is happening rather than relying on some subtle
>> property of overload resolution:
>>
>> class Foo
>> {
>>  public:
>>   Foo(tuple args, dict kw)
>>     : args(args), kw(kw) {}
>>
>>     tuple args;
>>     dict kw;
>> };
>>
>> // our custom factory function
>> object init_foo(tuple args, dict kw)
>> {
>>     tuple rest(args.slice(1,_));
>>     return args[0].attr("__real_init__")(rest, kw);
>> }
>>
>> BOOST_PYTHON_MODULE(raw_ctor_ext)
>> {
>>   // to get the desired effect we register
>>   // the actual constructor and our 'raw' constructor,
>>   // and then rename them
>>   class_<Foo> c("Foo", init<tuple, dict>());
>>   c
>>     .def("__tmp_init__", raw_function(&init_foo))
>>     .def_readwrite("args", &Foo::args)
>>     .def_readwrite("kw", &Foo::kw)
>>     ;
>>
>>   //
>>   //  __init__ => __real_init__
>>   //  __tmp_init__ => __init__
>>   //
>>   object real_constructor = getattr(c, "__init__");
>>   object raw_constructor = getattr(c, "__tmp_init__");
>>
>>   setattr(c, "__init__", raw_constructor);
>>   delattr(c, "__tmp_init__");
>>   setattr(c, "__real_init__", real_constructor);
>>
>> }
>>
>> And that basically covers it.  Looking forward to comments/feedback.
>>
>> The code is here:
>>
>> http://gitorious.org/~straszheim/boost/straszheims-python/trees/ 
>> master
>>
>> Note that the headers are in subdirectory 'include'...  If there is
>> enough response to this mail it I'll send out instructions on how  
>> to get
>> a working build going.
>>
>> - t
>> _______________________________________________
>> Cplusplus-sig mailing list
>> Cplusplus-sig at python.org
>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig


From ndbecker2 at gmail.com  Thu Dec 17 23:13:35 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Thu, 17 Dec 2009 17:13:35 -0500
Subject: [C++-sig] Implementation of proper overload resolution
References: <87aaxha8ih.fsf@resophonic.com>
Message-ID: <hgeaef$noo$1@ger.gmane.org>

I assume overload resolution extends to scoring multiple arguments as well?


From troy at resophonic.com  Thu Dec 17 23:54:58 2009
From: troy at resophonic.com (Troy D. Straszheim)
Date: Thu, 17 Dec 2009 17:54:58 -0500
Subject: [C++-sig] Implementation of proper overload resolution
In-Reply-To: <hgeaef$noo$1@ger.gmane.org> (Neal Becker's message of "Thu, 17
	Dec 2009 17:13:35 -0500")
References: <87aaxha8ih.fsf@resophonic.com> <hgeaef$noo$1@ger.gmane.org>
Message-ID: <874onptdml.fsf@resophonic.com>

Neal Becker <ndbecker2 at gmail.com> writes:

> I assume overload resolution extends to scoring multiple arguments as well?
>

Sure.  This is the reason that scores are optional<unsigned>.  If any
single argument scorer returns optional<unsigned>() (meaning 'unsuitable'),
this stops evaluation and kills the score for the entire signature.
Here are some excerpts from the test suites:

//------------------------------------------------------

std::string f1(float, bool)         { return "float,bool";        }
std::string f2(float, int)          { return "float,int";         }
std::string f3(float, std::string)  { return "float,std::string"; }

BOOST_PYTHON_MODULE(ambig6_ext)
{
  def("f", &f1);
  def("f", &f2);
  def("f", &f3);
}

//------------------------------------------------------

>>> from ambig6_ext import f
>>> help(f)
Help on built-in function f:

f(...)
    f( (float)arg1, (bool)arg2) -> str :
    
        C++ signature :
            std::string f(float,bool)
    
    f( (float)arg1, (int)arg2) -> str :
    
        C++ signature :
            std::string f(float,int)
    
    f( (float)arg1, (str)arg2) -> str :
    
        C++ signature :
            std::string f(float,std::string)

>>> f(1.0, True)    # perfect match, score 0
'float,bool'
>>> f(1.0, 1)       # perfect match, score 0 
'float,int'
>>> f(True, True)   # best match arg1 score is 1, arg2 score is 0
'float,bool'


#
# Here, note that f3 is not listed in the set of ambiguous candidates,
# as the arg2 score (conversion of float to string) is 'unsuitable' and
# removes it from consideration
#
>>> f(1.0,1.0)      # ambiguous, f1 & f2 both score 1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Boost.Python.AmbiguousCall: Ambiguous call to Boost.Python function ambig6_ext.f
C++ signatures:
    f(float, int)
    f(float, bool)


#
#  If the second arg is a string, then the 3rd overload
#  always matches...
#
>>> f(True, 'helloverloading')   # best match, score 1
'float,std::string'
>>> f(1, 'helloverloading')      # best match, score 1
'float,std::string'
>>> f(1.0, 'helloverloading')    # perfect match, score 0
'float,std::string'


#
# ... Unless the arg1 is also a string.  Note this one is ArgumentError,
#  listing all overloads, not AmbiguousCall listing the ambig. ones.
#
>>> f('uh', 'oh')                # all overloads score 'unsuitable'
Boost.Python.ArgumentError: Python argument types in
    ambig6_ext.f(str, str)
did not match C++ signature:
    f(float, std::string)
    f(float, int)
    f(float, bool)


-t

From ndbecker2 at gmail.com  Fri Dec 18 13:01:42 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Fri, 18 Dec 2009 07:01:42 -0500
Subject: [C++-sig] Implementation of proper overload resolution
References: <87aaxha8ih.fsf@resophonic.com> <hgeaef$noo$1@ger.gmane.org>
	<874onptdml.fsf@resophonic.com>
Message-ID: <hgfqv5$le5$1@ger.gmane.org>

I am concerned that this doesn't introduce too much overhead for the common 
case where there is no ambiguity.  I suppose this has been optimized?


From gjcarneiro at gmail.com  Fri Dec 18 16:08:24 2009
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Fri, 18 Dec 2009 15:08:24 +0000
Subject: [C++-sig] Implementation of proper overload resolution
In-Reply-To: <87aaxha8ih.fsf@resophonic.com>
References: <87aaxha8ih.fsf@resophonic.com>
Message-ID: <a467ca4f0912180708h7dd7b2e3of1dce8e1f5da7de0@mail.gmail.com>

Don't you think that when these overloading problems become an issue it is a
sign of a poorly designed API?  I mean, if overloaded functions parameters
are not completely different in type or number, then maybe they are already
too confusing to use and should not be overloaded in the first place?...

Just a thought...

2009/12/17 Troy D. Straszheim <troy at resophonic.com>

>
> Here's what I've got on overloading.  This turned out to be a lot more
> work, and this mail much longer, than I'd hoped.  Would appreciate a
> readthrough and comments/questions, I've done about all I can do.  First
> a review of the problems, then a walkthrough of an implementation I have
> that fixes them.
>
> 1.a)  Problem: Ambiguous calls from python to overloaded functions
>
> Boost.python's 'overloading' is and has always been broken w.r.t. is
> ambiguity.  The library currently has no notion, given some tuple of
> arguments, of one function being a better match than another.  It knows
> only "this function matches" or not.  What the user observes is the
> following: when a call to a c++ function is made from python e.g.
>
> >>> m.f(1)
>
> boost.python looks through the overloads for f, in the reverse order of
> registration, and calls the first one that is callable.  For instance if
> the registrations are
>
> void f_int(int);
> void f_bool(bool);
>
> BOOST_PYTHON_MODULE(m)
> {
>  def("f", f_int);
>  def("f", f_bool);
> }
>
> then f_bool will execute this call of f(1), since the python 'int'
> converts to c++ bool.  If the overloads were registered in the reverse
> order, f_int would be executed.
>
> In this case the c++ overloads in question are potentially
> distinguishable from one another, as python has distinct types bool and
> int.  So these aren't "necessarily" ambiguous.
>
> 1.b) Problem: "Necessarily" ambiguous overloads
>
> There are some overload sets that *are* necessarily ambiguous.  This
> set, for instance,
>
>  void f_double(double);
>  void f_float(float);
>
>  BOOST_PYTHON_MODULE(m)
>  {
>    def("f", f_double);
>    def("f", f_float);
>  }
>
> can never be unambiguous, since python has no 'float' type.  I.e.
>
>  >>> f(x)
>
> will call f_float for any value of x where the call succeeds at all.
>
> 1.c)  Problem: Multiple values for keyword argument
>
> At the same place in the boost.python code where this 'first-match'
> overload resolution is done, the user error 'multiple values for keyword
> argument' is not checked.  Neal Becker recently pointed this out.  With
> boost.python it looks like this:
>
>  int addem(int x, int y, int z) { return x*100 + y*10 + z; }
>
>  BOOST_PYTHON_MODULE(M)
>  {
>    def("addem", &addem, (arg("x"), arg("y"), arg("z")));
>  }
>
>  >>> from M import addem
>  >>> addem(1, 8, 2, x=4)
>  Traceback (most recent call last):
>  ...
>  ArgumentError: Python argument types in
>      M.addem(int, int, int)
>  did not match C++ signature:
>      addem(int x, int y, int z)
>
> That error message is very confusing...  f(int,int,int) doesn't match
> f(int,int,int)?  The pure python version reports something more
> sensible:
>
>  >>> def g(x,y,z, **kwargs):
>  ...     print x,y,z,kwargs
>  ...
>  >>> g(1,2,3,x=4)
>  Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>  TypeError: g() got multiple values for keyword argument 'x'
>
>
> 2.) An implemented solution
>
> I've got something implemented.  Here's what it does.
>
> 2.a) Solution: multiple values for keyword
>
> The easiest case to catch is the last [1.c].  It is also orthogonal to
> the others.  If a registered function has keyword arguments, check for
> multiple keyword values, and raise an error if so:
>
>  >>> from M import addem
>  >>> addem(1,2,3,x=1)
>  Boost.Python.TypeError: M.addem() got multiple values for keyword argument
> 'x'
>
> 2.b) Solution: "necessarily" ambiguous overloads
>
> The next easiest thing to catch is case [1.b], "necessarily" ambiguous
> registrations.  It proceeds as follows: at import time, as each new
> overload V for function F is registered, compare V to the existing
> overloads EO for F.  If V has the same signature as something in EO,
> raise an AmbiguousOverload exception.  For instance, if you load the
> module from [1.b] above, you get:
>
>  >>> import m
>  Traceback (most recent call last):
>  ...
>  AmbiguousOverload: Boost.Python function m.f
>  has ambiguous overloads.  C++ signatures
>    f(float) -> None
>  and
>    f(double) -> None
>  are indistinguishable to python.
>
> Again this is because c++ float and c++ double both map to python
> 'float'.   This one is "easy" as it happens only once (not at every
> function call) and doesn't take up any space.
>
> 2.c) Solution: ambiguous calls
>
> The hard case is [1.a]:
>
>  void f_bool(bool);
>  void f_int(int);
>
>  BOOST_PYTHON_MODULE(m)
>  {
>     def(f, f_bool);
>     def(f, f_int);
>  }
>
> For module 'm' above, a call to f(True) or f(1) should succeed and call
> the corresponding function.  Passing a float, however, is ambiguous:
>
>  >>> f(True)  # ok
>  >>> f(1)     # ok
>  >>> f(1.0)
>  Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>  Boost.Python.AmbiguousCall: Ambiguous call to Boost.Python function m.f
>  C++ signatures:
>      f(int)
>      f(bool)
>
> So the implementation has some how 'scored' each possible overload and
> in each case either used the best one or reported ambiguity if multiple
> overloads are tied for 'best'.
>
> The scoring works as follows.
>
> In the guts of boost.python we recieve a tuple of arguments of type
> PyObject*.  We also have a list of overloads, each with an mpl vector
> representing the signature of the associated C++ function.  This is
> unchanged from the released version.
>
> What has been done until now is to attempt to call each overload in
> order and return the result of the first that succeeds.
>
> In this new implmentation, for each c++ argument of type T_n, we ask a
> class overload_score<T_n> to score the conversion of the type
> (PyTypeObject*) of this PyObject* to T_n.  Example:
>
>   template <>
>   struct overload_score<int>
>   {
>     boost::optional<unsigned> operator()(PyObject* type)
>     {
>       if(PyInt_CheckExact(type))
>           return boost::optional<unsigned>(0); // int == perfect
>       else if (PyBool_Check(type))
>           return boost::optional<unsigned>(1); // bool == okay
>       else if (PyFloat_CheckExact(type))
>         return boost::optional<unsigned>(1);   // float == okay
>       else if(arg_from_python<int>(type).convertible())
>         return boost::optional<unsigned>(1);   // fallback
>       else
>         return boost::optional<unsigned>();    // unsuitable
>     }
>   };
>
> The "score" type is optional<unsigned>.  A valueless,
> (default-constructed) optional<unsigned> means 'unsuitable'.
> optional<unsigned>(0) is a perfect match, and optional<unsigned>(value)
> for value > 0 is a workable but less than perfect match.
>
> These per-argument scores are added together for all arguments: this is
> the overload's total score.  If any argument is 'unsuitable', the total
> score is 'unsuitable'.
>
> If there is a tie for the best (lowest) score, the call is ambiguous and
> a Boost.Python.AmbiguousCall exception is raised.  If there is only one
> function in first place, call it.
>
> 3.)  Implications
>
> This breaks a few corner cases that I've found in the tests.
>
> 3.a) implied init<> registration
>
> I have found a few instances like this one:
>
>  struct X
>  {
>    X();
>    X(int);
>  };
>
>  class_<X>("X")       // #1
>    .def(init<>())     // #2
>    .def(init<int>())
>    ;
>
> Here, #1 causes a default constructor to be registered, as does #2.
> This will cause a throw at load time as in [2.b].  It is simple to fix.
>
> 3.b)  raw_constructor
>
> The test suite for raw_constructor depends on our first_match overload
> resolution.  The fix is to move a few things around to get the same
> effect without relying on this behavior.
>
> The "before" code looks like this:
>
>  class Foo
>  {
>   public:
>      Foo(tuple args, dict kw)
>        : args(args), kw(kw) {}
>
>      tuple args;
>      dict kw;
>  };
>
>  object init_foo(tuple args, dict kw)
>  {
>      tuple rest(args.slice(1,_));
>      return args[0].attr("__init__")(rest, kw);
>  }
>
>  BOOST_PYTHON_MODULE(raw_ctor_ext)
>  {
>      // using no_init postpones defining __init__ function until after
>      // raw_function for proper overload resolution order, since later
>      // defs get higher priority.
>      class_<Foo>("Foo", no_init)
>          .def("__init__", raw_function(&init_foo))
>          .def(init<tuple, dict>())
>          .def_readwrite("args", &Foo::args)
>          .def_readwrite("kw", &Foo::kw)
>          ;
>  }
>
> The "after" code does the registration piecemeal: Maybe there is a
> better way to do this.  To my mind this is a little better because it
> makes explicit what is happening rather than relying on some subtle
> property of overload resolution:
>
>  class Foo
>  {
>   public:
>    Foo(tuple args, dict kw)
>      : args(args), kw(kw) {}
>
>      tuple args;
>      dict kw;
>  };
>
>  // our custom factory function
>  object init_foo(tuple args, dict kw)
>  {
>      tuple rest(args.slice(1,_));
>      return args[0].attr("__real_init__")(rest, kw);
>  }
>
>  BOOST_PYTHON_MODULE(raw_ctor_ext)
>  {
>    // to get the desired effect we register
>    // the actual constructor and our 'raw' constructor,
>    // and then rename them
>    class_<Foo> c("Foo", init<tuple, dict>());
>    c
>      .def("__tmp_init__", raw_function(&init_foo))
>      .def_readwrite("args", &Foo::args)
>      .def_readwrite("kw", &Foo::kw)
>      ;
>
>    //
>    //  __init__ => __real_init__
>    //  __tmp_init__ => __init__
>    //
>    object real_constructor = getattr(c, "__init__");
>    object raw_constructor = getattr(c, "__tmp_init__");
>
>    setattr(c, "__init__", raw_constructor);
>    delattr(c, "__tmp_init__");
>    setattr(c, "__real_init__", real_constructor);
>
>  }
>
> And that basically covers it.  Looking forward to comments/feedback.
>
> The code is here:
>
>  http://gitorious.org/~straszheim/boost/straszheims-python/trees/master
>
> Note that the headers are in subdirectory 'include'...  If there is
> enough response to this mail it I'll send out instructions on how to get
> a working build going.
>
> - t
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091218/eec9f158/attachment-0001.htm>

From charlessolar at gmail.com  Sat Dec 19 08:11:59 2009
From: charlessolar at gmail.com (Charles Solar)
Date: Sat, 19 Dec 2009 01:11:59 -0600
Subject: [C++-sig] PyThreadState_Get: no current thread
Message-ID: <708326ee0912182311n5cc725f0y3c6ff055643fa67c@mail.gmail.com>

Its getting kind of late and my eyes are getting tired so I wanted to throw
up this problem I am having and see if I can get some feedback on what might
be going wrong here.

I built boost::python into my app and exported a bunch of stuff.  All is
well when I just call functions and do not much at all (the example apps all
work for example).
However, when I start actually trying to use the features of my library I
crash python.

In debug mode I get a ton of "PyThreadState_Get: no current thread" messages
printed before it crashes from what I assume is a stack overflow.  In
release mode it just crashes.

Both crash while executing

Py_ALLOW_RECURSION
        t = PyDict_GetItem(interned, (PyObject *)s);
Py_END_ALLOW_RECURSION

in unicodeobject.c line 9725

The stack trace once the program enters the python dll looks like this
(going bottom to up)
     python31_d.dll!Py_FatalError(const char * msg=0x000007fef0c350f8)  Line
2013 + 0x5 bytes    C
     python31_d.dll!PyThreadState_Get()  Line 351    C
     python31_d.dll!PyErr_Occurred()  Line 133 + 0x5 bytes    C
     python31_d.dll!Py_FatalError(const char * msg=0x000007fef0c350f8)  Line
2013 + 0x5 bytes    C
     python31_d.dll!PyThreadState_Get()  Line 351    C
     python31_d.dll!PyUnicode_InternInPlace(_object * *
p=0x000000000330ef18)  Line 9725 + 0x5 bytes    C
     python31_d.dll!PyUnicode_InternFromString(const char *
cp=0x000007fef113ef28)  Line 9766    C
     python31_d.dll!PyObject_GetAttrString(_object * v=0x000000000245f658,
const char * name=0x000007fef113ef28)  Line 751 + 0xa bytes    C

i have a suspicion about what may be causing this error but I am afraid I do
not know enough about python to know for sure, so here where you guys come
in :)

My program ( that python is controlling ) is multithreaded.
I am thinking that one of the other program threads is calling a python
method and that is what is causing this crash.

I am using the observer pattern in my design and I have a python object that
I attach to an object to watch for changes.  When the object is changed, my
python code should get executed.  The notify method to tell all the
observers there was a change could get called from a separate thread.
I am pretty sure I have wrapped the classes and definitions correctly to
allow this to happen thanks to the boost::python doc.  So my question is,
would calling python code from a thread other than the main thread cause
this problem?  And if so, how can I fix this?

Thank you for your time
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091219/14cdc7d7/attachment.htm>

From troy at resophonic.com  Sun Dec 20 00:59:13 2009
From: troy at resophonic.com (Troy D. Straszheim)
Date: Sat, 19 Dec 2009 18:59:13 -0500
Subject: [C++-sig] Implementation of proper overload resolution
In-Reply-To: <hgfqv5$le5$1@ger.gmane.org> (Neal Becker's message of "Fri, 18
	Dec 2009 07:01:42 -0500")
References: <87aaxha8ih.fsf@resophonic.com> <hgeaef$noo$1@ger.gmane.org>
	<874onptdml.fsf@resophonic.com> <hgfqv5$le5$1@ger.gmane.org>
Message-ID: <87k4wiqzvy.fsf@resophonic.com>

Neal Becker <ndbecker2 at gmail.com> writes:

> I am concerned that this doesn't introduce too much overhead for the common 
> case where there is no ambiguity.  I suppose this has been optimized?
>

For the following module:

  int f(int x, int y, int z) { return x*100 + y*10 + z; }
  BOOST_PYTHON_MODULE(m) { def("f", &f); } 

and the following script:

  from m import f
  for i in range(10000000):
      f(i, i, i)

With all optimizations turned on, Boost.Python from the 1.41.0 release
runs this in (average over several runs) 4.25 seconds, and this new
implementation in 4.38 seconds, so an increase of about 3% wall-clock
time.  This test case is designed to make the performance hit look as
bad as possible: if int f(...) did anything substantial, of course the
relative slowdown would be less.

In an attempt to get an idea of exactly how many additional CPU ticks
are involved I ran a script that just does

   from m import f

under 'valgrind --tool=lackey' and compared the number of 'guest
instructions', some measure of how much work the virtual CPU is doing:

  1.41.0:  guest instrs:  26,559,194

  New:     guest instrs:  26,864,330

So there's an additional total 305k instructions required to create and
destroy the module. If you then add a single call to f() to the script:

   from m import f
   f(1,1,1)

you get:

  1.41.0: guest instrs:  26,593,334

  New:    guest instrs:  26,899,095

So 1.41.0 requires 34140 of these cpu 'ticks' to call f(), and the new
version requires 34765 of them, or 625 instructions ~= 2% more.

Note that this implementation is also 'fusionized', i.e. where function
calling is concerned, much of the boost.preprocessor guts have been
removed and replaced with boost fusion, so it is a little hard to track
where this 2% is actually going.  OTOH polymorphic function objects and
phoenix expressions are passable to def().

The fusionization also involves some increase in library size.
Presumably the difference would be less if inlining were turned off.

Test Module                             1.41.0    New
------------------------------------    ------    ----
andreas_beyer_ext.so                    120K      120K
args_ext.so                             208K      264K
auto_ptr_ext.so                         132K      156K
back_reference_ext.so                   148K      176K
ben_scott1_ext.so                       112K      132K
bienstman1_ext.so                       72K       72K
bienstman2_ext.so                       80K       96K
bienstman3_ext.so                       44K       60K
builtin_converters_ext.so               428K      544K 
callbacks_ext.so                        200K      240K
const_argument_ext.so                   28K       28K
crossmod_exception_a.so                 24K       28K
crossmod_exception_b.so                 24K       28K
crossmod_opaque_a.so                    28K       36K
crossmod_opaque_b.so                    28K       36K
data_members_ext.so                     320K      372K
defaults_ext.so                         304K      392K
dict_ext.so                             80K       96K
docstring_ext.so                        104K      116K
enum_ext.so                             84K       92K
exception_translator_ext.so             28K       36K
extract_ext.so                          192K      220K
implicit_ext.so                         100K      116K
injected_ext.so                         100K      124K
input_iterator.so                       124K      140K
iterator_ext.so                         324K      364K
list_ext.so                             176K      204K
long_ext.so                             100K      116K
m1.so                                   292K      352K
m2.so                                   128K      156K
map_indexing_suite_ext.so               736K      876K
minimal_ext.so                          20K       20K
multi_arg_constructor_ext.so            48K       60K
nested_ext.so                           112K      124K
object_ext.so                           332K      392K
opaque_ext.so                           80K       96K
operators_ext.so                        228K      268K
pickle1_ext.so                          92K       104K
pickle2_ext.so                          104K      124K
pickle3_ext.so                          116K      136K
pickle4_ext.so                          60K       68K
pointer_vector_ext.so                   220K      252K
polymorphism2_auto_ptr_ext.so           200K      268K
polymorphism_ext.so                     216K      264K
properties_ext.so                       164K      188K
raw_ctor_ext.so                         104K      120K
return_arg_ext.so                       112K      132K
shared_ptr_ext.so                       432K      672K
slice_ext.so                            96K       112K
staticmethod_ext.so                     76K       96K
stl_iterator_ext.so                     116K      128K
str_ext.so                              68K       76K
tuple_ext.so                            68K       80K
vector_indexing_suite_ext.so            612K      720K
virtual_functions_ext.so                176K      216K
voidptr_ext.so                          48K       60K
wrapper_held_type_ext.so                92K       112K

-t






From ndbecker2 at gmail.com  Sun Dec 20 13:46:31 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Sun, 20 Dec 2009 07:46:31 -0500
Subject: [C++-sig] Implementation of proper overload resolution
References: <87aaxha8ih.fsf@resophonic.com>
Message-ID: <hgl6b8$622$1@ger.gmane.org>

When it's ready, I'd like to try it on my code collection (see that at least 
nothing breaks).


From ndbecker2 at gmail.com  Sun Dec 20 16:39:00 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Sun, 20 Dec 2009 10:39 -0500
Subject: [C++-sig] interrupting long running c++ code
Message-ID: <hglgek$qu$1@ger.gmane.org>

I don't suppose there's any way to allow interrupt signal through while a 
long-running boost::python wrapped function is running?  It seems that 
python blocks the signal until the function completes.

This is a single-threaded environment. 


From hans_meine at gmx.net  Mon Dec 21 09:45:10 2009
From: hans_meine at gmx.net (Hans Meine)
Date: Mon, 21 Dec 2009 09:45:10 +0100
Subject: [C++-sig] what do I need to correctly port boost python
	extensions
Message-ID: <200912210945.11013.hans_meine@gmx.net>

Hi

Am Montag, 14. Dezember 2009 23:09:31 schrieb Nikolaus Rath:
> Alexey Akimov <alexey.akimov85 at gmail.com> writes:
> > However I am still wondering if there is a way to avoid rebuilding the
> > extension when one travel from machine to machine.
>
> The only way is to have exactly the same runtime environment (i.e.,
> having installed the same distributions with the same version on the
> same architecture). Otherwise you have to recompile.

I think you're a bit too pessimistic.  While I agree that the 
"normal"/standard way on Linux(*) is to recompile for a specific distribution 
/ machine type, there is the LSB standard which makes certain distributions 
"similar enough" so that your binary might work.

Read: You could try an existing binary before recompiling, when the machines 
are not too different.

Ciao, /  /
     /--/
    /  / ANS

*: OK, there is proprietary, closed-source software which is usualy 
distributed together with most dependencies, either statically linked or with 
a bit .so collection and some shell script / LD_LIBRARY_PATH startup magic.


From hans_meine at gmx.net  Mon Dec 21 09:45:33 2009
From: hans_meine at gmx.net (Hans Meine)
Date: Mon, 21 Dec 2009 09:45:33 +0100
Subject: [C++-sig] Implementation of proper overload resolution
Message-ID: <200912210945.33714.hans_meine@gmx.net>

Am Freitag, 18. Dezember 2009 16:08:24 schrieb Gustavo Carneiro:
> Don't you think that when these overloading problems become an issue it is
> a sign of a poorly designed API?  I mean, if overloaded functions
> parameters are not completely different in type or number, then maybe they
> are already too confusing to use and should not be overloaded in the first
> place?...

No, in C++ overloading rules are (quite) well-specified and everybody's used 
to them.  In python, overloading is absent.  Now python bindings of C++ 
libraries are supposed to mimick C++'s overloading, and AFAICS Troy did great 
work in order to let the BPL be better at this, i.e. more closely follow the 
rules of C++ w.r.t. overload resolution.  (Previously, BPL's first[/last]-
come-first-serve rule would not match people's expectations in many cases.)

Ciao, /  /
     /--/
    /  / ANS


From troy at resophonic.com  Mon Dec 21 18:17:48 2009
From: troy at resophonic.com (Troy D. Straszheim)
Date: Mon, 21 Dec 2009 12:17:48 -0500
Subject: [C++-sig] Implementation of proper overload resolution
In-Reply-To: <a467ca4f0912180708h7dd7b2e3of1dce8e1f5da7de0@mail.gmail.com>
	(Gustavo Carneiro's message of "Fri, 18 Dec 2009 15:08:24 +0000")
References: <87aaxha8ih.fsf@resophonic.com>
	<a467ca4f0912180708h7dd7b2e3of1dce8e1f5da7de0@mail.gmail.com>
Message-ID: <87pr68s0ub.fsf@resophonic.com>

Gustavo Carneiro <gjcarneiro at gmail.com> writes:

> Don't you think that when these overloading problems become an issue it is a
> sign of a poorly designed API? 

In this case the intention is to fix bugs in boost.python.  The broken
example we've been working with is where a parameter is bool in one
overload and int in another.  Does this indicate a broken API?  That's
above my pay grade...  I'm willing to say "probably not in all cases".

> ?I mean, if overloaded functions parameters are not completely
> different in type or number, then maybe they are already too confusing
> to use and should not be overloaded in the first place?... ?

Maybe yes, though to my mind the real issue is an aesthetic one: should
one at all attempt to emulate overloading in a language that doesn't
have it natively?  A similar discussion ensues w.r.t. "const".  I wasn't
around when the feature was added, which was ~7 years ago; to remove
overloading now would break lots of code, and that decides it.  So I'd
phrase it like this: let's fix what's there so long as the cost in
runtime and complexity is sufficiently small.

-t

From gjcarneiro at gmail.com  Mon Dec 21 23:53:45 2009
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Mon, 21 Dec 2009 22:53:45 +0000
Subject: [C++-sig] ANN: PyBindGen 0.13 released
Message-ID: <a467ca4f0912211453v645ac259uc5aecb1bed3ce050@mail.gmail.com>

PyBindGen is a Python module that is geared to generating C/C++ code that
binds a C/C++ library for Python. It does so without extensive use of either
C++ templates or C pre-processor macros. It has modular handling of C/C++
types, and can be easily extended with Python plugins. The generated code is
almost as clean as what a human programmer would write.

It can be downloaded from:

        http://code.google.com/p/pybindgen/

Bug reports should be filed here:

        https://bugs.launchpad.net/ <https://bugs.launchpad.net/pybindgen>
pybindgen <https://bugs.launchpad.net/pybindgen>

<https://bugs.launchpad.net/pybindgen>Documentation:

        http://packages.python.org/PyBindGen/

NEWS:

    - Lots of small bug fixes

    - Custodian/ward-style memory management works better now

    - Add 'reference_existing_object' and 'return_interal_reference'
      options for pointer/reference return values

    - New Sphinx based documentation
-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091221/2c19e7e9/attachment.htm>

From troy at resophonic.com  Tue Dec 22 04:42:57 2009
From: troy at resophonic.com (Troy D. Straszheim)
Date: Mon, 21 Dec 2009 22:42:57 -0500
Subject: [C++-sig] Implementation of proper overload resolution
In-Reply-To: <9D77FF39-BA7A-476F-9CBF-48009A1D7919@hailmail.net> (Dane
	Springmeyer's message of "Thu, 17 Dec 2009 11:31:02 -0800")
References: <87aaxha8ih.fsf@resophonic.com>
	<0EFC2868-BA76-4E73-8DF2-075C2D73ABD9@hailmail.net>
	<9D77FF39-BA7A-476F-9CBF-48009A1D7919@hailmail.net>
Message-ID: <874onjd67y.fsf@resophonic.com>

Dane Springmeyer <blake at hailmail.net> writes:

> Troy,
>
> Incidentally, do you know the proper way  (or is there a proper way?)
> to support None type keyword arguments?
>
> Cheers,
>
> Dane
>

Could you elaborate?  What're you trying to do?  

-t



From Nikolaus at rath.org  Tue Dec 22 19:28:00 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Tue, 22 Dec 2009 13:28:00 -0500
Subject: [C++-sig] [Py++] Include additional code with ctypes code
	generator
References: <87r5qxu02v.fsf@vostro.rath.org>
	<7465b6170912141133w46fdd115mb56f0845a3c3ef3e@mail.gmail.com>
Message-ID: <87vdfyg8y7.fsf@inspiron.ap.columbia.edu>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
> On Mon, Dec 14, 2009 at 4:00 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>> Hello,
>>
>> Is there a way to add additional code to the output generated by the
>> ctypes code generator? Specifically, I would like to add some special
>> comments to prevent the autogenerated files being checked by PyLink and
>> PyDev.
>>
>> http://language-binding.net/pyplusplus/documentation/inserting_code.html
>> talks about a add_registration_code method, but it doesn't seem to exist
>> for the ctypes generator:
>>
>> mb = ctypes_module_builder_t( [header_file], symbols_file, gccxml_config=gccxml_cfg )
>> mb.build_code_creator(shared_library_file)
>> mb.add_registration_code( '#@PyDevCodeAnalysisIgnore', False )
>>
>> yields
>>
>> AttributeError: 'ctypes_module_builder_t' object has no attribute
>> 'add_registration_code'
>>
>
> Right now - there is no such way. I will try to add this functionality
> today or tomorrow.

Have you had the time to do this yet? If not, I'll just manually add the
required lines by rewriting the file after it has been generated. Just
let me know.

Best,


   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From roman.yakovenko at gmail.com  Tue Dec 22 20:53:52 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Tue, 22 Dec 2009 21:53:52 +0200
Subject: [C++-sig] [Py++] Include additional code with ctypes code
	generator
In-Reply-To: <87vdfyg8y7.fsf@inspiron.ap.columbia.edu>
References: <87r5qxu02v.fsf@vostro.rath.org>
	<7465b6170912141133w46fdd115mb56f0845a3c3ef3e@mail.gmail.com>
	<87vdfyg8y7.fsf@inspiron.ap.columbia.edu>
Message-ID: <7465b6170912221153s4dd065f5mff1ecbd6d3cf1d9b@mail.gmail.com>

On Tue, Dec 22, 2009 at 8:28 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> Have you had the time to do this yet? If not, I'll just manually add the
> required lines by rewriting the file after it has been generated. Just
> let me know.

Sorry, I forgot about this :-(

I suggest you to rewrite the file and I will let you you when I
implement the feature.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From anitaa1981 at gmail.com  Wed Dec 23 04:20:31 2009
From: anitaa1981 at gmail.com (Anita A)
Date: Wed, 23 Dec 2009 08:50:31 +0530
Subject: [C++-sig] Trying to implement subscriptable type in c++.
Message-ID: <3c0065cc0912221920i35a01c8dm7371e8bcdb4190ec@mail.gmail.com>

Hi all,

I am trying to implement a subscriptable type in c++ (equivalent to python
dict), but I get a 'type is unsubscriptable' error. Here is a minimal
code sample I tried:

--------------------------------------------------------------------------
#include <Python.h>
#include <iostream>
#include "pyerror.hpp"

/////////////////////////////////////////////////////////
PyObject* getitem(PyObject* self, PyObject* args)
{
    std::cout << "getitem" << std::endl;
    return PyString_FromString("this is an item");
}

PyMethodDef method[] = {
     {"__getitem__", &getitem, METH_VARARGS, ""},
     {0, 0, 0, 0}
};

static PyMethodDef module_methods[] = {
     { 0, 0, 0, 0 }
};

struct MyObject
{
    PyObject_HEAD
};

extern const char* code;

int main(int argc, char *argv[])
{
    Py_InitializeEx(0);

    PyObject* myModule = Py_InitModule3("test", module_methods, "test module");
    assert(myModule);

    PyTypeObject myType;
    memset(&myType, 0, sizeof(PyTypeObject));

    myType.tp_name = "myType";
    myType.tp_basicsize = sizeof(MyObject);
    myType.tp_methods = method;
    if (PyType_Ready(&myType) < 0)
    {
        std::cout << "type error" << std::endl;
        return 1;
    }

    PyModule_AddObject(myModule, myType.tp_name, (PyObject*)&myType);

    PyObject* m = PyImport_AddModule("__main__");
    PyObject* d = PyModule_GetDict(m);
    MyObject* odef = PyObject_New(MyObject, &myType);
    PyDict_SetItemString(d, "obj", (PyObject*)odef);

    PyObject* rv = PyRun_String(code, Py_file_input, d, d);
    if(rv == 0)
    {
        std::cout << "run error" << std::endl;
        printException();
        return 1;
    }

    Py_Finalize();
    std::cout << "Done.\n";

    return 0;
}

const char* code =
"print dir(obj)\n"
"x = obj.__getitem__(1)\n"
"print 'direct call:', x\n"
"x = obj[1]\n"
;

--------------------------------------------------------------------------
Compile and run session.
--------------------------------------------------------------------------
$ g++ -I/usr/include/python2.6 pymin.cpp -lpython2.6
$ ./a.out
['__class__', '__delattr__', '__doc__', '__format__',
'__getattribute__', '__getitem__', '__hash__', '__init__', '__new__',
'__reduce__', '__reduce_ex__', '__repr__', '__setattr__',
'__sizeof__', '__str__', '__subclasshook__']
getitem
direct call: this is an item
run error
Error in execution:
Traceback (most recent call last):
line 4, in ??
<type 'exceptions.TypeError'>: 'myType' object is unsubscriptable
--------------------------------------------------------------------------

As you can see, when I call __getitem__() directly, it works, but when I
call it as a subscriptable object, I get an error. What am I missing?

A simple python equivalent class works as expected:
--------------------------------------------------------------------------
class Testi:
    def __getitem__(self, idx):
        return 10

if __name__ == '__main__':
    x = Testi()
    print dir(x)
    print x['a']
--------------------------------------------------------------------------
Run session.
--------------------------------------------------------------------------
$ python tst.py
['__doc__', '__getitem__', '__module__']
10
--------------------------------------------------------------------------

I had tried using the PyMappingMethods structure as documented, but there
is apparently no way to implement the __contains__() method, that is, to
check if a key is present in the dict.

Any help would be appreciated.

From gjcarneiro at gmail.com  Wed Dec 23 11:45:49 2009
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Wed, 23 Dec 2009 10:45:49 +0000
Subject: [C++-sig] Trying to implement subscriptable type in c++.
In-Reply-To: <3c0065cc0912221920i35a01c8dm7371e8bcdb4190ec@mail.gmail.com>
References: <3c0065cc0912221920i35a01c8dm7371e8bcdb4190ec@mail.gmail.com>
Message-ID: <a467ca4f0912230245i510f6376jb89371e685db83e@mail.gmail.com>

In Python C API you cannot define a __getitem__ method; instead define a
tp_as_mapping slot.  See:

http://docs.python.org/c-api/typeobj.html#tp_as_mapping


2009/12/23 Anita A <anitaa1981 at gmail.com>

> Hi all,
>
> I am trying to implement a subscriptable type in c++ (equivalent to python
> dict), but I get a 'type is unsubscriptable' error. Here is a minimal
> code sample I tried:
>
> --------------------------------------------------------------------------
> #include <Python.h>
> #include <iostream>
> #include "pyerror.hpp"
>
> /////////////////////////////////////////////////////////
> PyObject* getitem(PyObject* self, PyObject* args)
> {
>    std::cout << "getitem" << std::endl;
>    return PyString_FromString("this is an item");
> }
>
> PyMethodDef method[] = {
>     {"__getitem__", &getitem, METH_VARARGS, ""},
>     {0, 0, 0, 0}
> };
>
> static PyMethodDef module_methods[] = {
>     { 0, 0, 0, 0 }
> };
>
> struct MyObject
> {
>    PyObject_HEAD
> };
>
> extern const char* code;
>
> int main(int argc, char *argv[])
> {
>    Py_InitializeEx(0);
>
>    PyObject* myModule = Py_InitModule3("test", module_methods, "test
> module");
>    assert(myModule);
>
>    PyTypeObject myType;
>    memset(&myType, 0, sizeof(PyTypeObject));
>
>    myType.tp_name = "myType";
>    myType.tp_basicsize = sizeof(MyObject);
>    myType.tp_methods = method;
>    if (PyType_Ready(&myType) < 0)
>    {
>        std::cout << "type error" << std::endl;
>        return 1;
>    }
>
>    PyModule_AddObject(myModule, myType.tp_name, (PyObject*)&myType);
>
>    PyObject* m = PyImport_AddModule("__main__");
>    PyObject* d = PyModule_GetDict(m);
>    MyObject* odef = PyObject_New(MyObject, &myType);
>    PyDict_SetItemString(d, "obj", (PyObject*)odef);
>
>    PyObject* rv = PyRun_String(code, Py_file_input, d, d);
>    if(rv == 0)
>    {
>        std::cout << "run error" << std::endl;
>        printException();
>        return 1;
>    }
>
>    Py_Finalize();
>    std::cout << "Done.\n";
>
>    return 0;
> }
>
> const char* code =
> "print dir(obj)\n"
> "x = obj.__getitem__(1)\n"
> "print 'direct call:', x\n"
> "x = obj[1]\n"
> ;
>
> --------------------------------------------------------------------------
> Compile and run session.
> --------------------------------------------------------------------------
> $ g++ -I/usr/include/python2.6 pymin.cpp -lpython2.6
> $ ./a.out
> ['__class__', '__delattr__', '__doc__', '__format__',
> '__getattribute__', '__getitem__', '__hash__', '__init__', '__new__',
> '__reduce__', '__reduce_ex__', '__repr__', '__setattr__',
> '__sizeof__', '__str__', '__subclasshook__']
> getitem
> direct call: this is an item
> run error
> Error in execution:
> Traceback (most recent call last):
> line 4, in ??
> <type 'exceptions.TypeError'>: 'myType' object is unsubscriptable
> --------------------------------------------------------------------------
>
> As you can see, when I call __getitem__() directly, it works, but when I
> call it as a subscriptable object, I get an error. What am I missing?
>
> A simple python equivalent class works as expected:
> --------------------------------------------------------------------------
> class Testi:
>    def __getitem__(self, idx):
>        return 10
>
> if __name__ == '__main__':
>    x = Testi()
>    print dir(x)
>    print x['a']
> --------------------------------------------------------------------------
> Run session.
> --------------------------------------------------------------------------
> $ python tst.py
> ['__doc__', '__getitem__', '__module__']
> 10
> --------------------------------------------------------------------------
>
> I had tried using the PyMappingMethods structure as documented, but there
> is apparently no way to implement the __contains__() method, that is, to
> check if a key is present in the dict.
>
> Any help would be appreciated.
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091223/1bfabeb1/attachment-0001.htm>

From micdestefano at gmail.com  Wed Dec 23 12:19:50 2009
From: micdestefano at gmail.com (Michele De Stefano)
Date: Wed, 23 Dec 2009 12:19:50 +0100
Subject: [C++-sig] Trying to implement subscriptable type in c++.
In-Reply-To: <3c0065cc0912221920i35a01c8dm7371e8bcdb4190ec@mail.gmail.com>
References: <3c0065cc0912221920i35a01c8dm7371e8bcdb4190ec@mail.gmail.com>
Message-ID: <c410adc90912230319q27eaa24oe482f47ba56f32f6@mail.gmail.com>

I'm not sure if this can help.
http://code.google.com/p/mds-utils
If you try to use Boost Python and the
mds_utils::python::support_random_access< Derived >
class, contained into container_operators.hpp of the mds-utils library
(http://code.google.com/p/mds-utils), may be you can easily define a
subscriptable container (look at the example program attached to the
library documentation).

Another approach I think is to use the Boost Python's indexing suite
(http://www.boost.org/doc/libs/1_41_0/libs/python/doc/v2/indexing.html).

Best regards.
Michele

2009/12/23 Anita A <anitaa1981 at gmail.com>:
> Hi all,
>
> I am trying to implement a subscriptable type in c++ (equivalent to python
> dict), but I get a 'type is unsubscriptable' error. Here is a minimal
> code sample I tried:
>
> --------------------------------------------------------------------------
> #include <Python.h>
> #include <iostream>
> #include "pyerror.hpp"
>
> /////////////////////////////////////////////////////////
> PyObject* getitem(PyObject* self, PyObject* args)
> {
> ? ?std::cout << "getitem" << std::endl;
> ? ?return PyString_FromString("this is an item");
> }
>
> PyMethodDef method[] = {
> ? ? {"__getitem__", &getitem, METH_VARARGS, ""},
> ? ? {0, 0, 0, 0}
> };
>
> static PyMethodDef module_methods[] = {
> ? ? { 0, 0, 0, 0 }
> };
>
> struct MyObject
> {
> ? ?PyObject_HEAD
> };
>
> extern const char* code;
>
> int main(int argc, char *argv[])
> {
> ? ?Py_InitializeEx(0);
>
> ? ?PyObject* myModule = Py_InitModule3("test", module_methods, "test module");
> ? ?assert(myModule);
>
> ? ?PyTypeObject myType;
> ? ?memset(&myType, 0, sizeof(PyTypeObject));
>
> ? ?myType.tp_name = "myType";
> ? ?myType.tp_basicsize = sizeof(MyObject);
> ? ?myType.tp_methods = method;
> ? ?if (PyType_Ready(&myType) < 0)
> ? ?{
> ? ? ? ?std::cout << "type error" << std::endl;
> ? ? ? ?return 1;
> ? ?}
>
> ? ?PyModule_AddObject(myModule, myType.tp_name, (PyObject*)&myType);
>
> ? ?PyObject* m = PyImport_AddModule("__main__");
> ? ?PyObject* d = PyModule_GetDict(m);
> ? ?MyObject* odef = PyObject_New(MyObject, &myType);
> ? ?PyDict_SetItemString(d, "obj", (PyObject*)odef);
>
> ? ?PyObject* rv = PyRun_String(code, Py_file_input, d, d);
> ? ?if(rv == 0)
> ? ?{
> ? ? ? ?std::cout << "run error" << std::endl;
> ? ? ? ?printException();
> ? ? ? ?return 1;
> ? ?}
>
> ? ?Py_Finalize();
> ? ?std::cout << "Done.\n";
>
> ? ?return 0;
> }
>
> const char* code =
> "print dir(obj)\n"
> "x = obj.__getitem__(1)\n"
> "print 'direct call:', x\n"
> "x = obj[1]\n"
> ;
>
> --------------------------------------------------------------------------
> Compile and run session.
> --------------------------------------------------------------------------
> $ g++ -I/usr/include/python2.6 pymin.cpp -lpython2.6
> $ ./a.out
> ['__class__', '__delattr__', '__doc__', '__format__',
> '__getattribute__', '__getitem__', '__hash__', '__init__', '__new__',
> '__reduce__', '__reduce_ex__', '__repr__', '__setattr__',
> '__sizeof__', '__str__', '__subclasshook__']
> getitem
> direct call: this is an item
> run error
> Error in execution:
> Traceback (most recent call last):
> line 4, in ??
> <type 'exceptions.TypeError'>: 'myType' object is unsubscriptable
> --------------------------------------------------------------------------
>
> As you can see, when I call __getitem__() directly, it works, but when I
> call it as a subscriptable object, I get an error. What am I missing?
>
> A simple python equivalent class works as expected:
> --------------------------------------------------------------------------
> class Testi:
> ? ?def __getitem__(self, idx):
> ? ? ? ?return 10
>
> if __name__ == '__main__':
> ? ?x = Testi()
> ? ?print dir(x)
> ? ?print x['a']
> --------------------------------------------------------------------------
> Run session.
> --------------------------------------------------------------------------
> $ python tst.py
> ['__doc__', '__getitem__', '__module__']
> 10
> --------------------------------------------------------------------------
>
> I had tried using the PyMappingMethods structure as documented, but there
> is apparently no way to implement the __contains__() method, that is, to
> check if a key is present in the dict.
>
> Any help would be appreciated.
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Michele De Stefano
http://www.linkedin.com/in/micdestefano
http://code.google.com/p/mds-utils
http://xoomer.virgilio.it/michele_de_stefano

From Nikolaus at rath.org  Wed Dec 23 20:21:19 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Wed, 23 Dec 2009 14:21:19 -0500
Subject: [C++-sig] [Py++] Don't transform char* to c_char_p
Message-ID: <87zl59qyxc.fsf@vostro.rath.org>

Hi Roman,

I know you are busy, so I have already monkeypatched pyplusplus locally
for my purposes. But maybe you are interested to add this feature add
some point in the future:

It would be great if there was a way to tell Py++ that it should not
represent the C char * type to the ctypes c_char_p, because the later is
supposed to be used only for \0 terminated strings. A general char*
should be translated to POINTER(c_char).


Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From roman.yakovenko at gmail.com  Wed Dec 23 22:12:39 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Wed, 23 Dec 2009 23:12:39 +0200
Subject: [C++-sig] [Py++] Don't transform char* to c_char_p
In-Reply-To: <87zl59qyxc.fsf@vostro.rath.org>
References: <87zl59qyxc.fsf@vostro.rath.org>
Message-ID: <7465b6170912231312l2505e519j366f1cc1a4f79605@mail.gmail.com>

On Wed, Dec 23, 2009 at 9:21 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> Hi Roman,

Good evening.

> I know you are busy, so I have already monkeypatched pyplusplus locally
> for my purposes. But maybe you are interested to add this feature add
> some point in the future:
>
> It would be great if there was a way to tell Py++ that it should not
> represent the C char * type to the ctypes c_char_p, because the later is
> supposed to be used only for \0 terminated strings. A general char*
> should be translated to POINTER(c_char).

Yes I do. I saw your post and the issue(
http://bugs.python.org/issue7569 ). I think it is a good think.

I have only one problem: in Py++( pygccxml) types are "singletones".
What is the best interface for the user to customize such thing?
Should Py++ generate POINTER(c_char) in all cases by default?

I would like to know your opinion.

Thank you.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From Nikolaus at rath.org  Thu Dec 24 00:55:25 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Wed, 23 Dec 2009 18:55:25 -0500
Subject: [C++-sig] [Py++] Don't transform char* to c_char_p
References: <87zl59qyxc.fsf@vostro.rath.org>
	<7465b6170912231312l2505e519j366f1cc1a4f79605@mail.gmail.com>
Message-ID: <87vdfx9rf6.fsf@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
> On Wed, Dec 23, 2009 at 9:21 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>> It would be great if there was a way to tell Py++ that it should not
>> represent the C char * type to the ctypes c_char_p, because the later is
>> supposed to be used only for \0 terminated strings. A general char*
>> should be translated to POINTER(c_char).
>
> Yes I do. I saw your post and the issue(
> http://bugs.python.org/issue7569 ). I think it is a good think.
>
> I have only one problem: in Py++( pygccxml) types are "singletones".
> What is the best interface for the user to customize such thing?
> Should Py++ generate POINTER(c_char) in all cases by default?
>
> I would like to know your opinion.

I think POINTER(c_char) should indeed be the default because it is the
safer choice.

Maybe a parameter a new keyword parameter in the ctypes_module_builder_t
constructor would be a good way to customize this.

On the other hand, the only situation were the user can actually notice
this change would be in a Python callback function that's called from C,
because it will receive a ctypes object rather than a Python string. But
a string can still easily be obtained with string_at (while the reverse
is not possible). So maybe it is also possible to always generate
POINTER(c_char) without the option to change this to c_char_p.


Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From Nikolaus at rath.org  Fri Dec 25 23:43:55 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Fri, 25 Dec 2009 17:43:55 -0500
Subject: [C++-sig] [Py++] ctypes generator produces wrong Python code for
	function pointers in structs
Message-ID: <871vii3c9g.fsf@vostro.rath.org>

Hello,

Py++ translates the struct

typedef int cb_fun_t(char *);
struct info {
   cb_fun_t* cb_fun;
   int flag;
};

into the Python Structure

class info(ctypes.Structure):
    """class info"""
info._fields_ = [ #class info
    ("cb_fun", ctypes.POINTER( ctypes.CFUNCTYPE( ctypes.c_int, ctypes.c_char_p ) )),
    ("flag", ctypes.c_int),
]

This is wrong and produces segfaults. The correct version is

class info(ctypes.Structure):
    """class info"""
info._fields_ = [ #class info
    ("cb_fun", ctypes.CFUNCTYPE( ctypes.c_int, ctypes.c_char_p ) ),
    ("flag", ctypes.c_int),
]


I have attached a small example that demonstrates the problem. lib.c is
a small C library, build_api.py creates the broken api.py with Py++.
api_correct.py is the manually corrected API with the correct structure.
call.py demonstrates that the first API segfaults and the second API
works:

$ python call.py 
Calling do_callback with flag=7
do_callback called with flag=7
cb_fun received 'Hello World'
cb_fun returns 42
cb_fun returned 42
do_callback returned 42
Calling do_callback with flag=7
do_callback called with flag=7
Segmentation fault


Please let me know if you can reproduce the problem. It took me quite
some time to figure this out.


Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C
-------------- next part --------------
A non-text attachment was scrubbed...
Name: lib.c
Type: text/x-csrc
Size: 334 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091225/1ea612ed/attachment.c>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: build_api.py
Type: text/x-python
Size: 499 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091225/1ea612ed/attachment.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: api.py
Type: text/x-python
Size: 792 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091225/1ea612ed/attachment-0001.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: api_correct.py
Type: text/x-python
Size: 775 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091225/1ea612ed/attachment-0002.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: call.py
Type: text/x-python
Size: 650 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091225/1ea612ed/attachment-0003.py>

From Nikolaus at rath.org  Sat Dec 26 01:26:28 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Fri, 25 Dec 2009 19:26:28 -0500
Subject: [C++-sig] [Py++] ctypes generator ignores bit fields
Message-ID: <87my16oa17.fsf@vostro.rath.org>

Hello,

The ctypes code generator translates the struct

struct fuse_file_info {
	int flags;
	unsigned long fh_old;
	int writepage;
	unsigned int direct_io : 1;
	unsigned int keep_cache : 1;
	unsigned int flush : 1;
	unsigned int padding : 29;
	uint64_t fh;
	uint64_t lock_owner;
};

into

fuse_file_info._fields_ = [ #class fuse_file_info
    ("flags", ctypes.c_int),
    ("fh_old", ctypes.c_ulong),
    ("writepage", ctypes.c_int),
    ("direct_io", ctypes.c_uint),
    ("keep_cache", ctypes.c_uint),
    ("flush", ctypes.c_uint),
    ("padding", ctypes.c_uint),
    ("fh", ctypes.c_ulonglong),
    ("lock_owner", ctypes.c_ulonglong),
]

but this is wrong, the correct translation is:

fuse_file_info._fields_ = [
        ('flags', ctypes.c_int),
        ('fh_old', ctypes.c_ulong),
        ('writepage', ctypes.c_int),
        ('direct_io', ctypes.c_uint, 1),
        ('keep_cache', ctypes.c_uint, 1),
        ('flush', ctypes.c_uint, 1),
        ('padding', ctypes.c_uint, 29),
        ('fh', ctypes.c_uint64),
        ('lock_owner', ctypes.c_uint64)]

Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From roman.yakovenko at gmail.com  Sat Dec 26 09:55:52 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Sat, 26 Dec 2009 10:55:52 +0200
Subject: [C++-sig] [Py++] ctypes generator produces wrong Python code
	for function pointers in structs
In-Reply-To: <871vii3c9g.fsf@vostro.rath.org>
References: <871vii3c9g.fsf@vostro.rath.org>
Message-ID: <7465b6170912260055h141d4919la9d841648f8bdbde@mail.gmail.com>

Thanks. I will try to fix the problem this evening.

On Sat, Dec 26, 2009 at 12:43 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> Hello,
>
> Py++ translates the struct
>
> typedef int cb_fun_t(char *);
> struct info {
> ? cb_fun_t* cb_fun;
> ? int flag;
> };
>
> into the Python Structure
>
> class info(ctypes.Structure):
> ? ?"""class info"""
> info._fields_ = [ #class info
> ? ?("cb_fun", ctypes.POINTER( ctypes.CFUNCTYPE( ctypes.c_int, ctypes.c_char_p ) )),
> ? ?("flag", ctypes.c_int),
> ]
>
> This is wrong and produces segfaults. The correct version is
>
> class info(ctypes.Structure):
> ? ?"""class info"""
> info._fields_ = [ #class info
> ? ?("cb_fun", ctypes.CFUNCTYPE( ctypes.c_int, ctypes.c_char_p ) ),
> ? ?("flag", ctypes.c_int),
> ]
>
>
> I have attached a small example that demonstrates the problem. lib.c is
> a small C library, build_api.py creates the broken api.py with Py++.
> api_correct.py is the manually corrected API with the correct structure.
> call.py demonstrates that the first API segfaults and the second API
> works:
>
> $ python call.py
> Calling do_callback with flag=7
> do_callback called with flag=7
> cb_fun received 'Hello World'
> cb_fun returns 42
> cb_fun returned 42
> do_callback returned 42
> Calling do_callback with flag=7
> do_callback called with flag=7
> Segmentation fault
>
>
> Please let me know if you can reproduce the problem. It took me quite
> some time to figure this out.
>
>
> Best,
>
> ? -Nikolaus
>
> --
> ??Time flies like an arrow, fruit flies like a Banana.?
>
> ?PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 ?02CF A9AD B7F8 AE4E 425C
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From roman.yakovenko at gmail.com  Sat Dec 26 09:57:00 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Sat, 26 Dec 2009 10:57:00 +0200
Subject: [C++-sig] [Py++] ctypes generator ignores bit fields
In-Reply-To: <87my16oa17.fsf@vostro.rath.org>
References: <87my16oa17.fsf@vostro.rath.org>
Message-ID: <7465b6170912260057v331b21edsd637545f01d16a35@mail.gmail.com>

Thanks. I will try to fix the problems you reported this evening and
will let you know.

On Sat, Dec 26, 2009 at 2:26 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> Hello,
>
> The ctypes code generator translates the struct
>
> struct fuse_file_info {
> ? ? ? ?int flags;
> ? ? ? ?unsigned long fh_old;
> ? ? ? ?int writepage;
> ? ? ? ?unsigned int direct_io : 1;
> ? ? ? ?unsigned int keep_cache : 1;
> ? ? ? ?unsigned int flush : 1;
> ? ? ? ?unsigned int padding : 29;
> ? ? ? ?uint64_t fh;
> ? ? ? ?uint64_t lock_owner;
> };
>
> into
>
> fuse_file_info._fields_ = [ #class fuse_file_info
> ? ?("flags", ctypes.c_int),
> ? ?("fh_old", ctypes.c_ulong),
> ? ?("writepage", ctypes.c_int),
> ? ?("direct_io", ctypes.c_uint),
> ? ?("keep_cache", ctypes.c_uint),
> ? ?("flush", ctypes.c_uint),
> ? ?("padding", ctypes.c_uint),
> ? ?("fh", ctypes.c_ulonglong),
> ? ?("lock_owner", ctypes.c_ulonglong),
> ]
>
> but this is wrong, the correct translation is:
>
> fuse_file_info._fields_ = [
> ? ? ? ?('flags', ctypes.c_int),
> ? ? ? ?('fh_old', ctypes.c_ulong),
> ? ? ? ?('writepage', ctypes.c_int),
> ? ? ? ?('direct_io', ctypes.c_uint, 1),
> ? ? ? ?('keep_cache', ctypes.c_uint, 1),
> ? ? ? ?('flush', ctypes.c_uint, 1),
> ? ? ? ?('padding', ctypes.c_uint, 29),
> ? ? ? ?('fh', ctypes.c_uint64),
> ? ? ? ?('lock_owner', ctypes.c_uint64)]
>
> Best,
>
> ? -Nikolaus
>
> --
> ??Time flies like an arrow, fruit flies like a Banana.?
>
> ?PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 ?02CF A9AD B7F8 AE4E 425C
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig



-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From roman.yakovenko at gmail.com  Sat Dec 26 21:33:48 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Sat, 26 Dec 2009 22:33:48 +0200
Subject: [C++-sig] [Py++] ctypes generator ignores bit fields
In-Reply-To: <7465b6170912260057v331b21edsd637545f01d16a35@mail.gmail.com>
References: <87my16oa17.fsf@vostro.rath.org>
	<7465b6170912260057v331b21edsd637545f01d16a35@mail.gmail.com>
Message-ID: <7465b6170912261233x618b1c49h530253e116110c08@mail.gmail.com>

On Sat, Dec 26, 2009 at 10:57 AM, Roman Yakovenko
<roman.yakovenko at gmail.com> wrote:
> Thanks. I will try to fix the problems you reported this evening and
> will let you know.

Nikolaus, I fixed both issues you reported:
* http://pygccxml.svn.sourceforge.net/pygccxml/?rev=1784&view=rev
* http://pygccxml.svn.sourceforge.net/pygccxml/?rev=1785&view=rev

Thanks for a good bug reports.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From Nikolaus at rath.org  Sun Dec 27 01:46:05 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Sat, 26 Dec 2009 19:46:05 -0500
Subject: [C++-sig] [Py++] ctypes generator ignores bit fields
References: <87my16oa17.fsf@vostro.rath.org>
	<7465b6170912260057v331b21edsd637545f01d16a35@mail.gmail.com>
	<7465b6170912261233x618b1c49h530253e116110c08@mail.gmail.com>
Message-ID: <87d4211bxu.fsf@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
> On Sat, Dec 26, 2009 at 10:57 AM, Roman Yakovenko
> <roman.yakovenko at gmail.com> wrote:
>> Thanks. I will try to fix the problems you reported this evening and
>> will let you know.
>
> Nikolaus, I fixed both issues you reported:
> * http://pygccxml.svn.sourceforge.net/pygccxml/?rev=1784&view=rev
> * http://pygccxml.svn.sourceforge.net/pygccxml/?rev=1785&view=rev

Hi,

Thanks for looking into this. I'm afraid the fix for the CFUNCTYPE vs
POINTER(CFUNCTYPE) issue is not completely working though.

The small testcase that I send around indeed generates correct code with the
new version. However, if you try the same code on a more complex header
(like the attached one, needs fuse headers to be installed) it still
produces the wrong result:


fuse_lowlevel_ops._fields_ = [ #class fuse_lowlevel_ops
    ("init", ctypes.POINTER( ctypes.CFUNCTYPE( None, ctypes.c_void_p, ctypes.POINTER( fuse_conn_info ) ) )),
    ("destroy", ctypes.POINTER( ctypes.CFUNCTYPE( None, ctypes.c_void_p
    ) )),
[...]


Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fuse_ctypes.h
Type: text/x-chdr
Size: 731 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091226/e7de6bbf/attachment.h>

From roman.yakovenko at gmail.com  Sun Dec 27 09:10:21 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Sun, 27 Dec 2009 10:10:21 +0200
Subject: [C++-sig] [Py++] ctypes generator ignores bit fields
In-Reply-To: <87d4211bxu.fsf@vostro.rath.org>
References: <87my16oa17.fsf@vostro.rath.org>
	<7465b6170912260057v331b21edsd637545f01d16a35@mail.gmail.com>
	<7465b6170912261233x618b1c49h530253e116110c08@mail.gmail.com>
	<87d4211bxu.fsf@vostro.rath.org>
Message-ID: <7465b6170912270010i1b46945bh69074ed96f0eee4@mail.gmail.com>

On Sun, Dec 27, 2009 at 2:46 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> Hi,
>
> Thanks for looking into this. I'm afraid the fix for the CFUNCTYPE vs
> POINTER(CFUNCTYPE) issue is not completely working though.
>
> The small testcase that I send around indeed generates correct code with the
> new version. However, if you try the same code on a more complex header
> (like the attached one, needs fuse headers to be installed) it still
> produces the wrong result:
>
>
> fuse_lowlevel_ops._fields_ = [ #class fuse_lowlevel_ops
> ? ?("init", ctypes.POINTER( ctypes.CFUNCTYPE( None, ctypes.c_void_p, ctypes.POINTER( fuse_conn_info ) ) )),
> ? ?("destroy", ctypes.POINTER( ctypes.CFUNCTYPE( None, ctypes.c_void_p
> ? ?) )),
> [...]

:-(. I will take a look on this today.


-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From roman.yakovenko at gmail.com  Sun Dec 27 20:16:21 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Sun, 27 Dec 2009 21:16:21 +0200
Subject: [C++-sig] [Py++] ctypes generator ignores bit fields
In-Reply-To: <7465b6170912270010i1b46945bh69074ed96f0eee4@mail.gmail.com>
References: <87my16oa17.fsf@vostro.rath.org>
	<7465b6170912260057v331b21edsd637545f01d16a35@mail.gmail.com>
	<7465b6170912261233x618b1c49h530253e116110c08@mail.gmail.com>
	<87d4211bxu.fsf@vostro.rath.org>
	<7465b6170912270010i1b46945bh69074ed96f0eee4@mail.gmail.com>
Message-ID: <7465b6170912271116o1061cd53p33fab27023b5f077@mail.gmail.com>

On Sun, Dec 27, 2009 at 10:10 AM, Roman Yakovenko
<roman.yakovenko at gmail.com> wrote:
> On Sun, Dec 27, 2009 at 2:46 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>> Hi,
>>
>> Thanks for looking into this. I'm afraid the fix for the CFUNCTYPE vs
>> POINTER(CFUNCTYPE) issue is not completely working though.
>>
>> The small testcase that I send around indeed generates correct code with the
>> new version. However, if you try the same code on a more complex header
>> (like the attached one, needs fuse headers to be installed) it still
>> produces the wrong result:
>>
>>
>> fuse_lowlevel_ops._fields_ = [ #class fuse_lowlevel_ops
>> ? ?("init", ctypes.POINTER( ctypes.CFUNCTYPE( None, ctypes.c_void_p, ctypes.POINTER( fuse_conn_info ) ) )),
>> ? ?("destroy", ctypes.POINTER( ctypes.CFUNCTYPE( None, ctypes.c_void_p
>> ? ?) )),
>> [...]
>
> :-(. I will take a look on this today.

Nikolaus, unfortunately I can't reproduce the error. I attached the
generated file and the "source" file.

Can you investigate a little bit more?

Thanks.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fuse_ctypes.h
Type: unknown/unknown
Size: 739 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091227/3c988031/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fuse_ctypes.py
Type: text/x-python
Size: 17720 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20091227/3c988031/attachment-0001.py>

From roman.yakovenko at gmail.com  Sun Dec 27 22:18:44 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Sun, 27 Dec 2009 23:18:44 +0200
Subject: [C++-sig] [Py++] Don't transform char* to c_char_p
In-Reply-To: <87vdfx9rf6.fsf@vostro.rath.org>
References: <87zl59qyxc.fsf@vostro.rath.org>
	<7465b6170912231312l2505e519j366f1cc1a4f79605@mail.gmail.com>
	<87vdfx9rf6.fsf@vostro.rath.org>
Message-ID: <7465b6170912271318s2fe9d01cx5ea2ab7bc336e336@mail.gmail.com>

On Thu, Dec 24, 2009 at 1:55 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> I think POINTER(c_char) should indeed be the default because it is the
> safer choice.

Unfortunately, I don't agree with you. It is a safer choice - right,
but it should not be the default.

> Maybe a parameter a new keyword parameter in the ctypes_module_builder_t
> constructor would be a good way to customize this.
>
> On the other hand, the only situation were the user can actually notice
> this change would be in a Python callback function that's called from C,
> because it will receive a ctypes object rather than a Python string. But
> a string can still easily be obtained with string_at (while the reverse
> is not possible). So maybe it is also possible to always generate
> POINTER(c_char) without the option to change this to c_char_p.

I added new functionality, which allows to generate POINTER( c_char )
instead of c_char_p.

The relevant commit are:
* http://pygccxml.svn.sourceforge.net/pygccxml/?rev=1786&view=rev
* http://pygccxml.svn.sourceforge.net/pygccxml/?rev=1790&view=rev

The functionality should be used like:

mb = ctypes_module_builder_t( ... )
mb.treat_char_ptr_as_binary_data = True

HTH

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From Nikolaus at rath.org  Sun Dec 27 23:54:16 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Sun, 27 Dec 2009 17:54:16 -0500
Subject: [C++-sig] [Py++] Export #define directives to ctypes
Message-ID: <87hbrcowo7.fsf@vostro.rath.org>

Hello,

Is there a way to export preprocessor definitions (#define ...) with the
ctypes code generator?

At the moment I'm using this rather nasty hack:

/* Header file */
// FUSE_SET_ATTR_* is defined with #define in fuse_lowlevel.h
#include <fuse_lowlevel.h>
const int D_FUSE_SET_ATTR_MODE = FUSE_SET_ATTR_MODE; 
const int D_FUSE_SET_ATTR_UID = FUSE_SET_ATTR_UID;
// ...

# Code generator
mb.write_module(os.path.join(code_path, 'ctypes_api.py'))
fh = open(os.path.join(code_path, 'ctypes_api.py'), 'a')
fh.write('\n')
for var in mb.global_ns.decls(lambda f: f.name.startswith('D_')):
    fh.write('{0} = {1}\n'.format(var.name[2:], var._value))
fh.close()


But it'd be nice if there was a better solution (so that I don't have to
define a new variable for every #define that I want to export).


Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From Nikolaus at rath.org  Sun Dec 27 23:47:47 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Sun, 27 Dec 2009 17:47:47 -0500
Subject: [C++-sig] [Py++] ctypes generator ignores bit fields
References: <87my16oa17.fsf@vostro.rath.org>
	<7465b6170912260057v331b21edsd637545f01d16a35@mail.gmail.com>
	<7465b6170912261233x618b1c49h530253e116110c08@mail.gmail.com>
	<87d4211bxu.fsf@vostro.rath.org>
	<7465b6170912270010i1b46945bh69074ed96f0eee4@mail.gmail.com>
	<7465b6170912271116o1061cd53p33fab27023b5f077@mail.gmail.com>
Message-ID: <87k4w8owz0.fsf@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
> On Sun, Dec 27, 2009 at 10:10 AM, Roman Yakovenko
> <roman.yakovenko at gmail.com> wrote:
>> On Sun, Dec 27, 2009 at 2:46 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>>> Hi,
>>>
>>> Thanks for looking into this. I'm afraid the fix for the CFUNCTYPE vs
>>> POINTER(CFUNCTYPE) issue is not completely working though.
>>>
>>> The small testcase that I send around indeed generates correct code with the
>>> new version. However, if you try the same code on a more complex header
>>> (like the attached one, needs fuse headers to be installed) it still
>>> produces the wrong result:
>>>
>>>
>>> fuse_lowlevel_ops._fields_ = [ #class fuse_lowlevel_ops
>>> ? ?("init", ctypes.POINTER( ctypes.CFUNCTYPE( None, ctypes.c_void_p, ctypes.POINTER( fuse_conn_info ) ) )),
>>> ? ?("destroy", ctypes.POINTER( ctypes.CFUNCTYPE( None, ctypes.c_void_p
>>> ? ?) )),
>>> [...]
>>
>> :-(. I will take a look on this today.
>
> Nikolaus, unfortunately I can't reproduce the error. I attached the
> generated file and the "source" file.
>
> Can you investigate a little bit more?

You are right. The reason why it didn't work for me at first was the
monkeypatching that I did to get POINTER(c_char) instead of c_char_p.
Luckily enough, with your other commit I don't need that patching
anymore either. Thanks again!


Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From Nikolaus at rath.org  Sun Dec 27 23:46:28 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Sun, 27 Dec 2009 17:46:28 -0500
Subject: [C++-sig] [Py++] Don't transform char* to c_char_p
References: <87zl59qyxc.fsf@vostro.rath.org>
	<7465b6170912231312l2505e519j366f1cc1a4f79605@mail.gmail.com>
	<87vdfx9rf6.fsf@vostro.rath.org>
	<7465b6170912271318s2fe9d01cx5ea2ab7bc336e336@mail.gmail.com>
Message-ID: <87my14ox17.fsf@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
> On Thu, Dec 24, 2009 at 1:55 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>> I think POINTER(c_char) should indeed be the default because it is the
>> safer choice.
>
> Unfortunately, I don't agree with you. It is a safer choice - right,
> but it should not be the default.

Don't worry, I don't care what the default is as long as I can change it
:-).

>> Maybe a parameter a new keyword parameter in the ctypes_module_builder_t
>> constructor would be a good way to customize this.
>>
>> On the other hand, the only situation were the user can actually notice
>> this change would be in a Python callback function that's called from C,
>> because it will receive a ctypes object rather than a Python string. But
>> a string can still easily be obtained with string_at (while the reverse
>> is not possible). So maybe it is also possible to always generate
>> POINTER(c_char) without the option to change this to c_char_p.
>
> I added new functionality, which allows to generate POINTER( c_char )
> instead of c_char_p.

Works like a charm, thanks a lot!

Best,

   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From roman.yakovenko at gmail.com  Mon Dec 28 06:36:46 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Mon, 28 Dec 2009 07:36:46 +0200
Subject: [C++-sig] [Py++] Export #define directives to ctypes
In-Reply-To: <87hbrcowo7.fsf@vostro.rath.org>
References: <87hbrcowo7.fsf@vostro.rath.org>
Message-ID: <7465b6170912272136p4fda6eb2n1dbd09828515e0ec@mail.gmail.com>

On Mon, Dec 28, 2009 at 12:54 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> Hello,
>
> Is there a way to export preprocessor definitions (#define ...) with the
> ctypes code generator?
>
> At the moment I'm using this rather nasty hack:
>
> /* Header file */
> // FUSE_SET_ATTR_* is defined with #define in fuse_lowlevel.h
> #include <fuse_lowlevel.h>
> const int D_FUSE_SET_ATTR_MODE = FUSE_SET_ATTR_MODE;
> const int D_FUSE_SET_ATTR_UID = FUSE_SET_ATTR_UID;
> // ...
>
> # Code generator
> mb.write_module(os.path.join(code_path, 'ctypes_api.py'))
> fh = open(os.path.join(code_path, 'ctypes_api.py'), 'a')
> fh.write('\n')
> for var in mb.global_ns.decls(lambda f: f.name.startswith('D_')):
> ? ?fh.write('{0} = {1}\n'.format(var.name[2:], var._value))
> fh.close()
>
>
> But it'd be nice if there was a better solution (so that I don't have to
> define a new variable for every #define that I want to export).

ctypes code generator is missing an important future to make it work -
adding user code. I will try to implement it pretty soon.

Let me explain.

mb = ctypes_module_builder_t( ... )
for file_path in pygccxml.declarations.declaration_files( mb.global_ns ):
    if file_path is not under "fuse" source code directory:
         continue
    else:
         use simple regex to search for "#define FUSE_*" expression,
extract its value and add it to mb.

Does this solution makes sense to you? If so, I will take a look on it
this evening.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From Nikolaus at rath.org  Mon Dec 28 16:57:10 2009
From: Nikolaus at rath.org (Nikolaus Rath)
Date: Mon, 28 Dec 2009 10:57:10 -0500
Subject: [C++-sig] [Py++] Export #define directives to ctypes
References: <87hbrcowo7.fsf@vostro.rath.org>
	<7465b6170912272136p4fda6eb2n1dbd09828515e0ec@mail.gmail.com>
Message-ID: <87eimf1489.fsf@vostro.rath.org>

Roman Yakovenko <roman.yakovenko at gmail.com> writes:
> On Mon, Dec 28, 2009 at 12:54 AM, Nikolaus Rath <Nikolaus at rath.org> wrote:
>> Hello,
>>
>> Is there a way to export preprocessor definitions (#define ...) with the
>> ctypes code generator?
>>
>> At the moment I'm using this rather nasty hack:
>>
>> /* Header file */
>> // FUSE_SET_ATTR_* is defined with #define in fuse_lowlevel.h
>> #include <fuse_lowlevel.h>
>> const int D_FUSE_SET_ATTR_MODE = FUSE_SET_ATTR_MODE;
>> const int D_FUSE_SET_ATTR_UID = FUSE_SET_ATTR_UID;
>> // ...
>>
>> # Code generator
>> mb.write_module(os.path.join(code_path, 'ctypes_api.py'))
>> fh = open(os.path.join(code_path, 'ctypes_api.py'), 'a')
>> fh.write('\n')
>> for var in mb.global_ns.decls(lambda f: f.name.startswith('D_')):
>> ? ?fh.write('{0} = {1}\n'.format(var.name[2:], var._value))
>> fh.close()
>>
>>
>> But it'd be nice if there was a better solution (so that I don't have to
>> define a new variable for every #define that I want to export).
>
> ctypes code generator is missing an important future to make it work -
> adding user code. I will try to implement it pretty soon.
>
> Let me explain.
>
> mb = ctypes_module_builder_t( ... )
> for file_path in pygccxml.declarations.declaration_files( mb.global_ns ):
>     if file_path is not under "fuse" source code directory:
>          continue
>     else:
>          use simple regex to search for "#define FUSE_*" expression,
> extract its value and add it to mb.
>
> Does this solution makes sense to you?

Not quite. I'd rather have the "use simple regex to search for "#define
FUSE_*" expression" part in Py++ rather than in my own code. Also, I'm
not sure if a simple regex is really a good solution for this. The
following will probably fail already (i.e., give the value of FUSE_FOO
as the string 'FUSE_BLA' rather than the number 42):

#define FUSE_BLA 42
#define FUSE_FOO FUSE_BLA


Best,


   -Nikolaus

-- 
 ?Time flies like an arrow, fruit flies like a Banana.?

  PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6  02CF A9AD B7F8 AE4E 425C


From roman.yakovenko at gmail.com  Mon Dec 28 21:32:44 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Mon, 28 Dec 2009 22:32:44 +0200
Subject: [C++-sig] [Py++] Export #define directives to ctypes
In-Reply-To: <87eimf1489.fsf@vostro.rath.org>
References: <87hbrcowo7.fsf@vostro.rath.org>
	<7465b6170912272136p4fda6eb2n1dbd09828515e0ec@mail.gmail.com>
	<87eimf1489.fsf@vostro.rath.org>
Message-ID: <7465b6170912281232j526a142am874809ec134d25f6@mail.gmail.com>

On Mon, Dec 28, 2009 at 5:57 PM, Nikolaus Rath <Nikolaus at rath.org> wrote:
> Not quite. I'd rather have the "use simple regex to search for "#define
> FUSE_*" expression" part in Py++ rather than in my own code.

It is definitely possible, if we will agree on a desired/sufficient
functionality.

> Also, I'm
> not sure if a simple regex is really a good solution for this. The
> following will probably fail already (i.e., give the value of FUSE_FOO
> as the string 'FUSE_BLA' rather than the number 42):
>
> #define FUSE_BLA 42
> #define FUSE_FOO FUSE_BLA

It will fall on many other cases too, but lets take a look on your use case:

grep -nHIr -- FUSE_.* (in directory: /usr/include/fuse)

grep -nHIr -- FUSE_.* (in directory: /usr/include/fuse)

./fuse_compat.h:193:#define FUSE_DEBUG_COMPAT1	 (1 << 1)

./fuse.h:9:#ifndef _FUSE_H_

./fuse.h:10:#define _FUSE_H_

./fuse.h:16: * IMPORTANT: you should define FUSE_USE_VERSION before
including this

./fuse.h:22:#ifndef FUSE_USE_VERSION

./fuse.h:23:#define FUSE_USE_VERSION 21

./fuse.h:678: * Filesystem modules are registered with the
FUSE_REGISTER_MODULE()

./fuse.h:716: * This function is used by FUSE_REGISTER_MODULE and
there's usually

./fuse.h:727:#define FUSE_REGISTER_MODULE(name_, factory_)				  \

./fuse.h:779:#if FUSE_USE_VERSION < 26

./fuse.h:782:#  if FUSE_USE_VERSION == 25

./fuse.h:789:#  elif FUSE_USE_VERSION == 22

./fuse.h:797:#  elif FUSE_USE_VERSION == 24

./fuse.h:804:#    if FUSE_USE_VERSION == 21

./fuse.h:817:#      define FUSE_DEBUG FUSE_DEBUG_COMPAT1

./fuse.h:826:#endif /* _FUSE_H_ */

./fuse_common.h:11:#if !defined(_FUSE_H_) && !defined(_FUSE_LOWLEVEL_H_)

./fuse_common.h:15:#ifndef _FUSE_COMMON_H_

./fuse_common.h:16:#define _FUSE_COMMON_H_

./fuse_common.h:22:#define FUSE_MAJOR_VERSION 2

./fuse_common.h:25:#define FUSE_MINOR_VERSION 7

./fuse_common.h:27:#define FUSE_MAKE_VERSION(maj, min)  ((maj) * 10 + (min))

./fuse_common.h:28:#define FUSE_VERSION
FUSE_MAKE_VERSION(FUSE_MAJOR_VERSION, FUSE_MINOR_VERSION)

./fuse_common.h:209:#if FUSE_USE_VERSION < 26

./fuse_common.h:211:#	 if FUSE_USE_VERSION < 25

./fuse_common.h:216:#    undef FUSE_MINOR_VERSION

./fuse_common.h:219:#    if FUSE_USE_VERSION == 25

./fuse_common.h:220:#	 define FUSE_MINOR_VERSION 5

./fuse_common.h:222:#    elif FUSE_USE_VERSION == 24 || FUSE_USE_VERSION == 22

./fuse_common.h:223:#	 define FUSE_MINOR_VERSION 4

./fuse_common.h:225:#    elif FUSE_USE_VERSION == 21

./fuse_common.h:226:#	 define FUSE_MINOR_VERSION 1

./fuse_common.h:228:#    elif FUSE_USE_VERSION == 11

./fuse_common.h:230:#	 undef FUSE_MAJOR_VERSION

./fuse_common.h:231:#	 define FUSE_MAJOR_VERSION 1

./fuse_common.h:232:#	 define FUSE_MINOR_VERSION 1

./fuse_common.h:243:#endif /* _FUSE_COMMON_H_ */

./fuse_lowlevel.h:9:#ifndef _FUSE_LOWLEVEL_H_

./fuse_lowlevel.h:10:#define _FUSE_LOWLEVEL_H_

./fuse_lowlevel.h:16: * IMPORTANT: you should define FUSE_USE_VERSION
before including this

./fuse_lowlevel.h:22:#ifndef FUSE_USE_VERSION

./fuse_lowlevel.h:23:#define FUSE_USE_VERSION 24

./fuse_lowlevel.h:44:#define FUSE_ROOT_ID 1

./fuse_lowlevel.h:115:#define FUSE_SET_ATTR_MODE	(1 << 0)

./fuse_lowlevel.h:116:#define FUSE_SET_ATTR_UID	(1 << 1)

./fuse_lowlevel.h:117:#define FUSE_SET_ATTR_GID	(1 << 2)

./fuse_lowlevel.h:118:#define FUSE_SET_ATTR_SIZE	(1 << 3)

./fuse_lowlevel.h:119:#define FUSE_SET_ATTR_ATIME	(1 << 4)

./fuse_lowlevel.h:120:#define FUSE_SET_ATTR_MTIME	(1 << 5)

./fuse_lowlevel.h:1367:#if FUSE_USE_VERSION < 26

./fuse_lowlevel.h:1371:#  if FUSE_USE_VERSION == 25

./fuse_lowlevel.h:1374:#  elif FUSE_USE_VERSION == 24

./fuse_lowlevel.h:1389:#endif /* _FUSE_LOWLEVEL_H_ */

./fuse_opt.h:9:#ifndef _FUSE_OPT_H_

./fuse_opt.h:10:#define _FUSE_OPT_H_

./fuse_opt.h:97:#define FUSE_OPT_KEY(templ, key) { templ, -1U, key }

./fuse_opt.h:103:#define FUSE_OPT_END { .templ = NULL }

./fuse_opt.h:122:#define FUSE_ARGS_INIT(argc, argv) { argc, argv, 0 }

./fuse_opt.h:128:#define FUSE_OPT_KEY_OPT     -1

./fuse_opt.h:136:#define FUSE_OPT_KEY_NONOPT  -2

./fuse_opt.h:144:#define FUSE_OPT_KEY_KEEP -3

./fuse_opt.h:152:#define FUSE_OPT_KEY_DISCARD -4

./fuse_opt.h:261:#endif /* _FUSE_OPT_H_ */

Search completed with 63 matches.


I could be wrong, but a proposed solution successfully handles (
more-or-less ) 70% of use cases. It is not that bad.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From roman.yakovenko at gmail.com  Mon Dec 28 22:50:35 2009
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Mon, 28 Dec 2009 23:50:35 +0200
Subject: [C++-sig] [Py++] Export #define directives to ctypes
In-Reply-To: <7465b6170912281232j526a142am874809ec134d25f6@mail.gmail.com>
References: <87hbrcowo7.fsf@vostro.rath.org>
	<7465b6170912272136p4fda6eb2n1dbd09828515e0ec@mail.gmail.com>
	<87eimf1489.fsf@vostro.rath.org>
	<7465b6170912281232j526a142am874809ec134d25f6@mail.gmail.com>
Message-ID: <7465b6170912281350q1d09ac24i7cf3acb5fdffea91@mail.gmail.com>

Hi Nikolaus.

The ability to inject user code was added. The relevant commit is:
http://pygccxml.svn.sourceforge.net/pygccxml/?rev=1792&view=rev

The usage is pretty simple:

mb = ctypes_module_builder_t( ... )
#will add code to the top of the module
mb.add_module_code( "#top", tail=False )

#will add code to the bottom of the module
mb.add_module_code( "#bottom", tail=True )


Even if it doesn't solve you the current issue, it can help you with PyLint.

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From lists_ravi at lavabit.com  Tue Dec 29 03:43:58 2009
From: lists_ravi at lavabit.com (Ravi)
Date: Mon, 28 Dec 2009 21:43:58 -0500
Subject: [C++-sig] function types as user-defined converters
In-Reply-To: <4AF47E91.9070503@resophonic.com>
References: <55154.65.220.96.226.1257193994.squirrel@lavabit.com>
	<4AF47E91.9070503@resophonic.com>
Message-ID: <200912282143.58716.lists_ravi@lavabit.com>

On Friday 06 November 2009 14:52:49 troy d. straszheim wrote:
> Currently, the converter type still leaks out to python in the signature:
> 
>      class S(Boost.Python.instance)
>       |  add(...)
>       |      add( (S)arg1, (object)arg2) -> None :
>       |
>       |          C++ signature :
>       |              void add(S*,to_optional<int> 
> (*)(boost::python::api::object))
> 
> 
> converters won't nest:
> 
>    def("foo", as<returnvalue(arg1, conv1(conv2(arg2)))>(&somefn)); // nah
> 
> I dont have a mechanism for hooking a user defined converter into the
> return type.  I'm going to have to think about that one a bit more.

Did you have any ideas on this, Troy? This is the last bit I think I'd need in 
order to replace my code with yours.

Regards,
Ravi