From Christian.BERGER at 3ds.com  Tue Jan  5 14:15:32 2016
From: Christian.BERGER at 3ds.com (BERGER Christian)
Date: Tue, 5 Jan 2016 19:15:32 +0000
Subject: [C++-sig] Beginner - PyObject_CallObject fails because imported
 module is not recognized
Message-ID: <867172D4AC0F194AACCDD3853364A34882BDFB@AG-DCC-MBX13.dsone.3ds.com>

We have this simple (test) script that defines a function that we want to call via python's c api:

import pprint
import sys
counter= 0
def onInitialize(ctxt):
   global counter
   counter+=1
   pprint.pprint(sys.path)
   return counter

While executing this works fine from the python command line, when using PyObject_CallObject to execute 'onInitialize', we're getting an error that pprint was undefined: "name 'pprint' is not defined".
Moving the import statement into the function resolves the issue, but that seems wrong, and won't work for us (we want our customers to be able to provide their own implementation of onInitialize() and a bunch of other functions).
I'm assuming I'm missing something very basic, but I haven't found it...
The interesting thing is that the global variable counter is defined and increments as desired/expected when calling the function via PyObject_CallObject(.) repeatedly.

The environment is
Python 3.5.1 (64bit)
Win7


The source code is below, if anybody wants the vs solution, that can be provided.
Thanks for your help,
Christian



#include "stdafx.h"
#include "Python.h"

void hackIt()
{
    const char* const pTestScripts =
        "import pprint\n"
        "import sys\n"
        "counter= 0\n"
        "def onInitialize(ctxt):\n"
        "  global counter\n"
        "  counter+=1\n"
        "  pprint.pprint(sys.path)\n"
        "  return counter\n";

    Py_Initialize();
    PyObject* pModule = PyImport_AddModule("__main__");
    PyObject* pPyMainDict = PyModule_GetDict(pModule);
    _Py_static_string(PyId_string, "<string>");
    PyObject* pFileName = _PyUnicode_FromId(&PyId_string);
    PyObject* pScriptObject = Py_CompileStringObject(pTestScripts, pFileName, Py_file_input, NULL, 0);
    PyObject* pLocalDictionary = PyDict_New();
    if (pScriptObject)
    {
        PyObject* pResult = PyEval_EvalCode(pScriptObject, pPyMainDict, pLocalDictionary);
        if (NULL != pResult)
        {
            PyObject* pFxnObj = PyDict_GetItemString(pLocalDictionary, "onInitialize");
            long retval = -1;
            PyObject* pTuple = PyTuple_New(1);
            PyObject* pParam = PyLong_FromLong(42);

            PyTuple_SetItem(pTuple, 0, pParam); //tuple takes ownership of pParam
            PyObject* pResultObj = PyObject_CallObject(pFxnObj, pTuple);
            Py_XDECREF(pTuple);

           if (pResultObj)
            {
                printf("got a %ld\n", PyLong_AsLong(pResultObj));
                Py_DECREF(pResultObj);
            }
            else
            {
                PyObject* pPyErr = NULL;
                if (pPyErr = PyErr_Occurred())
                {
                    const char* pStr = NULL;
                    PyObject* pTypeObj = NULL;
                    PyObject* pValueObj = NULL;
                    PyObject* pWhatObj = NULL;
                    PyErr_Fetch(&pTypeObj, &pValueObj, &pWhatObj);
                    if (pValueObj)
                    {
                        if (PyBytes_Check(pValueObj))
                        {
                            printf("Error: %s\n", PyBytes_AsString(pValueObj));
                        }
                        if (PyUnicode_Check(pValueObj))
                        {
                            wchar_t buffer[1024];
                            PyUnicode_AsWideChar(pValueObj, buffer, 1024);
                            printf("Error: %S\n", buffer);
                        }
                    }
                    Py_XDECREF(pWhatObj);
                    Py_XDECREF(pValueObj);
                    Py_XDECREF(pTypeObj);
                }
            }
        }
    }
    Py_XDECREF(pLocalDictionary);
    Py_XDECREF(pScriptObject);
    Py_Finalize();
}

int _tmain(int argc, _TCHAR* argv[])
{
    hackIt();
    return 0;
}



This email and any attachments are intended solely for the use of the individual or entity to whom it is addressed and may be confidential and/or privileged.

If you are not one of the named recipients or have received this email in error,

(i) you should not read, disclose, or copy it,

(ii) please notify sender of your receipt by reply email and delete this email and all attachments,

(iii) Dassault Systemes does not accept or assume any liability or responsibility for any use of or reliance on this email.

For other languages, go to http://www.3ds.com/terms/email-disclaimer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20160105/6b6302d2/attachment.html>

From stefan at seefeld.name  Wed Jan  6 08:57:51 2016
From: stefan at seefeld.name (Stefan Seefeld)
Date: Wed, 6 Jan 2016 08:57:51 -0500
Subject: [C++-sig] Beginner - PyObject_CallObject fails because imported
 module is not recognized
In-Reply-To: <867172D4AC0F194AACCDD3853364A34882BDFB@AG-DCC-MBX13.dsone.3ds.com>
References: <867172D4AC0F194AACCDD3853364A34882BDFB@AG-DCC-MBX13.dsone.3ds.com>
Message-ID: <568D1D5F.8050408@seefeld.name>

Hi Christian,

for the sake of simplicity I have transcribed your example to pure
Python, which also exhibits the same behavior:

#!/usr/bin/env python

script = """import pprint
import sys
counter= 0
def onInitialize(ctxt):
  global counter
  counter+=1
  pprint.pprint(sys.path)
  return counter"""


code = compile(script, "<input>", 'exec')
globals = {}
locals = {}
eval(code, globals, locals)
callback = locals['onInitialize']
callback(())


The problem is that, while the "globals" object is shared between the
module and the function, the "locals" is not (i.e., the function gets
its own local namespace, which is distinct from the local namespace of
the module).
To get what you want you may simply replace the "locals" argument in the
"eval" call above by "globals" (or simply put "None" there, which has
the same effect.

In any case, I'd recommend you rewrite your code using the Boost.Python
bindings; it will become much more compact and readable, and will look
very much like the Python code above. :-)

Regards,
        Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...


From svalorzen at gmail.com  Thu Jan  7 05:43:30 2016
From: svalorzen at gmail.com (Eugenio Bargiacchi)
Date: Thu, 7 Jan 2016 11:43:30 +0100
Subject: [C++-sig] [Boost::Python] Registered class not recognized when
 returned by reference.
Message-ID: <CAHfn=+sE7uRY1iJ+-OdjtHggZrQC-DoyRc+KE2QxcL1uJ_2guw@mail.gmail.com>

Hello everyone,

I have a member function with signature

 const Matrix2D & foo const ();

Which returns an internal data structure. Matrix2D is

 using Matrix2D = Eigen::Matrix<double, Eigen::Dynamic,
Eigen::Dynamic, Eigen::RowMajor | Eigen::AutoAlign>;

I have explicitly set a to_python converter for Matrix2D, and I have
exported the function with a call policy of return_internal_reference.
However, when this function is called from Python I get

TypeError: No Python class registered for C++ class
Eigen::Matrix<double, -1, -1, 1, -1, -1>

This happens also if the reference is not const. At the same time, if I
modify the call policy to return_value_policy<copy_const_reference>()
Python can decode the matrix just fine. The same holds true if I return the
Matrix2D by value. However I really need to have a reference to the matrix
in order store it into other classes.

How can I make Python see the data structure through the reference?

The converter in case it is needed:

boost::python::list toPythonList(const double * data, size_t num) {
    boost::python::list list;
    for (size_t i = 0; i < num; ++i)
        list.append(data[i]);

    return list;}
struct Matrix2DToPython{
    static PyObject* convert(Matrix2D const & m)
    {
        boost::python::list list;
        for (int i = 0; i < m.rows(); ++i)
        {
            list.append(toPythonList(m.row(i).data(), m.cols()));
        }

        return boost::python::incref(list.ptr());
    }};

Thanks,

Eugenio
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20160107/24f58dea/attachment.html>

From stefan at seefeld.name  Thu Jan  7 09:16:58 2016
From: stefan at seefeld.name (Stefan Seefeld)
Date: Thu, 7 Jan 2016 09:16:58 -0500
Subject: [C++-sig] [Boost::Python] Registered class not recognized when
 returned by reference.
In-Reply-To: <CAHfn=+sE7uRY1iJ+-OdjtHggZrQC-DoyRc+KE2QxcL1uJ_2guw@mail.gmail.com>
References: <CAHfn=+sE7uRY1iJ+-OdjtHggZrQC-DoyRc+KE2QxcL1uJ_2guw@mail.gmail.com>
Message-ID: <568E735A.4040905@seefeld.name>

hi Eugenio,

is there any reason not to expose Matrix2D via class_<...>, rather than
define a converter ? Wouldn't that solve your problem trivially ?
(I'm not sure I understand the issue yet, but thought that the above
might be an expedient solution.)

HTH,

        Stefan

On 07.01.2016 05:43, Eugenio Bargiacchi wrote:
>
> Hello everyone,
>
> I have a member function with signature
>
> |constMatrix2D&foo const();|
>
> Which returns an internal data structure. |Matrix2D| is
>
> |usingMatrix2D=Eigen::Matrix<double,Eigen::Dynamic,Eigen::Dynamic,Eigen::RowMajor|Eigen::AutoAlign>;|
>
> I have explicitly set a |to_python| converter for |Matrix2D|, and I
> have exported the function with a call policy of
> |return_internal_reference|. However, when this function is called
> from Python I get
>
> |TypeError:NoPythonclassregistered
> forC++classEigen::Matrix<double,-1,-1,1,-1,-1>|
>
> This happens also if the reference is not |const|. At the same time,
> if I modify the call policy to
> |return_value_policy<copy_const_reference>()| Python can decode the
> matrix just fine. The same holds true if I return the |Matrix2D| by
> value. However I really need to have a reference to the matrix in
> order store it into other classes.
>
> How can I make Python see the data structure through the reference?
>
> The converter in case it is needed:
>
> |boost::python::list
> toPythonList(constdouble*data,size_tnum){boost::python::list
> list;for(size_ti =0;i
> <num;++i)list.append(data[i]);returnlist;}structMatrix2DToPython{staticPyObject*convert(Matrix2Dconst&m){boost::python::list
> list;for(inti =0;i
> <m.rows();++i){list.append(toPythonList(m.row(i).data(),m.cols()));}returnboost::python::incref(list.ptr());}};
> Thanks, |
> |Eugenio|
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> https://mail.python.org/mailman/listinfo/cplusplus-sig


-- 

      ...ich hab' noch einen Koffer in Berlin...


From cappy2112 at gmail.com  Mon Jan 11 17:56:45 2016
From: cappy2112 at gmail.com (Tony Cappellini)
Date: Mon, 11 Jan 2016 14:56:45 -0800
Subject: [C++-sig] Passing memory allocated in C++ to Python
Message-ID: <CABRXM4k0pM2PUtcDkPGsDwf8Fw4_YJ0LFYOXQRFaM_YgQ6LwiQ@mail.gmail.com>

I've already got a project built for Linux, using Boost-python. My Python
code can import the C++ extension and access the members that are exposed.

My C++ code needs to call fill in C++ structures in order to call ioctl()
functions, while passing the addresses of the structures to the ioctl
functions.

I also need to allocate large (10's of MB - 100s of MB) buffers which must
be aligned to specific boundaries. A pointer to these buffers are part of
the above-mentioned C++ structs.
(I already have the C++ functions which handle the alignment and memory
allocation)

The problems I'm facing are:
     1. How to initiate the memory allocation from Python, so that the user
has access to the
         data after the ioctl() completes. This would eliminate the need
from copying the data from
        C++ to Python, after each ioctl() call.
     2. Using boost-python, which type of Python data structure is best
suited to be shared across
         the C++/Python boundary? Keep in mind that the allocation needs to
happen in C++ so that
         memory alignment can be maintained, but initiating the
allocation/alignment will be done from
         Python.

Essentially- Python needs access to the return value from a malloc() call.


A link link to a working example of Python & C++ code would be helpful.
.
The boost wiki only has examples for exposing classes to Python.
https://wiki.python.org/moin/boost.python/EmbeddingPython

The examples I've found on StackOverflow aren't quite what I'm looking for.


Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20160111/e7b51185/attachment.html>

From stefan at seefeld.name  Mon Jan 11 20:00:38 2016
From: stefan at seefeld.name (Stefan Seefeld)
Date: Mon, 11 Jan 2016 20:00:38 -0500
Subject: [C++-sig] Passing memory allocated in C++ to Python
In-Reply-To: <CABRXM4k0pM2PUtcDkPGsDwf8Fw4_YJ0LFYOXQRFaM_YgQ6LwiQ@mail.gmail.com>
References: <CABRXM4k0pM2PUtcDkPGsDwf8Fw4_YJ0LFYOXQRFaM_YgQ6LwiQ@mail.gmail.com>
Message-ID: <56945036.8080302@seefeld.name>

Hi Tony,

On 11.01.2016 17:56, Tony Cappellini wrote:
>
>
> I've already got a project built for Linux, using Boost-python. My
> Python code can import the C++ extension and access the members that
> are exposed.
>
> My C++ code needs to call fill in C++ structures in order to call
> ioctl() functions, while passing the addresses of the structures to
> the ioctl functions. 
>
> I also need to allocate large (10's of MB - 100s of MB) buffers which
> must be aligned to specific boundaries. A pointer to these buffers are
> part of the above-mentioned C++ structs.
> (I already have the C++ functions which handle the alignment and
> memory allocation)
>
> The problems I'm facing are:
>      1. How to initiate the memory allocation from Python, so that the
> user has access to the
>          data after the ioctl() completes. This would eliminate the
> need from copying the data from
>         C++ to Python, after each ioctl() call.
>      2. Using boost-python, which type of Python data structure is
> best suited to be shared across
>          the C++/Python boundary? Keep in mind that the allocation
> needs to happen in C++ so that 
>          memory alignment can be maintained, but initiating the
> allocation/alignment will be done from 
>          Python.

What you are describing all sounds quite feasible.

>
> Essentially- Python needs access to the return value from a malloc() call.

Really ? Why ? You say your C++ code needs to access the pointer(s).
Python has no notion of "pointer", but it will happily pass around
whatever data you expose from C++.

So let's say you have a C++ class (or struct if you prefer) that
encapsulates the data and methods, such as:

class something
{
  public:
    something() : buffer_(allocate_memory()) {}
    void call_ioctl() { /*...*/}
  private:
    char *buffer_;
};

if you expose the above class together with the constructor and the
'call_ioctl()' member function I think you have all you need.

If not, please elaborate.

        Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...


From roigcarlo at gmail.com  Mon Jan 11 12:09:28 2016
From: roigcarlo at gmail.com (Carlos Roig)
Date: Mon, 11 Jan 2016 18:09:28 +0100
Subject: [C++-sig] Problem with shared_ptr converter in boost 1.60
Message-ID: <CAMv+cnsHNpnc+z7SxiMwnX0dCzKNnEq9Aa1-VncR1wKzOmkuLw@mail.gmail.com>

Dear colleges,

I work in a project which makes use of the Boost.python module.

Las week we received feedback from some people that was having problems
after
updating boost from version 1.59 to 1.60.
Specifically the problem appears while trying to use a shared_ptr class.

Until now, we were registering our classes as follows:

class_<Foo, Foo::Pointer, bases<>, boost::noncopyable >("FooClass");

being Foo::Pointer defined as boost::shared_ptr<Foo>

Which automatically created the wrapper for python. Since version 1.60, the
following error appears while trying to access that elements:

>>> for Foo in Mesh.GetFooPointers()
>>>     print(Foo)

TypeError: No to_python (by-value) converter found for C++ type:
boost::shared_ptr<Foo>

We have been "googling" for a while and we found a work-around which
consist in adding:

register_ptr_to_python<Foo::Pointer>();

This would be pretty annoying to do for all our code, as is quite big and
was
designed under the assumption of that boost::shared_ptr<> was automatically
handled by boost.python and there was no need for calling
"register_ptr_to_python", as specified here:
https://wiki.python.org/moin/boost.python/PointersAndSmartPointers

We haven't been able to identify any change that may be affecting this, so
my question is:
Is there any way in which we can restore the old behaviour?
Does this change is inteended?

If it helps, we have perfom tests using clang 3.7 and gcc 5.3, both with
--std=c++11 enabled under linux64 (Kubuntu 15.10, Ubuntu 15.04 and 15.10,
and Arch) using python 3.4.3 with identical results..

Best regards and thank you in advance,

Charlie.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20160111/187bf53c/attachment.html>

From cappy2112 at gmail.com  Tue Jan 12 12:52:20 2016
From: cappy2112 at gmail.com (Tony Cappellini)
Date: Tue, 12 Jan 2016 09:52:20 -0800
Subject: [C++-sig] Passing memory allocated in C++ to Python
Message-ID: <CABRXM4mnre+XC=ERSCWX3SeySOMQcW5dBL2TkGP8JYMFz+r-AQ@mail.gmail.com>

Stefan,


To: cplusplus-sig at python.org
Subject: Re: [C++-sig] Passing memory allocated in C++ to Python
Message-ID: <56945036.8080302 at seefeld.name>
Content-Type: text/plain; charset=windows-1252

> Essentially- Python needs access to the return value from a malloc() call.

>>Really ? Why ? You say your C++ code needs to access the pointer(s).
>>Python has no notion of "pointer", but it will happily pass around
>>atever data you expose from C++.

Sorry about that Stefan, my post should have been clearer.

I know Python doesn't know about pointers.
I should have said "Python needs access to the memory pointed to by the
memory allocated with malloc()".


>>f you expose the above class together with the constructor and thei
>>'call_ioctl()' member function I think you have all you need.

In your something class, the data type returned from allocate_memory()
needs to be something that Python understands. Since that allocation
function (member) will be allocating 100s of MB of memory, how will this
memory map to a Python data type?

Is a bytearray better to use than a list (as far as performance is
concerned)? Is a list better to use than a string (as far as performance is
concerned). Are there other data types that should be considered?

The user will need to access that data as bytes, words, longs, and possibly
sequences.

I've tried using boost's extract, to convert the char * returned from
malloc() into a Python string,
and a Python list, but these result in compile errors.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20160112/2348b391/attachment.html>

From stefan at seefeld.name  Tue Jan 12 13:14:43 2016
From: stefan at seefeld.name (Stefan Seefeld)
Date: Tue, 12 Jan 2016 13:14:43 -0500
Subject: [C++-sig] Passing memory allocated in C++ to Python
In-Reply-To: <CABRXM4mnre+XC=ERSCWX3SeySOMQcW5dBL2TkGP8JYMFz+r-AQ@mail.gmail.com>
References: <CABRXM4mnre+XC=ERSCWX3SeySOMQcW5dBL2TkGP8JYMFz+r-AQ@mail.gmail.com>
Message-ID: <56954293.9030504@seefeld.name>

On 12.01.2016 12:52, Tony Cappellini wrote:
>
> Stefan,
>
>
> To: cplusplus-sig at python.org <mailto:cplusplus-sig at python.org>
> Subject: Re: [C++-sig] Passing memory allocated in C++ to Python
> Message-ID: <56945036.8080302 at seefeld.name
> <mailto:56945036.8080302 at seefeld.name>>
> Content-Type: text/plain; charset=windows-1252
>
> > Essentially- Python needs access to the return value from a malloc() call.
>
> >>Really ? Why ? You say your C++ code needs to access the pointer(s).
> >>Python has no notion of "pointer", but it will happily pass around
> >>atever data you expose from C++.
>
> Sorry about that Stefan, my post should have been clearer.
>
> I know Python doesn't know about pointers.
> I should have said "Python needs access to the memory pointed to by
> the memory allocated with malloc()".
>
>
> >>f you expose the above class together with the constructor and thei
> >>'call_ioctl()' member function I think you have all you need.
>
> In your something class, the data type returned from allocate_memory()
> needs to be something that Python understands. Since that allocation
> function (member) will be allocating 100s of MB of memory, how will
> this memory map to a Python data type?

Does it have to be a Python (native) data type ? Could you explain your
use-case a little more ?

>
> Is a bytearray better to use than a list (as far as performance is
> concerned)? Is a list better to use than a string (as far as
> performance is concerned). Are there other data types that should be
> considered?

It all depends on what you intend to do with the memory.

>
> The user will need to access that data as bytes, words, longs, and
> possibly sequences.
>
> I've tried using boost's extract, to convert the char * returned from
> malloc() into a Python string,
> and a Python list, but these result in compile errors.

That's because the extract<>() logic is meant to be used for the
inverse: to get access to C/C++ objects embedded into a Python object.
Once you reflect your C++ type "something" to Python, boost.python will
implicitly convert between the Python (wrapper) type and the C++
"something" type for all function calls.
However, sometimes you may still have a Python object, knowing that it
actually contains a "something". extract<> is meant to give access to
that (either by-value or by-reference, depending on your usage).

If you really need to expose the memory as a buffer to Python, you may
want to use one of the newer C APIs such as
|PyMemoryView_FromMemory
(https://docs.python.org/3/c-api/memoryview.html). Support for that
isn't built into boost.python yet, so you'll have to add some glue code
(e.g.
http://stackoverflow.com/questions/23064407/expose-c-buffer-as-python-3-bytes).
This may actually be worth including into the library. We just haven't
had anyone asking for such a feature - yet.
|
(I have used similar approaches in the past, where memory from a C++
library I wrote is bound to a NumPy array, so the data can be accessed
copy-free from other Python modules via the NumPy interface.)

Regards,
        Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...


From cappy2112 at gmail.com  Tue Jan 12 13:54:35 2016
From: cappy2112 at gmail.com (Tony Cappellini)
Date: Tue, 12 Jan 2016 10:54:35 -0800
Subject: [C++-sig] Passing memory allocated in C++ to Python
In-Reply-To: <56954293.9030504@seefeld.name>
References: <CABRXM4mnre+XC=ERSCWX3SeySOMQcW5dBL2TkGP8JYMFz+r-AQ@mail.gmail.com>
 <56954293.9030504@seefeld.name>
Message-ID: <CABRXM4=vjvZxpZ6YF+upr5fxMB3LH0LAhM4X-K6OPSdXOGyEVQ@mail.gmail.com>

> In your something class, the data type returned from allocate_memory()
> needs to be something that Python understands. Since that allocation
> function (member) will be allocating 100s of MB of memory, how will
> this memory map to a Python data type?

>>Does it have to be a Python (native) data type ? Could you explain your
>>use-case a little more ?

It needs to be a data type that can be allocated in C++ but accessible in
Python

>>Could you explain your use-case a little more ?
I have C++ code which is interfacing to a driver, from Python.
The user invokes a python function which calls my C++ code (via Boost). The
C++ code makes a C++ structure, allocates memory for the ioctl call,
puts a pointer to the allocated memory into the C++ structure (as well as
filling in other structure members), calls the ioctl and returns the
success/failure
results to Python. The user needs a way to get access to the allocated
memory, which was filled in by the ioctl call.

>>f you really need to expose the memory as a buffer to Python, you may want
to use one of the newer C APIs such as
I'm sure there are many ways to do this, but I've already chosen boost for
this project. Boost exists in order to
pass data back and forth between C++ & Python, there must be some way to do
this

The boost documentation is not very helpful, but I know many people have
shared data between Python & C++ for
a long time this

Thanks

On Tue, Jan 12, 2016 at 10:14 AM, Stefan Seefeld <stefan at seefeld.name>
wrote:

> On 12.01.2016 12:52, Tony Cappellini wrote:
> >
> > Stefan,
> >
> >
> > To: cplusplus-sig at python.org <mailto:cplusplus-sig at python.org>
> > Subject: Re: [C++-sig] Passing memory allocated in C++ to Python
> > Message-ID: <56945036.8080302 at seefeld.name
> > <mailto:56945036.8080302 at seefeld.name>>
> > Content-Type: text/plain; charset=windows-1252
> >
> > > Essentially- Python needs access to the return value from a malloc()
> call.
> >
> > >>Really ? Why ? You say your C++ code needs to access the pointer(s).
> > >>Python has no notion of "pointer", but it will happily pass around
> > >>atever data you expose from C++.
> >
> > Sorry about that Stefan, my post should have been clearer.
> >
> > I know Python doesn't know about pointers.
> > I should have said "Python needs access to the memory pointed to by
> > the memory allocated with malloc()".
> >
> >
> > >>f you expose the above class together with the constructor and thei
> > >>'call_ioctl()' member function I think you have all you need.
> >
> > In your something class, the data type returned from allocate_memory()
> > needs to be something that Python understands. Since that allocation
> > function (member) will be allocating 100s of MB of memory, how will
> > this memory map to a Python data type?
>
> Does it have to be a Python (native) data type ? Could you explain your
> use-case a little more ?
>
> >
> > Is a bytearray better to use than a list (as far as performance is
> > concerned)? Is a list better to use than a string (as far as
> > performance is concerned). Are there other data types that should be
> > considered?
>
> It all depends on what you intend to do with the memory.
>
> >
> > The user will need to access that data as bytes, words, longs, and
> > possibly sequences.
> >
> > I've tried using boost's extract, to convert the char * returned from
> > malloc() into a Python string,
> > and a Python list, but these result in compile errors.
>
> That's because the extract<>() logic is meant to be used for the
> inverse: to get access to C/C++ objects embedded into a Python object.
> Once you reflect your C++ type "something" to Python, boost.python will
> implicitly convert between the Python (wrapper) type and the C++
> "something" type for all function calls.
> However, sometimes you may still have a Python object, knowing that it
> actually contains a "something". extract<> is meant to give access to
> that (either by-value or by-reference, depending on your usage).
>
> If you really need to expose the memory as a buffer to Python, you may
> want to use one of the newer C APIs such as
> |PyMemoryView_FromMemory
> (https://docs.python.org/3/c-api/memoryview.html). Support for that
> isn't built into boost.python yet, so you'll have to add some glue code
> (e.g.
>
> http://stackoverflow.com/questions/23064407/expose-c-buffer-as-python-3-bytes
> ).
> This may actually be worth including into the library. We just haven't
> had anyone asking for such a feature - yet.
> |
> (I have used similar approaches in the past, where memory from a C++
> library I wrote is bound to a NumPy array, so the data can be accessed
> copy-free from other Python modules via the NumPy interface.)
>
> Regards,
>         Stefan
>
> --
>
>       ...ich hab' noch einen Koffer in Berlin...
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20160112/84117a10/attachment-0001.html>

From cappy2112 at gmail.com  Tue Jan 12 14:01:18 2016
From: cappy2112 at gmail.com (Tony Cappellini)
Date: Tue, 12 Jan 2016 11:01:18 -0800
Subject: [C++-sig] Passing memory allocated in C++ to Python
In-Reply-To: <56954293.9030504@seefeld.name>
References: <CABRXM4mnre+XC=ERSCWX3SeySOMQcW5dBL2TkGP8JYMFz+r-AQ@mail.gmail.com>
 <56954293.9030504@seefeld.name>
Message-ID: <CABRXM4k+asBVY8vVj4hi0cirDaxLZzg6mB02_y-V6S_3qtac+w@mail.gmail.com>

>>(https://docs.python.org/3/c-api/memoryview.html).
I should have mentioned that I am using Python 2.7 (as part of a group
project- others are using Python 2.7 as well).

Python 3.x is out of the question at the moment, but the memory view looks
like an interesting idea.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20160112/04fb4a64/attachment.html>

From stefan at seefeld.name  Tue Jan 12 14:06:03 2016
From: stefan at seefeld.name (Stefan Seefeld)
Date: Tue, 12 Jan 2016 14:06:03 -0500
Subject: [C++-sig] Passing memory allocated in C++ to Python
In-Reply-To: <CABRXM4=vjvZxpZ6YF+upr5fxMB3LH0LAhM4X-K6OPSdXOGyEVQ@mail.gmail.com>
References: <CABRXM4mnre+XC=ERSCWX3SeySOMQcW5dBL2TkGP8JYMFz+r-AQ@mail.gmail.com>
 <56954293.9030504@seefeld.name>
 <CABRXM4=vjvZxpZ6YF+upr5fxMB3LH0LAhM4X-K6OPSdXOGyEVQ@mail.gmail.com>
Message-ID: <56954E9B.9020803@seefeld.name>

On 12.01.2016 13:54, Tony Cappellini wrote:
>
> > In your something class, the data type returned from allocate_memory()
> > needs to be something that Python understands. Since that allocation
> > function (member) will be allocating 100s of MB of memory, how will
> > this memory map to a Python data type?
>
> >>Does it have to be a Python (native) data type ? Could you explain your
> >>use-case a little more ?
>
> It needs to be a data type that can be allocated in C++ but accessible
> in Python

"accessible" is too vague. The question is what interface you need,
whether an object-oriented one with methods to access and modify the
internal state, or one giving you unstructured access to the raw data.

>
> >>Could you explain your use-case a little more ?
> I have C++ code which is interfacing to a driver, from Python.
> The user invokes a python function which calls my C++ code (via
> Boost). The C++ code makes a C++ structure, allocates memory for the
> ioctl call,
> puts a pointer to the allocated memory into the C++ structure (as well
> as filling in other structure members), calls the ioctl and returns
> the success/failure
> results to Python. The user needs a way to get access to the allocated
> memory, which was filled in by the ioctl call.

Can you share your code, or at least a stripped-down version ? I think
at this point it would make more sense to go over some specifics.
Your description above doesn't sound very object-oriented. You are
calling a function that returns the success of the operation. What you
actually want is return an object that holds the state (including the
memory you are talking about), which you can then manipulate in
subsequent calls.

Even some pseudo-code would do, to discuss the general idea.

> I should have mentioned that I am using Python 2.7 (as part of a group
> project- others are using Python 2.7 as well).
>
> Python 3.x is out of the question at the moment, but the memory view
> looks like an interesting idea.

Fine, I just mentioned it to illustrate the idea. There are other APIs
to achieve the same.

Regards,
    Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...


From herron at ELLINGTON.com  Fri Jan 29 13:15:58 2016
From: herron at ELLINGTON.com (Liam Herron)
Date: Fri, 29 Jan 2016 18:15:58 +0000
Subject: [C++-sig] passing NoneType as argument to constructor
Message-ID: <96426055F003C542B3FB8A0928736DF9023A647437@ex0.ELLINGTON.com>

For the following code:
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------
#include <boost/python.hpp>
using namespace boost::python;

namespace // unnamed
{

class NullableDouble
{
public:
   NullableDouble()
      : isNull_(true)
      , value_(0.0)
   {
   }

   NullableDouble(double x)
      : isNull_(false)
      , value_(x)
   {
   }

   double value() const
   {
      return value_;  // null check not relevant to this example
   }

   bool isNull() const
   {
      return isNull_;
   }

   // ... more functions but not needed for this example

private:
   bool isNull_;
   double value_;
};

} // end namespace unnamed

void init_pnic()
{
   class_<NullableDouble>
   ("NullableDouble", "NullableDouble", init<>())
      .def(init<double>())
      .def("value", &NullableDouble::value)
      .def("isNull", &NullableDouble::isNull)
   ;
}

BOOST_PYTHON_MODULE(_testPNIC)
{
   init_pnic();
}
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------

I would like to have another constructor method in python that takes a 'None' and returns the same as the no arg constructor 'NullableDouble()'

What is the best way to do this?

Thanks,
--Liam




=============================================================================================
Email transmissions can not be guaranteed to be secure or error-free, as information 
could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain 
viruses.  The sender therefore does not accept liability for any errors or omissions in 
the contents of this message which arise as a result of email transmission.  In addition,
the information contained in this email message is intended only for use of the 
individual or entity named above.  If the reader of this message is not the intended
recipient, or the employee or agent responsible to deliver it to the intended recipient,
you are hereby notified that any dissemination, distribution,or copying of this communication,
disclosure of the parties to it, or any action taken or omitted to be taken in reliance on it,
is strictly prohibited, and may be unlawful.  If you are not the intended recipient please
delete this email message. 
==============================================================================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20160129/7e9a5f6f/attachment.html>

From stefan at seefeld.name  Fri Jan 29 13:37:33 2016
From: stefan at seefeld.name (Stefan Seefeld)
Date: Fri, 29 Jan 2016 13:37:33 -0500
Subject: [C++-sig] passing NoneType as argument to constructor
In-Reply-To: <96426055F003C542B3FB8A0928736DF9023A647437@ex0.ELLINGTON.com>
References: <96426055F003C542B3FB8A0928736DF9023A647437@ex0.ELLINGTON.com>
Message-ID: <56ABB16D.3000604@seefeld.name>

On 29.01.2016 13:15, Liam Herron wrote:
>
> For the following code:
>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> #include <boost/python.hpp>
>
> using namespace boost::python;
>
>  
>
> namespace // unnamed
>
> {
>
>  
>
> class NullableDouble
>
> {
>
> public:
>
>    NullableDouble()
>
>       : isNull_(true)
>
>       , value_(0.0)
>
>    {
>
>    }
>
>  
>
>    NullableDouble(double x)
>
>       : isNull_(false)
>
>       , value_(x)
>
>    {
>
>    }
>
>  
>
>    double value() const
>
>    {
>
>       return value_;  // null check not relevant to this example
>
>    }
>
>  
>
>    bool isNull() const
>
>    {
>
>       return isNull_;
>
>    }
>
>  
>
>    // ... more functions but not needed for this example
>
>  
>
> private:
>
>    bool isNull_;
>
>    double value_;
>
> };
>
>  
>
> } // end namespace unnamed
>
>  
>
> void init_pnic()
>
> {
>
>    class_<NullableDouble>
>
>    ("NullableDouble", "NullableDouble", init<>())
>
>       .def(init<double>())
>
>       .def("value", &NullableDouble::value)
>
>       .def("isNull", &NullableDouble::isNull)
>
>    ;
>
> }
>
>  
>
> BOOST_PYTHON_MODULE(_testPNIC)
>
> {
>
>    init_pnic();
>
> }
>
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>  
>
> I would like to have another constructor method in python that takes a
> ?None? and returns the same as the no arg constructor ?NullableDouble()?
>

Just create a 'factory' function that takes a Python object (and which
you then check for None) to instantiate and return a "null". Add that
function to your wrapper like any other method (using '__init__' and
'make_constructor').

HTH,
        Stefan




>  
>
> What is the best way to do this?
>
>  
>
> Thanks,
>
> --Liam
>
>  
>
>  
>

-- 

      ...ich hab' noch einen Koffer in Berlin...


From camior at gmail.com  Fri Jan 29 19:19:37 2016
From: camior at gmail.com (David Sankel)
Date: Fri, 29 Jan 2016 17:19:37 -0700
Subject: [C++-sig] Multiple registration safety and unit testing
Message-ID: <CAFYdOO+0rK+tKJyRjs3tW32CdrTc-uGEezrnU2N54_r6RyrFNg@mail.gmail.com>

Hello all,

I'd like to allow a class to be registered more than once and wanted to get
your opinion if this change would be a good idea.

'boost::python::class_' always adds a class to the current scope and
registers it in the global registry. The global registry code (particularly
the 'insert' function in 'src/converter/registry.cpp') asserts that the
class hasn't been registered already. This, of course, prevents multiple
registrations of the same type.

There is a use case, though, where multiple registrations of the same type
is both necessary and safe. Generally, a c++ component which has a 'class_'
call may not know the name of the module it will eventually end up in. It
would still be desirable to write a unit test for this component. If we
write a unit test and put the 'class_' into some dummy module we run into a
problem where other unit tests might call 'class_' in another scope.

The safety of multiple 'class_' calls stems from always using the same
conversion function. Semantically everything is a-okay.

I'm proposing to change the precondition of the
'boost::python::converter::registry::insert' function from:

The behavior is undefined unless the specified 'type_info' object has not
already been registered.


to:

The behavior is undefined unless the specified 'type_info' object has not
already been registered with a semantically different converter

Any objections?

-- David Sankel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20160129/f6e8d453/attachment-0001.html>

From stefan at seefeld.name  Sat Jan 30 09:30:06 2016
From: stefan at seefeld.name (Stefan Seefeld)
Date: Sat, 30 Jan 2016 09:30:06 -0500
Subject: [C++-sig] Multiple registration safety and unit testing
In-Reply-To: <CAFYdOO+0rK+tKJyRjs3tW32CdrTc-uGEezrnU2N54_r6RyrFNg@mail.gmail.com>
References: <CAFYdOO+0rK+tKJyRjs3tW32CdrTc-uGEezrnU2N54_r6RyrFNg@mail.gmail.com>
Message-ID: <56ACC8EE.30006@seefeld.name>

On 29.01.2016 19:19, David Sankel wrote:
> Hello all,
>
> I'd like to allow a class to be registered more than once and wanted
> to get your opinion if this change would be a good idea.
>
> 'boost::python::class_' always adds a class to the current scope and
> registers it in the global registry. The global registry code
> (particularly the 'insert' function in 'src/converter/registry.cpp')
> asserts that the class hasn't been registered already. This, of
> course, prevents multiple registrations of the same type.
>
> There is a use case, though, where multiple registrations of the same
> type is both necessary and safe. Generally, a c++ component which has
> a 'class_' call may not know the name of the module it will eventually
> end up in. It would still be desirable to write a unit test for this
> component. If we write a unit test and put the 'class_' into some
> dummy module we run into a problem where other unit tests might call
> 'class_' in another scope.
>
> The safety of multiple 'class_' calls stems from always using the same
> conversion function. Semantically everything is a-okay.
>
> I'm proposing to change the precondition of the
> 'boost::python::converter::registry::insert' function from:
>
>     The behavior is undefined unless the specified 'type_info' object
>     has not already been registered.
>
>
> to:
>
>     The behavior is undefined unless the specified 'type_info' object
>     has not already been registered with a semantically different
>     converter
>
> Any objections?

Yes. Can you describe your use-case ? And what do you mean exactly by
"semantically different" ? :-)

Thanks,
        Stefan


-- 

      ...ich hab' noch einen Koffer in Berlin...