return a ctypes object to C

[Arnaud Loonstra]

On 31-10-2019 15:39, Arnaud Loonstra wrote:
> On 31-10-2019 14:44, Thomas Jollans wrote:
>> On 31/10/2019 14.13, Arnaud Loonstra wrote:
>>> On 30-10-2019 09:32, Arnaud Loonstra wrote:
>>>> Hi all,
>>>>
>>>> I'm trying to wrap my head around the ctypes API. I have a C
>>>> structure I wish to create in Python and then return from python to C.
>>>>
>>>> So a python method is called from C and needs to return an object
>>>> which we then process in C again.
>>>>
>>>> I have a binding to access and create the C methods and structures so
>>>> in Python I can call the Zmsg() constructor. I now need to return this.
>>>>
>>>> My python test method is simply:
>>>>
>>>> def actor_test( *args, **kwargs):
>>>>       print("test")
>>>>       msg = Zmsg()
>>>>       frame = Zframe(b"Hello", 5)
>>>>       msg.prepend(frame)
>>>>       return msg
>>>>
>>>> the method is called from C as follows:
>>>>
>>>> PyObject *pReturn = PyObject_CallObject(pFunc, NULL);
>>>>
>>>> This correctly calls the method. However the returned object is of
>>>> course a PyObject*. The debugger says it's
>>>>
>>>> "<czmq._czmq_ctypes.Zmsg object at 0x7ffff5f18e50>"    PyObject
>>>>               [class]    "<class 'czmq._czmq_ctypes.Zmsg'>"
>>>>               [super class]    "<class 'object'>"
>>>>               [meta type]    "<class 'type'>"
>>>>               ob_refcnt    1    Py_ssize_t
>>>>
>>>> However how I can I get it back to the original C type (zmsg_t *)
>>>>
>>>> Any help really appreciated.
>>>>
>>>
>>> What I've found so far is that I can return the address of the ctypes
>>> object.
>>>
>>> msg = Zmsg()
>>> frame = Zframe(b"Hello", 5)
>>> msg.prepend(frame)
>>> return addressof(msg._as_parameter_.contents)
>>>
>>> In C I can then cast it back to the original type.
>>>
>>> PyObject *pReturn = PyObject_CallObject(pFunc, NULL);
>>> assert(pReturn);
>>> long bla = PyLong_AsLong(pReturn);
>>> zmsg_t* test = (zmsg_t *)bla;
>>> assert(test);
>>> char *hello = zmsg_popstr(test);
>>> assert(hello);
>>> assert(streq(hello, "Hello"));
>>>
>>> This works, I'm not sure if this is the right way. It also creates a
>>> complicated setup with the garbage collector.
>>>
>>> Anybody better ideas?
>>
>> You've already got a complicated setup with your ctypes objects...
>>
>> If you're using the Python C API anyway, why would you use ctypes at
>> all? You can create custom Python types in C to wrap your C pointers.
>>
>> Alternatively: if you're using ctypes anyway, why use the Python C API
>> at all? You can create C function pointers from Python functions with
>> ctypes.
>>
>> If you're mixing two different ways of interfacing Python and C, the
>> result will ALWAYS be messy. Better to stick to one. Personally, I
>> prefer cffi or cython depending on the situation, as I find them clearer
>> and easier to use than ctypes. Using the Python C API directly is
>> usually the hardest to understand and the easiest to get wrong.
>>
>> -- Thomas
> 
> Hi Thomas,
> 
> I have an engine running which can call handlers. These handlers return 
> a zmsg_t (a message) which the engine then can process.
> 
> In this engine we have Python embedded and I want to use a Python method 
> as a handler. To embed Python we need to use the Python C API. To 
> construct a zmsg_t type in Python we need to call the corresponding C 
> method and we use ctypes to do that.
> 
> I'm using a ctypes binding because it already exists, it's here: 
> https://github.com/zeromq/czmq/blob/d6283985ba52fd8c3f8fbdc7cd5c08372ff69ca1/bindings/python/czmq/_czmq_ctypes.py#L4392 
> 
> 
> I know I can use cffi for example but that's IMHO just a ctypes 
> alternative.
> 
> Are you saying it's better to create some approach to create a zmsg_t 
> using the Python C API?
> 
> Please enlighten me if this should be done differently.
> 
> Rg,
> 
> Arnaud

Ok, you're reply triggered me to do some more reading into the C-API. 
I've found I can do it all using the C-API. Especially this guide was 
very helpful for my situation:

https://docs.python.org/3/extending/newtypes_tutorial.html

Thnx,

Arnaud