Issue with ctypes and callback functions

[Bill Somerville]

-----Original Message-----
From: Bill Somerville 
Sent: 04 August 2016 18:23
To: 'eryk sun' <eryksun at gmail.com>; python-list at python.org
Subject: RE: Issue with ctypes and callback functions

-----Original Message-----
From: eryk sun [mailto:eryksun at gmail.com] 

from_param is a hook method for a type that's set in a function pointer's argtypes. It gets called to convert an argument when the function pointer is called from Python. The return value can be a ctypes instance, an object with a hard-coded conversion (e.g. a string or integer), or an object that defines an _as_parameter_ attribute.

Only ctypes types are supported in callbacks, which unfortunately isn't documented clearly. Specifically, the class dict needs to be an extended C storage dict (i.e. StgDictObject), either to look up the getfunc of a simple type or to ensure that instantiating a non-simple type returns a ctypes instance (i.e. CDataObject) with a known size.
The relevant code in _CallPythonObject is as follows (when stripped of declarations and error handling):

    cnv = PySequence_GetItem(converters, i);
    dict = PyType_stgdict(cnv);
    if (dict && dict->getfunc && !_ctypes_simple_instance(cnv)) {
        v = dict->getfunc(*pArgs, dict->size);
        PyTuple_SET_ITEM(arglist, i, v);
    } else if (dict) {
        obj = (CDataObject *)PyObject_CallFunctionObjArgs(cnv, NULL);
        memcpy(obj->b_ptr, *pArgs, dict->size);
        PyTuple_SET_ITEM(arglist, i, (PyObject *)obj);
    } else {
        PyErr_SetString(PyExc_TypeError,
                        "cannot build parameter");

I don't have much experience with SWIG. Does it provide some means to instantiate a wrapped type from an address? If it does, then you can use a void pointer as the callback parameter.

Hi Eryk,

Thanks for looking at this. I had got as far as looking at the code above and guessed that only ctypes types were supported. This does seem like a bit of a functionality gap that cannot be solved easily by users of ctypes. It would appear to be fairly easy to define a protocol for objects that can be instantiated from ctypes arguments before passing to Python callables, i.e. some requirement for a class method factory that takes a ctypes type as a parameter and returns a Python object. This would seem to be a natural complement to from_param()/_as_parameter_ for the C/C++ to Python direction of function calls.

I had started with an implementation that converted a c_void_p value and instantiated the required SWIG Python wrapper. This is not too difficult as SWIG allows its types to be extended in both the C/C++ world and the Python world. The result is only a one liner for each callback parameter but it is still ugly as the writer of the callback has to know what type the argument is and remember to call the "constructor" before accessing the attributes of the object.

My requirement is to wrap an API for testers and customers to write simple test cases and any ugliness in the test cases is undesirable given that the users will probably not be Python experts and maybe not even developers.

Regards
Bill.