returning a python array from a C extension

[Scott Gilbert]

Maxwell Todd Sayles <sayles at cpsc.ucalgary.ca> wrote: 
> in python i can write:
> 
> from array import *
> a = array ('B', 'some test data')
> 
> and a refers to an array object.
> 
> now i have a C extension for python that returns a PyString, and within
> python i can convert it to an array
> 
> e.g.:
> 
> from array import *
> from MyCExtension import *
> a = array ('B', cfunction())
> 
> but this method requires an extra copy and construction from the string
> to the array.
> 
> is there a way i can have cfunction return a python array?  i tried
> looking through the include files with the Python/C API and couldn't
> find anything.  any help would be appreciated.
> 

Yes, but since arraymodule.c doesn't have a C level API, you have to
use C to do what you would do from Python.  I haven't compiled this,
and I'm being a little sloppy so you should read the docs, but it
would be something like:

  /* declarations */
  PyObject* globals;
  PyObject* ignore;
  PyObject* array;
  char* pointer;
  int length;

  /* setup */
  globals = PyDict_New();

  /* import array */
  ignore = PyRun_String(
      "import array",
      Py_file_input, globals, globals
  );
  Py_XDECREF(ignore);

  /* create a 1234 byte array object */
  array = PyRun_String(
      "array.array('B', 'X')*1234", 
      Py_eval_input, globals, globals
  );

  /* use the PyBufferProcs to get a pointer to your n bytes */
  PyObject_AsWriteBuffer(array, &pointer, &length);

  /* build your data in your new pointer (no copy required) */
  memset(pointer, 'Z', length);

  /* cleanup */
  Py_XDECREF(globals);

  /* return put it some where... */
  return array;


Also note that if your array is really big, it would be faster to
multiply in parts:

  /* create a 16 meg array */
  array = PyRun_String(
      "array.array('B', 'X')*1024*1024*16",
      Py_eval_input, globals, globals
  );

Finally, it would probably be better to get the globals dict from a
real module - using the PyModule_GetDict(...) API instead of
PyDict_New().  I think the above will work today, but I suspect
someday that might not be the case...


Cheers,
    -Scott