[Numpy-discussion] Problem accessing elements of an array of dtype="O" from C

[C. Ball]

Travis E. Oliphant <oliphant <at> enthought.com> writes:
[...]
> The problem here is that iter->dataptr should be re-cast to a PyObject 
> ** because what is contained at the memory location is a *pointer* to 
> the PyObject.   Thus, you have to de-reference iter->dataptr to get the 
> PyObject * that you want:

Thanks for pointing out that mistake, Travis. You were correct, and
the code now works - great!

...Well, it works when called from Instant, but we still have a
problem using it with Weave. As I've mentioned before, we tried to
adapt Weave to work with arrays of dtype="O", but something is still
not working.

The small pieces of C below show code that works when called from
Instant, and almost identical C code that does not work when called
from Weave. Presumably, Weave still has a problem with converting
arrays of dtype="O".


------------------------------------------------------------

Working C code (used with Instant; as posted before, but with Travis's
correction):

"""
void loop(PyObject* a_){

 PyArrayIterObject *iter;
 iter = (PyArrayIterObject *)PyArray_IterNew(a_);

 while (iter->index < iter->size) {
     PyObject **cf = (PyObject **)(iter->dataptr);
     PyObject *num_obj = PyObject_GetAttrString(*cf,"num");

     double num_ = PyFloat_AsDouble(num_obj);
     printf("%f\\n",num_);

     PyArray_ITER_NEXT(iter);
 }
return;
}
"""

C code used with Weave (Weave patched as given in
http://article.gmane.org/gmane.comp.python.scientific.devel/7264):

code="""
   // get "cannot convert 'py::object*' to 'PyObject*' 
   // in argument passing" without this...
   PyObject *a_ = (PyObject *)a;

   PyArrayIterObject *iter;
   iter = (PyArrayIterObject *)PyArray_IterNew(a_);

   while (iter->index < iter->size) {
       PyObject **cf = (PyObject **)(iter->dataptr);
       PyObject *num_obj = PyObject_GetAttrString(*cf,"num");

       double num_ = PyFloat_AsDouble(num_obj);
       printf("%f\\n",num_);

       PyArray_ITER_NEXT(iter);
   }
"""
weave.inline(code,['a'],local_dict=locals(),verbose=1)


Python array of objects:

class CF(object):
    def __init__(self,num=0.0):
        self.num = num

objs = numpy.array([[CF(0.0),CF(0.1),CF(0.2)],
  		    [CF(1.0),CF(1.1),CF(1.2)]],dtype=object)


# Pass objs to the two functions above. The first version, used with
# Instant, works fine. The second version, used with Weave, causes
# a segmentation fault.

------------------------------------------------------------



We already have a lot of code written for Weave, and it seems to allow
simpler inlining of C code than does Instant (though I might be wrong),
so we'd really like to be able to continue to use Weave.

I'll file a bug report for Weave, but I wanted to point out the problem 
on the numpy list in case anyone here has an idea (since Weave was part
of numpy for a long time).


Thanks for your help,
Chris