[SciPy-dev] Status of weave

[C. Ball]

Travis E. Oliphant <oliphant <at> enthought.com> writes:
[...]
> This is not quite accurate.  Eric has spent time recently, in fact on 
> updating and fixing the weave type converters.   I don't know of plans 
> to support 'O' types, but I don't believe it would be difficult with 
> what I know about it.
[...]
> But, absolutely, you have to either convince someone to work on it, or 
> dig in and do it yourself.  


Ok, we (at topographica.org) had a go at adding a converter for 'O'
types. In fact, we did almost nothing to the weave code; I have
included a diff against r3652 [1].

These changes allowed us to run the previous example I posted, but
then we went on to run something a little more realistic [2].

There seems to be a bizarre problem with the indexing. In
test_loop_objs(), "j-1" appears rather than "j" because it was the
only way to get the correct elements. Using "j", the numbers returned
were all for the next element (rather than the expected one), with
garbage or a segmentation fault at the end. Somehow it seems that the
starting value is off by maybe the size of one PyObject.

I've also included a version of test_loop_objs() that does not use a
macro to access the elements; we have the same problem with this
version. (The macros are described in the Guide to NumPy, page 336.)

There might be a stupid mistake in the example, or we might have
missed something, so I'd be really grateful if someone could take a
look.  For one thing, the Guide to NumPy says (also on page 336):
"ensure the array is aligned and in correct byte-swap order in order
to get useful results". I don't understand what that means, so that
could be the problem.



Thanks very much,

Chris



[1] Changes to weave:

--- lib/python2.5/site-packages/weave/c_spec.py	
+++ lib/python2.5/site-packages/weave/c_spec.py	
@@ -314,6 +314,8 @@ num_to_c_types['?'] = 'bool'
 num_to_c_types['l'] = 'long'
 num_to_c_types['L'] = 'npy_ulong'
 
+num_to_c_types['O'] = 'py::object'
+
 num_to_c_types['q'] = 'npy_longlong'
 num_to_c_types['Q'] = 'npy_ulonglong'
 
--- lib/python2.5/site-packages/weave/standard_array_spec.py
+++ lib/python2.5/site-packages/weave/standard_array_spec.py
@@ -20,6 +20,7 @@ num_typecode['g'] = 'PyArray_LONGDOUBLE'
 num_typecode['F'] = 'PyArray_CFLOAT'
 num_typecode['D'] = 'PyArray_CDOUBLE'
 num_typecode['G'] = 'PyArray_CLONGDOUBLE'
+num_typecode['O'] = 'PyArray_OBJECT'
 
 type_check_code = \
 """




[2] Example code:

import weave

# Version using floats
def test_loop_floats(cfs):

    code = """
        for (int i=0; i<Ncfs[0]; ++i) {
            for (int j=0; j<Ncfs[1]; ++j) {
                printf("(%d,%d) %f\\n",i,j,CFS2(i,j));
            }
        }
    """
    weave.inline(code,['cfs'],local_dict=locals(),verbose=1)


# Version using objects
def test_loop_objs(cfs):

    code = """
        for (int i=0; i<Ncfs[0]; ++i) {
            for (int j=0; j<Ncfs[1]; ++j) {
                // ** Why does it need j-1? **
                PyObject *cf = CFS2(i,j-1);
                PyObject *num_obj = PyObject_GetAttrString(cf,"num");
                double num_ = PyFloat_AsDouble(num_obj);
                
                printf("(%d,%d) %f\\n",i,j,num_);
            }
        }
    """
    weave.inline(code,['cfs'],local_dict=locals(),verbose=1)



from numpy import array,zeros

class CF(object):
    def __init__(self,num=0.0):
        self.num = num
        

array_of_floats = array([[0.0,0.1,0.2],
                          [1.0,1.1,1.2]],
                         dtype=float)

array_of_objects = array([[CF(0.0),CF(0.1),CF(0.2)],
                          [CF(1.0),CF(1.1),CF(1.2)]],
                         dtype=object)


print "floats"
test_loop_floats(array_of_floats)
print "objs"
test_loop_objs(array_of_objects)





[3] test_loop_objs() without the CFS2 macro:

def test_loop_objs(cfs):

    code = """
       PyObject *cfs_ = (PyObject *)(((PyArrayObject*)cfs)->data);
       for (int i=0; i<Ncfs[0]; ++i) {
           for (int j=0; j<Ncfs[1]; ++j) {
               PyObject *cf = cfs_++;
               PyObject *num_obj = PyObject_GetAttrString(cf,"num");
               double num_ = PyFloat_AsDouble(num_obj);

               printf("(%d,%d) %f\\n",i,j,num_);
           }
       }
    """
    weave.inline(code,['cfs'],local_dict=locals(),verbose=1)