[Numpy-discussion] PyArray_SimpleNewFromData segfaults

[Ondrej Certik]

Hi,

I am getting a segfault in PyArray_SimpleNewFromData in Cython. I am
trying to debug it for the last 4 hours, but still absolutely no clue,
so I am posting it here, maybe someone knows where the problem is:

cdef ndarray array_double_c2numpy(double *A, int len):
    from numpy import empty
    print "got len:", len
    cdef npy_intp dims[10]
    cdef double X[500]
    print "1"
    dims[0] = 3
    print "2x"
    print dims[0], len
    print X[0], X[1], X[2]
    cdef npy_intp size
    cdef ndarray newarr
    cdef double *arrsource

    size = 10
    arrsource = <double *>malloc(sizeof(double) * size)
    print "still alive"
    newarr = PyArray_SimpleNewFromData(1, &size, 12,
                <void *>arrsource)
    print "I am already dead. :("
    print "3"
    return empty([len])



Essential is just the line:

    newarr = PyArray_SimpleNewFromData(1, &size, 12,
                <void *>arrsource)

Then I removed all numpy from my computer, downloaded the latest git
repository from:

http://projects.scipy.org/git/numpy.git

applied the following patch:

diff --git a/numpy/core/src/multiarray/ctors.c b/numpy/core/src/multiarray/ctors
index 3fdded0..777563c 100644
--- a/numpy/core/src/multiarray/ctors.c
+++ b/numpy/core/src/multiarray/ctors.c
@@ -1318,6 +1318,7 @@ PyArray_NewFromDescr(PyTypeObject *subtype, PyArray_Descr
                      intp *dims, intp *strides, void *data,
                      int flags, PyObject *obj)
 {
+    printf("entering PyArray_NewFromDescr\n");
     PyArrayObject *self;
     int i;
     size_t sd;
@@ -1553,6 +1554,7 @@ PyArray_New(PyTypeObject *subtype, int nd, intp *dims, int
 {
     PyArray_Descr *descr;
     PyObject *new;
+    printf("entering PyArray_New, still kicking\n");

     descr = PyArray_DescrFromType(type_num);
     if (descr == NULL) {



then installed with:

python setup.py install --home=~/usr

and run my cython program. Here is the output:

$ ./schroedinger

-------------------------------------------
   This is Hermes1D - a free ODE solver
 based on the hp-FEM and Newton's method,
   developed by the hp-FEM group at UNR
  and distributed under the BSD license.
 For more details visit http://hpfem.org/.
-------------------------------------------
Importing hermes1d
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_New, still kicking
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_New, still kicking
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
entering PyArray_NewFromDescr
Python initialized
got len: 39601
1
2x
3 39601
0.0 0.0 0.0
still alive
Segmentation fault



What puzzles me is that there is no debugging print statement just
before the segfault. So like if the PyArray_New was not being called.
But looking into numpy/core/include/numpy/ndarrayobject.h, line 1359:

#define PyArray_SimpleNewFromData(nd, dims, typenum, data)                    \
        PyArray_New(&PyArray_Type, nd, dims, typenum, NULL,                   \
                    data, 0, NPY_CARRAY, NULL)


It should be called. Does it segfault in the printf() statement above?
Hm. I also tried gdb, but it doesn't step into
PyArray_SimpleNewFromData  (in the C file), not sure why.

So both print statements and gdb failed to bring me to the cause,
pretty sad day for me. I am going home now and start with a fresh
head, it just can't segfault like this... I guess I'll start by
creating a simple cython project to reproduce it (the schroedinger
code above is quite involved, it starts a python interpreter inside a
C++ program, etc. etc.).


Ondrej