[Cython] Upcoming issues with NumPy deprecated APIs and Cython's sizeof checks

[mark florisson]

On 30 January 2012 21:03, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> I'm testing my code with numpy-dev. They are trying to discourage use
> of deprecated APIs, this includes direct access to the ndarray struct.
> In order to update your code, you have to pass -DNPY_NO_DEPRECATED_API
> to the C compiler (or #define it before including NumPy headers).
>
> However, they have implemented this feature by exposing the ndarray
> type with just the Python object header:
> https://github.com/numpy/numpy/blob/master/numpy/core/include/numpy/ndarraytypes.h#L695
>
> Obviously, this interact bad with Cython's sizeof check, I'm getting
> this runtime warning:
>
> build/lib.linux-x86_64-2.7/petsc4py/lib/__init__.py:64:
> RuntimeWarning: numpy.ndarray size changed, may indicate binary
> incompatibility
>
> I think there is nothing Cython can do about this (other than
> special-casing NumPy to disable this VERY useful warning).

Weird, shouldn't you be getting an error? Because the size of the
PyArrayObject should be less than what Cython expects.

>  I've tried the patch below with success, but I'm not convinced...
> Does any of you have a suggestion for NumPy folks about how to improve
> this?
>

I'm not sure this should be fixed in NumPy. Their entire point is that
people shouldn't use those attributes directly. I think numpy.pxd
should be fixed, but the problem is that some attributes might be used
in user code (especially shape), and we still want that to work in
nogil mode. As such, I'm not sure what the best way of fixing it is,
without special casing these attributes in the compiler directly.
Maybe Dag will have some thoughts about this.

> diff --git a/numpy/core/include/numpy/ndarraytypes.h
> b/numpy/core/include/numpy/ndarraytypes.h
> index 0288272..1fcbf52 100644
> --- a/numpy/core/include/numpy/ndarraytypes.h
> +++ b/numpy/core/include/numpy/ndarraytypes.h
> @@ -695,6 +695,7 @@ typedef struct tagPyArrayObject_fields {
>  #ifdef NPY_NO_DEPRECATED_API
>  typedef struct tagPyArrayObject {
>         PyObject_HEAD
> +        char _npy_array_fields[sizeof(PyArrayObject_fields)-sizeof(PyObject)];
>  } PyArrayObject;
>  #else
>  /*
>
> --
> Lisandro Dalcin
> ---------------
> CIMEC (INTEC/CONICET-UNL)
> Predio CONICET-Santa Fe
> Colectora RN 168 Km 472, Paraje El Pozo
> 3000 Santa Fe, Argentina
> Tel: +54-342-4511594 (ext 1011)
> Tel/Fax: +54-342-4511169
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel