[Numpy-discussion] @Dag re numpy.pxd

[Dag Sverre Seljebotn]

On 02/11/2012 10:27 PM, mark florisson wrote:
> On 11 February 2012 20:31, Charles R Harris<charlesr.harris at gmail.com>  wrote:
>> Hi Dag,
>>
>> This probably needs to be on the cython mailing list at some point, but I
>> thought I'd start the discussion here. Numpy is going to begin deprecating
>> direct access to ndarray/dtype internals, ala arr->data etc. There are
>> currently macros/functions for many of these operations in the numpy
>> development branch and I expect more to go in over the coming year. Also,
>> some of the macros have been renamed. I don't know the best way for Cython
>> to support this, but the current version (0.15 here) generates code that
>> will fail if the deprecated things are excluded. Ideally, numpy.pxd would
>> have numpy version dependent parts but I don't know if that is possible. In
>> any case, I'd like your thoughts on the best way to coordinate this
>> migration with Cython.
>>
>> Chuck
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>
> This was discussed not too long ago on the cython-devel mailing list:
> http://mail.python.org/pipermail/cython-devel/2012-January/001848.html
>
> I personally think it'd be nice to not break existing Cython code, by
> e.g. writing nogil cdef properties (something which doesn't currently
> exist). That way the properties could use the non-deprecated way to
> actually access the data from numpy. (In any case the deprecated numpy
> functionality should go through a deprecation process before being
> removed).

The only attribute that really concerns me is shape.

For the rest, I think it's OK to require using the C API. E.g., the data 
is a Python attribute returning something else, and exposing the C field 
was a mistake in the first place.

If we remove the shape field, it would still work (through the Python 
getattr), but this might in some locations give a speed regresssion 
(and, in many places, not).

If we do anything about this, then I really think we should emulate the 
Python API in full, so that one could finally do "print a.shape", 
"len(a.shape)"; even if "a.shape[0]" is fast.

This requires something else that just cdef properties though -- perhaps 
a typed tuple type in addition.

> Alternatively, as Dag mentioned in the cython-devel thread, we could
> just deprecate the fields in Cython as well and place the burden on
> the user (and possibly issue warnings for their use).

Something this list may not be aware of is that Cython 0.16 will support 
a different mechanism for fast array access, implemented by Mark F.:

cdef double[:, :] a = np.zeros((3, 3))

In this case, "a" is NOT a NumPy array but is coerced to a "typed memory 
view", where Cython lays down semantics for the shape attribute etc.

So I think the recommended approach for old code will be to move to this 
syntax. This makes it less important to cook up fast NumPy-specific 
.shape access; it could just revert to using the Python .shape 
attribute, and then if there's a speed regression one could port the 
code to the new memoryviews...

Dag Sverre