[Numpy-discussion] Numpy 1.7b1 API change cause big trouble

[Frédéric Bastien]

Hi,

On Thu, Sep 6, 2012 at 11:32 AM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
>
> On Thu, Sep 6, 2012 at 10:07 AM, Frédéric Bastien <nouiz at nouiz.org> wrote:
>>
>> Hi,
>>
>> I reply with more information probably later today or tomorrow, but I
>> think i need to finish everything to give you the exact information.
>>
>> Part of the problem I had was that by default there is a warning that
>> is generated. It tell that to remove this warning we need to set
>> NPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION.
>
>
> You don't want to define this macro if you need to directly access the
> fields. What warning are you getting if you don't define it? Are you using
> Cython?

If I don't define it and I remove the -Werror, I got 3 errors. 1 is
related to an error message that was changed.

The second was that we called numpy.dot() with 2 sparse matrix(from
scipy). It worked in the past, but not now. Changing the test is easy.
I don't expect people to have done this frequently, but maybe warning
about this in the release note would help people to fix it faster. The
error message is not helpful, it tell that it can't find a common
dtype between float32 and float32 dtype. I changed the np.dot(a,b) to
a*b as this is the matrix multiplication function for sparse matrix in
scipy. This change remove the possibility to make a function that use
matrix product to work with both ndarray and sparse matrix without
special case for the object type. Not great, but there is an easy work
around. So this stay like this in the release, there should be a
warning.

The third is releated to change to the casting rules in numpy. Before
a scalar complex128 * vector float32 gived a vector of dtype
complex128. Now it give a vector of complex64. The reason is that now
the scalar of different category only change the category, not the
precision. I would consider a must that we warn clearly about this
interface change. Most people won't see it, but people that optimize
there code heavily could depend on such thing.

The other problem I had was related to the fact that I tryed to use
only the new API. This took me a few day and it is not finished, as
now I have a seg fault that is not easy to trigger. It happen in one
tests, but only when other tests a ran before... This is probably an
error from my change

The sed script that replace some macro helped, but there is few macro
change that is not in the file: NPY_ALIGNED to NPY_ARRAY_ALIGNED. idem
for NPY_WRITABLE, NPY_UPDATE_ALL, NPY_C_CONTIGUOUS and
NPY_F_CONTIGUOUS.

The sed script change NPY_ENSURECOPY to NPY_ARRAY_ENSURECOPY, but I
think that NPY_ARRAY_ENSURECOPY was introduced in numpy 1.7. Maybe
warning somewhere in the API transition doc that if people want to
stay compatible with older version of numpy, the should use an
"#ifndef NPY_ARRAY_ENSURECOPY ..." in there code.

I won't have the time to make a PR with those small change as I have a
deadline the 16 september and the 1 october. I hope my comment will be
helpful. If you still have questions, don't hesitate.

Fred