[Numpy-discussion] NumPy SVN broken

[David Cournapeau]

On Wed, Oct 7, 2009 at 2:04 AM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
>
>
> On Tue, Oct 6, 2009 at 10:50 AM, David Cournapeau <cournape at gmail.com>
> wrote:
>>
>> On Wed, Oct 7, 2009 at 1:36 AM, Charles R Harris
>> <charlesr.harris at gmail.com> wrote:
>> >
>> >
>> > 2009/10/6 Stéfan van der Walt <stefan at sun.ac.za>
>> >>
>> >> Hi all,
>> >>
>> >> The current SVN HEAD of NumPy is broken and should not be used.
>> >> Extensions compiled against this version may (will) segfault.
>> >>
>> >
>> > Can you be more specific? I haven't had any problems running current svn
>> > with scipy.
>>
>> The version itself is fine, but the ABI has been changed in an
>> incompatible way: if you have an extension built against say numpy
>> 1.2.1, and then use a numpy built from sources after the datetime
>> merge, it will segfault right away. It does so for scipy and several
>> custom extensions. The abi breakage was found to be the datetime
>> merge.
>>
>
> Ah... That's a fine kettle of fish. Any idea what ABI calls are causing the
> problem? Maybe the dtype change wasn't made in a compatible way. IIRC,
> something was added to the dtype?

Yes, but that should not cause trouble. Adding members to structure
should be fine.

I quickly look at the diff, and some changes in the code generators
look suspicious, e.g.:

 types = ['Generic','Number','Integer','SignedInteger','UnsignedInteger',
-         'Inexact',
+         'Inexact', 'TimeInteger',
          'Floating', 'ComplexFloating', 'Flexible', 'Character',
          'Byte','Short','Int', 'Long', 'LongLong', 'UByte', 'UShort',
          'UInt', 'ULong', 'ULongLong', 'Float', 'Double', 'LongDouble',
          'CFloat', 'CDouble', 'CLongDouble', 'Object', 'String', 'Unicode',
-         'Void']
+         'Void', 'Datetime', 'Timedelta']

As the list is used to initialize some values from the API function
pointer array, inserts  should be avoided. You can see the consequence
on the generated files, e.g. part of __multiarray_api.h diff between
datetimemerge and just before:

< #define PyFloatingArrType_Type (*(PyTypeObject *)PyArray_API[16])
< #define PyComplexFloatingArrType_Type (*(PyTypeObject *)PyArray_API[17])
< #define PyFlexibleArrType_Type (*(PyTypeObject *)PyArray_API[18])
< #define PyCharacterArrType_Type (*(PyTypeObject *)PyArray_API[19])
< #define PyByteArrType_Type (*(PyTypeObject *)PyArray_API[20])
< #define PyShortArrType_Type (*(PyTypeObject *)PyArray_API[21])
< #define PyIntArrType_Type (*(PyTypeObject *)PyArray_API[22])
< #define PyLongArrType_Type (*(PyTypeObject *)PyArray_API[23])
< #define PyLongLongArrType_Type (*(PyTypeObject *)PyArray_API[24])
< #define PyUByteArrType_Type (*(PyTypeObject *)PyArray_API[25])
< #define PyUShortArrType_Type (*(PyTypeObject *)PyArray_API[26])
< #define PyUIntArrType_Type (*(PyTypeObject *)PyArray_API[27])
< #define PyULongArrType_Type (*(PyTypeObject *)PyArray_API[28])
< #define PyULongLongArrType_Type (*(PyTypeObject *)PyArray_API[29])
< #define PyFloatArrType_Type (*(PyTypeObject *)PyArray_API[30])
< #define PyDoubleArrType_Type (*(PyTypeObject *)PyArray_API[31])
< #define PyLongDoubleArrType_Type (*(PyTypeObject *)PyArray_API[32])
< #define PyCFloatArrType_Type (*(PyTypeObject *)PyArray_API[33])
< #define PyCDoubleArrType_Type (*(PyTypeObject *)PyArray_API[34])
< #define PyCLongDoubleArrType_Type (*(PyTypeObject *)PyArray_API[35])
< #define PyObjectArrType_Type (*(PyTypeObject *)PyArray_API[36])
< #define PyStringArrType_Type (*(PyTypeObject *)PyArray_API[37])
< #define PyUnicodeArrType_Type (*(PyTypeObject *)PyArray_API[38])
< #define PyVoidArrType_Type (*(PyTypeObject *)PyArray_API[39])
---
> #define PyTimeIntegerArrType_Type (*(PyTypeObject *)PyArray_API[16])
> #define PyFloatingArrType_Type (*(PyTypeObject *)PyArray_API[17])
> #define PyComplexFloatingArrType_Type (*(PyTypeObject *)PyArray_API[18])
> #define PyFlexibleArrType_Type (*(PyTypeObject *)PyArray_API[19])
> #define PyCharacterArrType_Type (*(PyTypeObject *)PyArray_API[20])
> #define PyByteArrType_Type (*(PyTypeObject *)PyArray_API[21])
> #define PyShortArrType_Type (*(PyTypeObject *)PyArray_API[22])
> #define PyIntArrType_Type (*(PyTypeObject *)PyArray_API[23])
> #define PyLongArrType_Type (*(PyTypeObject *)PyArray_API[24])
> #define PyLongLongArrType_Type (*(PyTypeObject *)PyArray_API[25])
> #define PyUByteArrType_Type (*(PyTypeObject *)PyArray_API[26])
> #define PyUShortArrType_Type (*(PyTypeObject *)PyArray_API[27])
> #define PyUIntArrType_Type (*(PyTypeObject *)PyArray_API[28])
> #define PyULongArrType_Type (*(PyTypeObject *)PyArray_API[29])
> #define PyULongLongArrType_Type (*(PyTypeObject *)PyArray_API[30])
> #define PyFloatArrType_Type (*(PyTypeObject *)PyArray_API[31])
> #define PyDoubleArrType_Type (*(PyTypeObject *)PyArray_API[32])
> #define PyLongDoubleArrType_Type (*(PyTypeObject *)PyArray_API[33])
> #define PyCFloatArrType_Type (*(PyTypeObject *)PyArray_API[34])
> #define PyCDoubleArrType_Type (*(PyTypeObject *)PyArray_API[35])
> #define PyCLongDoubleArrType_Type (*(PyTypeObject *)PyArray_API[36])
> #define PyObjectArrType_Type (*(PyTypeObject *)PyArray_API[37])
> #define PyStringArrType_Type (*(PyTypeObject *)PyArray_API[38])
> #define PyUnicodeArrType_Type (*(PyTypeObject *)PyArray_API[39])
> #define PyVoidArrType_Type (*(PyTypeObject *)PyArray_API[40])
> #define PyDatetimeArrType_Type (*(PyTypeObject *)PyArray_API[41])
> #define PyTimedeltaArrType_Type (*(PyTypeObject *)PyArray_API[42])

David