[Numpy-discussion] Removing datetime support for 1.4.x series ?

[Travis Oliphant]

On Feb 4, 2010, at 12:59 AM, Charles R Harris wrote:

>
>
> On Wed, Feb 3, 2010 at 11:48 PM, Travis Oliphant <oliphant at enthought.com 
> > wrote:
>
> On Feb 2, 2010, at 11:59 PM, David Cournapeau wrote:
>
>> Travis Oliphant wrote:
>>>
>>> On Feb 2, 2010, at 11:46 PM, David Cournapeau wrote:
>>>
>>>> On Wed, Feb 3, 2010 at 12:23 PM, David Cournapeau
>>>> <david at silveregg.co.jp <mailto:david at silveregg.co.jp>> wrote:
>>>>
>>>>>
>>>>> Sorry, my question was badly worded: besides the metadata  
>>>>> pointer, is
>>>>> there any other change related to the metadata infratructure  
>>>>> which may
>>>>> potentially change changes the publicly exported structures ? I  
>>>>> wonder
>>>>> whereas the metadata infrastructure can be kept in 1.4.x  
>>>>> independently
>>>>> of the datetime support without breaking the ABI
>>>>
>>>> FWIW, keeping the metadata pointer, and only removing datetime- 
>>>> related
>>>> things makes numpy 1.4.x backward compatible, at least as far as  
>>>> scipy
>>>> is concerned. So it seems the PyArray_Funcs change is the only
>>>> ABI-incompatible change.
>>>
>>> What do you mean by the "PyArray_Funcs change"?
>>
>> The change that broke the ABI is in the PyArray_Funcs structure
>> (ndarrayobject.h):
>>
>> struct {
>>         PyArray_VectorUnaryFunc *cast[NPY_NTYPES];
>>         ....
>>
>> Because NPY_NTYPES is bigger after the datetime change.
>>
>> If there is a way to have the datetime not expanding NPY_NTYPES,  
>> then I
>> think we can keep the ABI. I tried something with datetimes  
>> considered
>> as user types, but did not go very far (most certainly because I have
>> never used this part of the code before).
>
> Thanks for reminding me what the ABI problem is.  Yes, that will  
> break it (I was very suspicious that we could change the number of  
> basic types without ABI consequence but didn't have time to think  
> about the real problem).
>
> My intention in adding the datetime data-type was not to try and  
> preserve ABI in the process.
>
>
> If so, then it would have been better to have been upfront about  
> that when it went in. I know I pushed for inclusion, but I was told  
> that the ABI could be preserved. We've all been surprised by  
> unforeseen bugs, accidents happen. The question is what is the most  
> graceful way out. I think we should follow David's lead here as he  
> is the current release guy.

Yes, it would have been better.  But, I wasn't trying to hide  
anything.  There were suggestions that the ABI could be preserved, and  
I didn't see the argument to resisting those claims very clearly, and  
so couldn't refute them quickly.

Why the versioning matters is that we have a release with the needed  
ABI changes to support date-time.   The date-time data-type is useful  
in its current state (it's not complete but what is there is useable  
for storing date-time information).  I think giving it time for people  
to use it will help continue to improve what is there and encourage  
someone to finish the rest of the implementation (it's just not that  
much more work for someone with about 40-80 hours to spare).

Perhaps one way to articulate my perspective is the following:

There are currently 2 groups of NumPy users:

  1)  those who have re-compiled all of their code for 1.4.0
  2)  those who haven't

Group 1) will have to re-compile again no matter what we do (because  
we are either going to have to bump the ABI number or back-pedal).
Group 2) will not have to re-compile once the new release comes out.

I don't want to make Group 1) have to re-compile yet a third time when  
date-time support finally comes out.  If they have bitten the bullet  
now, they will be rewarded with a stable ABI (that will eventually  
have the benefit of better ufunc support for record arrays as well as  
the date-time features).

-Travis





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20100204/e37b5df9/attachment.html>