[Numpy-discussion] 1.6.0b1 half float buffer bug?

[Ralf Gommers]

On Thu, Mar 31, 2011 at 8:37 PM, Eli Stevens (Gmail)
<wickedgrey at gmail.com> wrote:
> On Thu, Mar 31, 2011 at 10:40 AM, Ralf Gommers
> <ralf.gommers at googlemail.com> wrote:
>> It would definitely need to get into beta 2, and even then I'm a
>> little hesitant to push in such a change at the last moment.
>
> If I miss the 1.6.0b2 cutoff, would a change like this be appropriate
> for 1.6.1, or will it need to wait until 1.7.0?  I wish I'd known that
> half floats were going to be in 1.6.0 (the bug tracker says 2.0); I'd
> have started working on this a few weeks earlier.  :)

There likely will not be a 1.7.0, if things with 2.0 go according to
plan. It probably wouldn't be a very appropriate change for 1.6.1.

>
>> It would
>> need some testing with other libraries that use the numpy buffer
>> protocol.
>
> I am currently working on a patch for cython to accept half-floats.
> ATM, it is able to accept the data from numpy (though it does not yet
> handle the bit-fiddling to expand the values into a float32 on read,
> nor collapse it from a float32 back into a float16 on write).  The
> code change to numpy was literally a one-line change to add NPY_HALF
> to a switch statement (though I haven't tried to import half floats
> yet).  Is one other library enough, or will I need to find another
> library to patch as well?  I think that OpenCL would probably be my
> next choice, since I think that we'll eventually want float16 support
> there too.

If it works with Cython I think that's enough. With other libraries I
did not mean that you should write patches for those, just some
testing to make sure the change is 100% backwards compatible.

>> Also, reading back the thread, Pauli seemed to disagree with
>> this.
>
> On Fri, Mar 25, 2011 at 10:21 AM, Pauli Virtanen <pav at iki.fi> wrote:
>> The buffer interface cannot be used to export the half-float types, since
>> the type is not specified in PEP 3118. Numpy cannot unilaterally add
>> nonstandard format codes to the spec.
>
> I'm currently in the process of submitting a patch to CPython to
> update the struct module to handle half-floats.  So far, the response
> on python-ideas has been generally positive (a couple +1s from people
> with commit bits, GVR has chimed in on the thread with an aside (but
> not an objection), etc.).  Unless I'm misunderstanding the objection,
> making half-floats an official part of the spec (the discussion on
> python-ideas explicitly notes that numpy driving force behind the
> change) would satisfy Pauli's concern.
>
> Next up for me is to get a patch onto the CPython issue tracker, but
> as soon as I have that done, I'll start working on adding unit tests
> to my numpy fork.  I anticipate being able to get that done today.

Sounds good.

Cheers,
Ralf