[Numpy-discussion] 1.6.0b1 half float buffer bug?

[Eli Stevens (Gmail)]

On Thu, Mar 31, 2011 at 10:40 AM, Ralf Gommers
<ralf.gommers at googlemail.com> wrote:
> It would definitely need to get into beta 2, and even then I'm a
> little hesitant to push in such a change at the last moment.

If I miss the 1.6.0b2 cutoff, would a change like this be appropriate
for 1.6.1, or will it need to wait until 1.7.0?  I wish I'd known that
half floats were going to be in 1.6.0 (the bug tracker says 2.0); I'd
have started working on this a few weeks earlier.  :)

> It would
> need some testing with other libraries that use the numpy buffer
> protocol.

I am currently working on a patch for cython to accept half-floats.
ATM, it is able to accept the data from numpy (though it does not yet
handle the bit-fiddling to expand the values into a float32 on read,
nor collapse it from a float32 back into a float16 on write).  The
code change to numpy was literally a one-line change to add NPY_HALF
to a switch statement (though I haven't tried to import half floats
yet).  Is one other library enough, or will I need to find another
library to patch as well?  I think that OpenCL would probably be my
next choice, since I think that we'll eventually want float16 support
there too.

> Also, reading back the thread, Pauli seemed to disagree with
> this.

On Fri, Mar 25, 2011 at 10:21 AM, Pauli Virtanen <pav at iki.fi> wrote:
> The buffer interface cannot be used to export the half-float types, since
> the type is not specified in PEP 3118. Numpy cannot unilaterally add
> nonstandard format codes to the spec.

I'm currently in the process of submitting a patch to CPython to
update the struct module to handle half-floats.  So far, the response
on python-ideas has been generally positive (a couple +1s from people
with commit bits, GVR has chimed in on the thread with an aside (but
not an objection), etc.).  Unless I'm misunderstanding the objection,
making half-floats an official part of the spec (the discussion on
python-ideas explicitly notes that numpy driving force behind the
change) would satisfy Pauli's concern.

Next up for me is to get a patch onto the CPython issue tracker, but
as soon as I have that done, I'll start working on adding unit tests
to my numpy fork.  I anticipate being able to get that done today.

Thanks!
Eli