[Numpy-discussion] longlong format error with Python <= 2.6 in scalartypes.c

[Alok Singhal]

On Thu, Aug 18, 2011 at 9:01 PM, Mark Wiebe <mwwiebe at gmail.com> wrote:
> On Thu, Aug 4, 2011 at 4:08 PM, Derek Homeier
> <derek at astro.physik.uni-goettingen.de> wrote:
>>
>> Hi,
>>
>> commits c15a807e and c135371e (thus most immediately addressed to Mark,
>> but I am sending this to the list hoping for more insight on the issue)
>> introduce a test failure with Python 2.5+2.6 on Mac:
>>
>> FAIL: test_timedelta_scalar_construction (test_datetime.TestDateTime)
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>>  File
>> "/Users/derek/lib/python2.6/site-packages/numpy/core/tests/test_datetime.py",
>> line 219, in test_timedelta_scalar_construction
>>    assert_equal(str(np.timedelta64(3, 's')), '3 seconds')
>>  File "/Users/derek/lib/python2.6/site-packages/numpy/testing/utils.py",
>> line 313, in assert_equal
>>    raise AssertionError(msg)
>> AssertionError:
>> Items are not equal:
>>  ACTUAL: '%lld seconds'
>>  DESIRED: '3 seconds'
>>
>> due to the "lld" format passed to PyUString_FromFormat in scalartypes.c.
>> In the current npy_common.h I found the comment
>>  *      in Python 2.6 the %lld formatter is not supported. In this
>>  *      case we work around the problem by using the %zd formatter.
>> though I did not notice that problem when I cleaned up the
>> NPY_LONGLONG_FMT definitions in that file (and it is not entirely clear
>> whether the comment only pertains to Windows...). Anyway changing the
>> formatters in scalartypes.c to "zd" as well removes the failure and still
>> works with Python 2.7 and 3.2 (at least on Mac OS). However I am wondering
>> if
>> a) NPY_[U]LONGLONG_FMT should also be defined conditional to the Python
>> version (and if "%zu" is a valid formatter), and
>> b) scalartypes.c should use NPY_LONGLONG_FMT from npy_common.h
>>
>> I am attaching a patch implementing a), but only the quick and dirty
>> solution to b).
>
> I've touched this stuff as little as possible, because I rather dislike the
> way the *_FMT macros are set up right now. I added a comment about
> NPY_INTP_FMT in npy_common.h which I see you read. If you're going to try to
> fix this, I hope you fix it deeper than this patch so it's not error-prone
> anymore.
> NPY_INTP_FMT is used together with PyErr_Format/PyString_FromFormat, whereas
> the other *_FMT are used with the *printf functions from the C libraries.
> These are not compatible, and the %zd hack was put in place because it
> exists even in Python 2.4, and Py_ssize_t seems matches the  pointer size in
> all CPython versions.
> Switching the timedelta64 format in scalartypes.c.src to "%zd" won't help on
> 32-bit platforms, because it won't be a 64-bit type there, unlike how it
> works ok for the NPY_INTP_FMT. In summary:
> * There need to be changes to create a clear distinction between the *_FMT
> for PyString_FromFormat vs the *_FMT for C library *printf functions
> * I suspect we're out of luck for 32-bit older versions of CPython with
> PyString_FromFormat
> Cheers,
> -Mark

By the way, the above bug is fixed in the current master (see
https://github.com/numpy/numpy/commit/730b861120094b1ab38670b9a8895a36c19296a7).
 I fixed it in the most direct way possible, because "the correct" way
would require changes to a lot of places.