[Numpy-discussion] numpy 2.0, what else to do?

[Charles R Harris]

On Mon, Feb 15, 2010 at 10:35 PM, Charles R Harris <
charlesr.harris at gmail.com> wrote:

>
>
> On Mon, Feb 15, 2010 at 10:19 PM, David Cournapeau <david at silveregg.co.jp>wrote:
>
>> Charles R Harris wrote:
>> >
>> >
>> > On Mon, Feb 15, 2010 at 8:13 PM, David Cournapeau <
>> david at silveregg.co.jp
>> > <mailto:david at silveregg.co.jp>> wrote:
>> >
>> >     Charles R Harris wrote:
>> >      >
>> >      >
>> >      > On Mon, Feb 15, 2010 at 3:58 PM, Charles R Harris
>> >      > <charlesr.harris at gmail.com <mailto:charlesr.harris at gmail.com>
>> >     <mailto:charlesr.harris at gmail.com
>> >     <mailto:charlesr.harris at gmail.com>>> wrote:
>> >      >
>> >      >
>> >      >
>> >      >     On Mon, Feb 15, 2010 at 3:34 PM, David Cournapeau
>> >      >     <cournape at gmail.com <mailto:cournape at gmail.com>
>> >     <mailto:cournape at gmail.com <mailto:cournape at gmail.com>>> wrote:
>> >      >
>> >      >         On Tue, Feb 16, 2010 at 7:04 AM, Charles R Harris
>> >      >         <charlesr.harris at gmail.com
>> >     <mailto:charlesr.harris at gmail.com> <mailto:
>> charlesr.harris at gmail.com
>> >     <mailto:charlesr.harris at gmail.com>>>
>> >      >         wrote:
>> >      >          >
>> >      >          >
>> >      >          > On Mon, Feb 15, 2010 at 2:46 PM, David Cournapeau
>> >      >         <cournape at gmail.com <mailto:cournape at gmail.com>
>> >     <mailto:cournape at gmail.com <mailto:cournape at gmail.com>>>
>> >      >          > wrote:
>> >      >          >>
>> >      >          >> On Tue, Feb 16, 2010 at 4:08 AM, Charles R Harris
>> >      >          >> <charlesr.harris at gmail.com
>> >     <mailto:charlesr.harris at gmail.com>
>> >      >         <mailto:charlesr.harris at gmail.com
>> >     <mailto:charlesr.harris at gmail.com>>> wrote:
>> >      >          >>
>> >      >          >> >
>> >      >          >> > I was wondering about that. Why do we have a
>> private
>> >      >         include directory?
>> >      >          >> > Would it make more sense to move it to
>> >      >         core/include/numpy/private.
>> >      >          >>
>> >      >          >> No, the whole point is to avoid other packages to
>> include
>> >      >         that by
>> >      >          >> mistake, to avoid namespace pollution.
>> >      >          >
>> >      >          > Isn't that what the npy prefix is for?
>> >      >
>> >      >         No, npy_ is for public symbols. Anything in private
>> should be
>> >      >         private :)
>> >      >
>> >      >          > In any case, if it needs to be at a
>> >      >          > higher level for easy inclusion, then it should move
>> up.
>> >      >
>> >      >         It is not that easy - we should avoid putting this code
>> into
>> >      >         core/include, because then we have to keep it compatible
>> >     across
>> >      >         releases, but there is no easy way to share headers
>> >     between modules
>> >      >         without making it public.
>> >      >
>> >      >
>> >      >     Py_TYPE, Py_Size, etc. are unlikely to cause compatibility
>> >     problems
>> >      >     across releases.
>> >      >
>> >      >
>> >      >
>> >      > In particular, I think
>> >      >
>> >      > #if (PY_VERSION_HEX < 0x02060000)
>> >      > #define Py_TYPE(o)    (((PyObject*)(o))->ob_type)
>> >      > #define Py_REFCNT(o)  (((PyObject*)(o))->ob_refcnt)
>> >      > #define Py_SIZE(o)    (((PyVarObject*)(o))->ob_size)
>> >      > #endif
>> >      >
>> >      > belongs somewhere near the top, maybe with a prefix (cython seems
>> to
>> >      > define them also)
>> >
>> >     The rule is easy: one should put in core/include/numpy whatever is
>> >     public, and put in private what is not.
>> >
>> >     Note that defining those macros above publicly is very likely to
>> cause
>> >     trouble because I am sure other people do define those macros,
>> without
>> >     caring about polluting the namespace as well. Given that it is
>> >     temporary, and is small, I think copying the compat header is better
>> >     than making it public, the best solution being to add something in
>> >     distutils to share it between submodules,
>> >
>> >
>> > You would prefer to fix the macros in ndarrayobject.h using #ifdef's
>> then?
>>
>> In case what I am worried about is not clear: if ndarrayobject.h defines
>> Py_TYPE, it means that every C extensions using the numpy C API will
>> have Py_TYPE in the public namespace. Now, if another python extension
>> with a C API does the same, you have issues. Having #ifdef/#endif around
>> only make it worse because then you have strange interactions depending
>> on the order of header inclusion (I really hate that behavior from the
>> python headers).
>>
>> The numpy C headers are already pretty messy, let's not make it worse.
>> Especially since the workaround is trivial.
>>
>>
> What is the work around? Mind, I think those macros need to be compatible
> with py3k just to make porting other applications easier. I still think we
> should call it NPY_Py_TYPE or some such. We also have some stray ob_refcnt.
> Note that the gnu headers also have implementation stuff hidden away in a
> folder. Whatever we do, I think it needs to be easy discover for anyone
> coming new to the code, it shouldn't be hidden away in somewhere in the
> distutils. That's like burying it on a small Caribbean island along with all
> the witnesses.
>
>
Just to be clear, there are *already* macros in the ndarrayobject.h file
that aren't py3k compatible. How do you propose to fix those?

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20100215/aebe2172/attachment.html>