[Numpy-discussion] question about future support for python-3

[Darren Dale]

On Wed, Sep 9, 2009 at 11:25 AM, Charles R
Harris<charlesr.harris at gmail.com> wrote:
>
>
> On Wed, Sep 9, 2009 at 7:15 AM, Darren Dale <dsdale24 at gmail.com> wrote:
>>
>> On Tue, Sep 8, 2009 at 9:02 PM, David Cournapeau<cournape at gmail.com>
>> wrote:
>> > On Wed, Sep 9, 2009 at 9:37 AM, Darren Dale<dsdale24 at gmail.com> wrote:
>> >> Hi David,
>> >
>> >>> I already gave my own opinion on py3k, which can be summarized as:
>> >>>  - it is a huge effort, and no core numpy/scipy developer has
>> >>> expressed the urge to transition to py3k, since py3k does not bring
>> >>> much for scientific computing.
>> >>>  - very few packages with a significant portion of C have been ported
>> >>> to my knowledge, hence very little experience on how to do it. AFAIK,
>> >>> only small packages have been ported. Even big, pure python projects
>> >>> have not been ported. The only big C project to have been ported is
>> >>> python itself, and it broke compatibility and used a different source
>> >>> tree than python 2.
>> >>>  - it remains to be seen whether we can do the py3k support in the
>> >>> same source tree as the one use for python >= 2.4. Having two source
>> >>> trees would make the effort even much bigger, well over the current
>> >>> developers capacity IMHO.
>> >>>
>> >>> The only area where I could see the PSF helping is the point 2: more
>> >>> documentation, more stories about 2->3 transition.
>> >>
>> >> I'm surprised to hear you say that. I would think additional developer
>> >> and/or financial resources would be useful, for all of the reasons you
>> >> listed.
>> >
>> > If there was enough resources to pay someone very familiar with numpy
>> > codebase for a long time, then yes, it could be useful - but I assume
>> > that's out of the question. This would be very expensive as it would
>> > requires several full months IMO.
>> >
>> > The PSF could help for the point 3, by porting other projects to py3k
>> > and documenting it. The only example I know so far is pycog2
>> >
>> > (http://mail.python.org/pipermail/python-porting/2008-December/000010.html).
>> >
>> > Paying people to do documentation about porting C code seems like a
>> > good way to spend money: it would be useful outside numpy community,
>> > and would presumably be less costly.
>>
>> Another topic concerning documentation is API compatibility. The
>> python devs have requested projects not use the 2-3 transition as an
>> excuse to change their APIs, but numpy is maybe a special case. I'm
>> thinking about PEP3118. Is numpy going to transition to python 3 and
>> then down the road transition again to the new buffer protocol? What
>> is the strategy here? My underinformed impression is that there isn't
>> one, since every time PEP3118 is considered in the context of the 2-3
>> transition somebody helpfully reminds the list that we aren't supposed
>> to break APIs. Numpy is a critical python library, perhaps the
>> transition presents an opportunity, if the community can yield a
>> little on numpy's C api. For example, in the long run, what would it
>> take to get numpy (or the core thereof) into the standard library, and
>> can we take steps now in that direction? Would the numpy devs be
>> receptive to comments from the python devs on the existing numpy
>> codebase?
>>
>> I'm willing to pitch in and work on the transition, not because I need
>> python-3 right now, but because the transition needs to happen and it
>> would benefit everyone in the long run. But I would like to know that
>> we are making the most of the opportunity, and have considered our
>> options.
>>
>
> Making numpy more buffer centric is an interesting idea and might be where
> we want to go with the ufuncs, but the new buffer protocol didn't go in
> until python 2.6. If there was no rush I'd go with Fernando and wait until
> we could be all python 2.6 all the time.

I wonder what such a timeframe would look like, what would decide when
to require python-2.6 for future releases of packages. Could a
maintenance-only branch be created for the numpy-1.4 or 1.5 series,
and then future development require 2.6 or 3.1?

> However, if anyone has the time to work on getting the c-code up to snuff
> and finding out what the problems are I'm all for that. I have some notes on
> the transition in the src directory and if you do anything please keep them
> current.

I will have a look, thank you for putting those notes together.

Darren