[Numpy-discussion] PSF GSoC 2010 (Py3K focus)

[David Cournapeau]

josef.pktd at gmail.com wrote:
> On Tue, Mar 9, 2010 at 9:02 AM, Pauli Virtanen <pav+sp at iki.fi> wrote:
>> Mon, 08 Mar 2010 22:39:00 -0700, Charles R Harris wrote:
>>> On Mon, Mar 8, 2010 at 10:29 PM, Jarrod Millman
>>> <millman at berkeley.edu>wrote:
>>>> I added Titus' email regarding the PSF's focus on Py3K-related
>>>> projects to our SoC ideas wiki page:
>>>> http://projects.scipy.org/scipy/wiki/SummerofCodeIdeas
>>>>
>>>> Given Titus' email, this is the most likely list of projects we will
>>>> get accepted this year:
>>>>
>>>> - finish porting NumPy to Py3K
>>> I think Numpy is pretty much done. It needs use testing to wring out any
>>> small oddities, but it doesn't look to me like a GSOC project at the
>>> moment. Maybe Pauli can weigh in here.
>> I think it's pretty much done. Even f2py should work. What's left is
>> mostly testing it, and fixing any issues that crop up.
>>
>> Note that the SVN Numpy should preferably still see more testing on
>> Python 2 against real-world packages that use it, before release. There's
>> been a significant amount of code churn.
>>
>>>> - port SciPy to Py3K
>>> This project might be more appropriate, although I'm not clear on what
>>> needs to be done.
>> I think porting Scipy proceeds in these steps:
>>
>> 1. Set up a similar 2to3-using build system for Python 3 as currently in
>>   Numpy.
>>
>> 2. Change the code so that it works both on Python 2 and Python 3.
>>   This can be done one submodule at a time.
>>
>>   For C code, the changes required are mainly PyString usage. Some macros
>>   need to be defined to allow the same code to build on Py2 and Py3.
>>   Scipy is probably easier to port than Numpy here, since IIRC it doesn't
>>   mess much with strings, and its use of the Python C-API is much more
>>   limited. Also, parts written with Cython need essentially no porting.
>>
>>   For Python code, the main issue is again bytes vs. unicode fixes,
>>   mainly inserting numpy.compat.asbytes() at correct locations to make
>>   e.g. bytes literals. All I/O code should be changed to work solely with
>>   Bytes. Since 2to3 takes care of most the other things, the remaining
>>   work is in fixing things it mishandles.
>>
>> I don't have a good estimate for how much work is in making Scipy work.
>> I'd guess the work needed is probably slightly less than for Numpy.
> 
> a few questions:
> 
> Is scipy.special difficult or time consuming to port?

I don't think it would be - most of it is fortran, so assuming f2py 
works correctly for py3k, there should not be big issues.

> In the build system, is it possible not to build some subpackages that
> might be slow in being ported, e.g. ndimage, weave?

The way I used to do when porting to a new compiler/new platform is 
simply to comment everything but one package at a time in 
scipy/setup.py. linalg/lib/clusters are the first ones to port. I don't 
think special depends on much more than linalg/lib, but I could be wrong.

David