[SciPy-dev] Porting SciPy to Py3k GSOC project

[ross smith]

Just to summarize what I'm seeing.

Porting all three (distutils, NumPy and SciPY) is going to be a too
ambitious idea for a summer.  Likewise,  all the work necisary to get even
NumPy to compile would make it a poor project as well.

on the other hand, porting relevant parts of NumPy to Cython would take a
more reasonable amount of work,  and would provide more room for testing and
bug squashing during the summer as well as helping with the eventual
porting. Another possible project is to port f2py, a smaller project that's
all python that would be valueable to the overall porting of the code.

I don't have any experience (yet) with cython so I'm a little worried about
the learning curve associated with it. I'd tend to lean toward the porting
of f2py as the project I'd be interested in.  Possibly with the addition of
a design document for what specifics will need to be done to get Numpy
completely ported.

I haven't had the chance to look at f2py yet, but I'll put together a rough
outline of a timeline for the f2py and Cython projects and post back for
your thoughts.

-Ross

On Tue, Mar 24, 2009 at 15:22, ross smith <gaurdro at gmail.com> wrote:

> Just to summarize what I'm seeing.
>
> Porting all three (distutils, NumPy and SciPY) is going to be a too
> ambitious idea for a summer.  Likewise,  all the work necisary to get even
> NumPy to compile would make it a poor project as well.
>
> on the other hand, porting relevant parts of NumPy to Cython would take a
> more reasonable amount of work,  and would provide more room for testing and
> bug squashing during the summer as well as helping with the eventual
> porting. Another possible project is to port f2py, a smaller project that's
> all python that would be valueable to the overall porting of the code.
>
> I don't have any experience (yet) with cython so I'm a little worried about
> the learning curve associated with it. I'd tend to lean toward the porting
> of f2py as the project I'd be interested in.  Possibly with the addition of
> a design document for what specifics will need to be done to get Numpy
> completely ported.
>
> I haven't had the chance to look at f2py yet, but I'll put together a rough
> outline of a timeline for the f2py and Cython projects and post back for
> your thoughts.
>
> -Ross
>
>
> 2009/3/24 Charles R Harris <charlesr.harris at gmail.com>
>
>>
>>
>> On Tue, Mar 24, 2009 at 8:17 AM, Charles R Harris <
>> charlesr.harris at gmail.com> wrote:
>>
>>>
>>>
>>> On Tue, Mar 24, 2009 at 2:21 AM, David Cournapeau <
>>> david at ar.media.kyoto-u.ac.jp> wrote:
>>>
>>>> Stéfan van der Walt wrote:
>>>> > 2009/3/24 David Cournapeau <david at ar.media.kyoto-u.ac.jp>:
>>>> >
>>>> >> You would need to port numpy itself before starting scipy as well. I
>>>> >> think it is fair to say that most work will be inside numpy/core -
>>>> which
>>>> >> is ~ 30 000 LOC according to sloccount (wo counting comments, empty
>>>> >> lines and the like); IOW, it is massive, and there is no chance to do
>>>> >> the conversion in a couple of months unless you are very familiar
>>>> with
>>>> >> numpy. I am not sure I can see a meaningful subpart of numpy/core
>>>> which
>>>> >> could be ported for python 3 for a SoC.
>>>> >>
>>>> >
>>>> > Is the situation really that dire?  How much has the C API changed
>>>> > between 2 and 3, and are these changes difficult to propagate?
>>>> >
>>>>
>>>> Talking only about the C code, here are some things which changed:
>>>>    - the buffer API is changed -> I don't know how significant this is
>>>>    - the basic C types/objects structures have changed a bit -> again,
>>>> no idea how significant this is
>>>>    - the Unicode/String unification
>>>>    - long/int unification
>>>>
>>>> But I would think the main problem is that numpy simply is a big,
>>>> complicated set of C code, whose parts can't simply be done separately.
>>>> You would need to do quite a bit of changes for the code to only
>>>> compile, making bugs hard to track - and that's assuming numpy.distutils
>>>> itself won't get in the way. Then, there is the problem on how to deal
>>>> with two codebases - if we can't handle things with a few #ifdef, the
>>>> situation will be really bad. That's why I am not convinced that it
>>>> would be a good project for a GSoC. You can't try things easily.
>>>>
>>>> Using cython for the C code is the advice given by the python doc itself
>>>> - although again, numpy is not the usual extension. I think almost
>>>> everything outside numpy/core should be relatively easy to convert to
>>>> cython (where easy means would take time, but could be done gradually
>>>> without impact everywhere). I don't know how usable cython would be to
>>>> define C-accessible, public extension types. But if it is, then things
>>>> can be done gradually - this can be tested by many people, etc...
>>>>
>>>
>>> A while back I took a shot at interfacing lapack_lite using cython just
>>> to see what it looked like, but decided that the current interface was
>>> actually pretty clean. There is also fftpack. Random is already done. Are
>>> there any separate bits folks can think of?
>>>
>>
>> Continuing the cython thoughts, there are parts of the current c code that
>> I think would look better in python. For instance, I think most of
>> arraymethods.c and umath_ufunc_object.inc would look cleaner in python.
>> However, I don't know what this would mean for the C-ABI and call overhead.
>>
>> Chuck
>>
>>
>>
>> _______________________________________________
>> Scipy-dev mailing list
>> Scipy-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090324/fe449a71/attachment.html>