[Numpy-discussion] I want to help with a numpy python 3.1.x port

[René Dudfield]

On Fri, Sep 18, 2009 at 3:10 PM, Charles R Harris
<charlesr.harris at gmail.com> wrote:
> Hi René,
>
> On Fri, Sep 18, 2009 at 6:01 AM, René Dudfield <renesd at gmail.com> wrote:
>>
>> Hello,
>>
>> as a big numpy user, and someone wanting to help with the python 3
>> migration, I'd like to help with a python 3.1 port of numpy.
>>
>> We(at the pygame project) have mostly completed our port of pygame to
>> python 3.0 and 3.1 so can offer some insight into what it takes with a
>> CPython extension.
>>
>> pygame supports python 2.3 through to 3.1, so it should be possible to
>> also keep backwards compatibility with the port for numpy.  We can
>> also use some of the helper code we used for the pygame port.
>>
>> I haven't asked the other pygame developers if they are interested in
>> helping too... but maybe they will be interested in helping too(since
>> they use numpy too).  I'd also be interested in other people helping
>> with the effort.  Once I have put some of the ground work in place,
>> and got a few parts done it should be easier for other people to see
>> what needs changing.
>>
>>
>> If the python 3 port is something you'd want included with numpy, then
>> I'd like to begin this weekend.
>>
>> I'm not super familiar with the numpy code base yet, so I'd like to do
>> it in small changes making small parts compatible with py3k and then
>> having them reviewed/tested.
>>
>> I can either start a branch myself using mecurial, or perhaps you'd
>> like me to do the work somewhere else ( like in a branch in the numpy
>> svn?).
>>
>> Which code should I base the port off?  trunk ?
>>
>
> Darren Dale and I are just getting started on a port and welcome any help
> you can offer.  Because of the difficulty of maintaining two branches the
> only route that looks good at this point is to get the python parts of numpy
> in a state that will allow 2to3 to work and use #ifdef's in the c code. What
> was your experience with pygames?
>

ah cool!  That's great to hear.  Where is your work going to be done?

Well, we managed to make the code base runnable in python 2.x and 3
without using the 2to3 tool.  We didn't need, or want to support
different versions of source code.  In lots of cases it's possible to
support versions of python at once... if you aren't using any new
python3 features that the older pythons don't have.


For the C code there's a header which helps with a lot of the backward
compat stuff.  Had to implement a few things because of missing
things... like simple slicing is gone from py3k (sq_slice) etc.

Here's a header file we used with some compatibility macros...
http://www.seul.org/viewcvs/viewcvs.cgi/trunk/src/pgcompat.h?rev=2049&root=PyGame&sortby=date&view=markup

but we also have some other compat stuff in other headers... but I
think most of the py3k stuff is in there.  You could probably just
copy that header file to use it.

The module initialisation works slightly differently so we have #ifdef
PY3 for those differences.  See the bottom of this file for an example
module:
    http://www.seul.org/viewcvs/viewcvs.cgi/trunk/src/rect.c?rev=2617&root=PyGame&sortby=date&view=markup

For things like the missing sq_slice, rather than rewriting things we
just wrote code to support stuff that py3k removed.



> Because the numpy c code is difficult to come to grips with the easiest part
> for inexperienced c coders and newbies is to start on is probably the python
> code. There is a fair amount of that so some plan of attack and a check list
> probably needs to be set up. Any such list should be kept in svn along with
> any helpful notes about problems/solutions encountered along the way.
>

Well plenty of tests are something that's needed for sure... so add
that to your checklist.

It can be a good idea to get the setup.py, and testing framework
ported early, so it's easier to check if code still works.

I'm happy to help with the C stuff, and the .py stuff too.


> We also need David C. to commit his build changes for py3k so we can
> actually build the whole thing when the time comes. (Hint, hint). Also, I'm
> thinking of skipping 3.0 and starting with 3.1 because of the fixes,
> particularly the disappearance of cmp, that went in between the versions.
>
> Chuck

yeah, skipping 3.0 is a good idea... just go straight to 3.1.  We
started our port during 3.0 before 3.1... but there weren't all that
many differences for us between the two versions.



cheers,