[Numpy-discussion] Python 3K merge

[René Dudfield]

On Thu, Dec 3, 2009 at 10:39 AM, Pauli Virtanen <pav+sp at iki.fi<pav%2Bsp at iki.fi>
> wrote:

> Tue, 01 Dec 2009 17:31:10 +0900, David Cournapeau wrote:
> > On Tue, Dec 1, 2009 at 5:04 AM, Charles R Harris
> > <charlesr.harris at gmail.com> wrote:
> >> It looks like you doing great stuff with the py3k transition. Do you
> >> and David have any sort of merge schedule in mind?
> >
> > I have updated my py3k branch for numpy.distutils, and it is ready to
> > merge:
> >
> > http://github.com/cournape/numpy/tree/py3k_bootstrap_take3
> >
> > I have not thoroughly tested it, but it can run on both 2.4 and 3.1 on
> > Linux at least. The patch is much smaller than my previous attempts as
> > well, so I would just push it to the trunk, and deal with the issues as
> > they come.
>
> I think I should rebase my branch on this, or vice versa, to avoid
> further duplicated work.
>
> I think most of my changes would be ready for SVN, after rebasing and
> regrouping via rebase -i -- they do not affect behavior on Py2, and for
> the most part the changes required in C code are quite obvious.
>
> The largest changes are probably related to the 'S' data type.
>
> In other news, we cannot support Py2 pickles in Py3 -- this is because
> Py2 str is unpickled as Py3 str, resulting to encoding failures even
> before the data is passed on to Numpy.
>
>
Is this just for the type codes?  Or is there other string data that needs
to be pickle loaded?  If it is just for the type codes, they are all within
the ansi character set and unpickle fine without errors.  I'm guessing numpy
uses strings to pickle arrays?

Note that the pickle module is extensible.  So we might be able to get it to
special case things.  You can subclass Unpickler to make extensions... and
there are other techniques.  Or it's even possible to submit patches to
python if we have a need for something it doesn't support.

It is even possible to change the pickle code for py2, so that py3
compatible pickles are saved.  In this case it would just require people to
load, and resave their pickles with the latest numpy version.

Using the python array module to store data might be the way to go(rather
than strings), since that is available in both py2 and py3.

The pickling/unpickling situtation should be marked as a todo, and
documented anyway.  As we should start a numpy specific 'porting your code
to py3k' document.

A set of pickles saved from python2 would be useful for testing.  Forwards
compatibility is also a useful thing to test.  That is py3.1 pickles saved
to be loaded with python2 numpy.



cheers!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20091203/bc03c7a1/attachment.html>