[Numpy-discussion] Questions about fixes for 1.9.0rc2

[Charles R Harris]

On Fri, Jul 4, 2014 at 3:33 PM, Nathaniel Smith <njs at pobox.com> wrote:

> On Fri, Jul 4, 2014 at 10:31 PM, Charles R Harris
> <charlesr.harris at gmail.com> wrote:
> >
> > On Fri, Jul 4, 2014 at 3:15 PM, Nathaniel Smith <njs at pobox.com> wrote:
> >>
> >> On Fri, Jul 4, 2014 at 9:48 PM, Charles R Harris
> >> <charlesr.harris at gmail.com> wrote:
> >> >
> >> > On Fri, Jul 4, 2014 at 2:41 PM, Nathaniel Smith <njs at pobox.com>
> wrote:
> >> >>
> >> >> On Fri, Jul 4, 2014 at 9:33 PM, Charles R Harris
> >> >> <charlesr.harris at gmail.com> wrote:
> >> >> >
> >> >> > On Fri, Jul 4, 2014 at 2:09 PM, Nathaniel Smith <njs at pobox.com>
> >> >> > wrote:
> >> >> >>
> >> >> >> On Fri, Jul 4, 2014 at 9:02 PM, Ralf Gommers
> >> >> >> <ralf.gommers at gmail.com>
> >> >> >> wrote:
> >> >> >> >
> >> >> >> > On Fri, Jul 4, 2014 at 10:00 PM, Charles R Harris
> >> >> >> > <charlesr.harris at gmail.com> wrote:
> >> >> >> >>
> >> >> >> >> On Fri, Jul 4, 2014 at 1:42 PM, Charles R Harris
> >> >> >> >> <charlesr.harris at gmail.com> wrote:
> >> >> >> >>>
> >> >> >> >>> Sebastian Seberg has fixed one class of test failures due to
> the
> >> >> >> >>> indexing
> >> >> >> >>> changes in numpy 1.9.0b1.  There are some remaining errors,
> and
> >> >> >> >>> in
> >> >> >> >>> the
> >> >> >> >>> case
> >> >> >> >>> of the Matplotlib failures, they look to me to be Matplotlib
> >> >> >> >>> bugs.
> >> >> >> >>> The
> >> >> >> >>> 2-d
> >> >> >> >>> arrays that cause the error are returned by the overloaded
> >> >> >> >>> _interpolate_single_key function in CubicTriInterpolator that
> is
> >> >> >> >>> documented
> >> >> >> >>> in the base class to return a 1-d array, whereas the actual
> >> >> >> >>> dimensions
> >> >> >> >>> are
> >> >> >> >>> of the form (n, 1). The question is, what is the best work
> >> >> >> >>> around
> >> >> >> >>> here
> >> >> >> >>> for
> >> >> >> >>> these sorts errors? Can we afford to break Matplotlib and
> other
> >> >> >> >>> packages on
> >> >> >> >>> account of a bug that was previously accepted by Numpy?
> >> >> >> >
> >> >> >> >
> >> >> >> > It depends how bad the break is, but in principle I'd say that
> >> >> >> > breaking
> >> >> >> > Matplotlib is not OK.
> >> >> >>
> >> >> >> I agree. If it's easy to hack around it and issue a warning for
> now,
> >> >> >> and doesn't have other negative consequences, then IMO we should
> >> >> >> give
> >> >> >> matplotlib a release or so worth of grace period to fix things.
> >> >> >
> >> >> >
> >> >> > Here is another example, from skimage.
> >> >> >
> >> >> >
> >> >> >
> ======================================================================
> >> >> > ERROR: test_join.test_relabel_sequential_offset1
> >> >> >
> >> >> >
> ----------------------------------------------------------------------
> >> >> > Traceback (most recent call last):
> >> >> >   File "X:\Python27-x64\lib\site-packages\nose\case.py", line 197,
> in
> >> >> > runTest
> >> >> >     self.test(*self.arg)
> >> >> >   File
> >> >> >
> >> >> >
> >> >> >
> "X:\Python27-x64\lib\site-packages\skimage\segmentation\tests\test_join.py",
> >> >> > line 30, in test_relabel_sequential_offset1
> >> >> >     ar_relab, fw, inv = relabel_sequential(ar)
> >> >> >   File
> >> >> > "X:\Python27-x64\lib\site-packages\skimage\segmentation\_join.py",
> >> >> > line 127, in relabel_sequential
> >> >> >     forward_map[labels0] = np.arange(offset, offset + len(labels0)
> +
> >> >> > 1)
> >> >> > ValueError: shape mismatch: value array of shape (6,) could not be
> >> >> > broadcast
> >> >> > to indexing result of shape (5,)
> >> >> >
> >> >> > Which is pretty clearly a coding error. Unfortunately, the error is
> >> >> > in
> >> >> > the
> >> >> > package rather than the test.
> >> >> >
> >> >> > The only easy way to fix all of these sorts of things is to revert
> >> >> > the
> >> >> > indexing changes, and I'm loathe to do that. Grrr...
> >> >>
> >> >> Ugh, that's pretty bad :-/. Do you really think we can't use a
> >> >> band-aid over the new indexing code, though?
> >> >
> >> >
> >> > Yeah, we can. But Sebastian doesn't have time and I'm unfamiliar with
> >> > the
> >> > code, so it may take a while...
> >>
> >> Fair enough!
> >>
> >> I guess that if what are (arguably) bugs in matplotlib and
> >> scikit-image are holding up the numpy release, then it's worth CC'ing
> >> their mailing lists in case someone feels like volunteering to fix
> >> it... ;-).
> >
> > I can do that ;) Doesn't help with the release though unless we want to
> > document the errors in the release notes and tell folks to wait on the
> next
> > release of the packages.
>
> Oh, I meant, in case they want to fix numpy so that their packages
> don't break :-).
>
>
I've filed issues with all the affected projects. Here is the current
status.

matplotlib -- Reported, being fixed, should be in 1.4 in a few days.
skimage -- Reported.
scikit-learn -- Reported.
tables -- Reported.
statsmodels -- Reported, fixed in master.
bottleneck -- Reported. IIRC, kwgoodman already knew of the changes.
pyfits -- Reported to astropy.
milk -- Reported.
pandas -- Reportedly fixed in master.

If the issues are fixed in matplotlib and pandas I'd be inclined to release
as is with a mention of versions in the release notes.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140704/2af14b8e/attachment.html>