[Numpy-discussion] Scipy 0.15.0 beta 1 release

[Julian Taylor]

On 11/26/2014 02:19 PM, Charles R Harris wrote:
> 
> 
> On Wed, Nov 26, 2014 at 2:06 AM, Julian Taylor
> <jtaylor.debian at googlemail.com <mailto:jtaylor.debian at googlemail.com>>
> wrote:
> 
>     On 11/26/2014 12:50 AM, Andrea Gavana wrote:
>     >
>     > On 25 November 2014 at 19:33, David Cournapeau <cournape at gmail.com <mailto:cournape at gmail.com>
>     > <mailto:cournape at gmail.com <mailto:cournape at gmail.com>>> wrote:
>     >
>     >
>     >
>     >     On Tue, Nov 25, 2014 at 6:10 PM, Sturla Molden
>     >     <sturla.molden at gmail.com <mailto:sturla.molden at gmail.com>
>     <mailto:sturla.molden at gmail.com <mailto:sturla.molden at gmail.com>>>
>     wrote:
>     >
>     >         David Cournapeau <cournape at gmail.com <mailto:cournape at gmail.com>
>     >         <mailto:cournape at gmail.com <mailto:cournape at gmail.com>>> wrote:
>     >         > Shall we consider <a
>     >         > href="https://github.com/scipy/scipy/issues/4168">https://github.com/scipy/scipy/issues/4168</a>
>     >         > to be a
>     >         > blocker (the issue arises on scipy master as well as 0.14.1) ?
>     >         >
>     >
>     >         It is really bad, but does anyone know what is really going on?
>     >
>     >
>     >     Yes, it is in the bug report.
>     >
>     >
>     >
>     > Nice move.
>     >
>     > I've now recommended to hold back any upgrade/update/pip-crap/enpkg-fun
>     > thing on NumPy/SciPy across the whole user base of Python in the
>     > company. We will probably move to 64bit-in-any-sense soon enough, I
>     > guess before this issue is solved. Tell me, NumPy, was the array aligned
>     > enough in 1.8? Is NumPy stricter in its checking because of SPARC? SPARC?!?
> 
>     yes, before the change numpy accepted a factor 10 performance regression
>     in complex indexing to satisfy sparc.
> 
>     >
>     > Dozens of f2py compiled extensions are going to fail soon here - which
>     > I'll reluctantly check tomorrow. I don't want to think about other
>     > people on Win32 facing the same issue elsewhere... :-)
>     >
> 
>     There haven't been any real complaints from applications yet, only
>     testsuite failure of scipy.
>     Eithe the one thing that is broken in scipy isn't much used or windows
>     32 users aren't using 1.9 yet.
>     The majority of f2py should still be working, numpys own f2py testsuite
>     passes on win32. I still don't know what exactly arpack is doing
>     different but I also did not have time yet to look at the testcase david
>     created.
> 
>     It should of course be fixed, I have an old branch with aligned
>     allocators lying around somewhere which should fix the issue or at least
>     give users a way to work around it.
> 
> 
> The lesson I take from this is that we need a test ;) Also, that it
> would be nice if we got more testing by users before releases. But
> things are as they are, this will get fixed, and we will move on with
> one more lesson learned.
> 
> If anyone is to blame, it is the reviewer, namely, myself.
> 


concerning actually fixing it I guess we have 3 options:

1. reduce maximum copy alignment from currently 16 to 8 on platforms
that need it.
That will automatically reduce the needed alignment of complex on win32
to 8 bytes. Do other compilers provide something similar to gccs
__BIGGEST_ALIGNMENT__?
2. remove bloating of complex alignment and let sparc crash.
3. add an aligned allocator

I somewhat favor 1, it has the risk that a vectorizing compiler might
wreak havoc but to my knowledge numpy does not actually have real 16
byte copies. There are some occurrences of npy_int128, but those likely
are not used on windows.

fwiw I could reproduce the issue on linux i386 with -malign-double, (I
could have sworn I tested that configuration too...)