[Distutils] Status update on the NumPy & SciPy vs SSE problem?

[Matthew Brett]

On Thu, Feb 4, 2016 at 9:01 AM, Nathaniel Smith <njs at pobox.com> wrote:
> On Feb 4, 2016 3:22 AM, "Nick Coghlan" <ncoghlan at gmail.com> wrote:
>>
>> While the manylinux PEP brings Linux up to comparable standing with
>> Windows and Mac OS X in terms of distributing wheel files through
>> PyPI, that does mean it still suffers from the same problem Windows
>> does in relation to NumPy and SciPy wheels: no standardisation of the
>> SSE capabilities of the machines.
>>
>> I figured that was independent of the manylinux PEP (since it affects
>> Windows as well), but I'm also curious as to the current status (I
>> found a couple of apparently relevant threads on the NumPy list, but
>> figured it made more sense to just ask for an update rather than
>> trusting my Google-fu)
>
> I'm not entirely sure what the SSE status is of the numpy OSX wheels. I
> think that they may be just following Apple's guidance on this (in the sense
> of: we tell their compiler to target a certain OS version and then use
> default options beyond that), but I'm not sure. It may even differ between
> the 32- and 64-bit "parts" of the fat binaries. Asking on numpy-discussion
> might net more details.

I'm more or less responsible for the numpy and scipy OSX wheels.  The
compiler flags for building come from the compiler flags for
Python.org Python via distutils. As Nathaniel says, the big speed
problem and opportunity is in the BLAS / LAPACK libraries, and we link
against the Accelerate library for this, which comes installed on OSX.
This seems to be well-tuned to the underlying hardware.

Another option for BLAS / LAPACK is OpenBLAS which can do run-time CPU
detection to select the fastest (and not-crashing) code-paths.

> Otherwise, yeah, the current plan is to jump to SSE2 as the minimum required.
> version as the new wheels become usable, since all the evidence seems to say
> that it's ubiquitous now.

Some of that evidence for Windows is listed at
https://github.com/numpy/numpy/wiki/Windows-versions

Also, SSE2 instructions are part of the specification of the AMD64
architecture [1] and so, quoting from [2] "The SSE2 instruction set is
supported on all 64-bit CPUs and operating systems".

Cheers,

Matthew

[1] https://courses.cs.washington.edu/courses/cse351/12wi/supp-docs/abi.pdf
[2] http://www.agner.org/optimize/optimizing_cpp.pdf