[Numpy-discussion] 64-bit windows numpy / scipy wheels for testing

[Ralf Gommers]

On Fri, May 23, 2014 at 2:41 AM, Matthew Brett <matthew.brett at gmail.com>
wrote:

> Hi,
>
> On Fri, May 9, 2014 at 4:06 AM, David Cournapeau <cournape at gmail.com>
> wrote:
> >
> >
> >
> > On Fri, May 9, 2014 at 11:49 AM, Julian Taylor
> > <jtaylor.debian at googlemail.com> wrote:
> >>
> >> On 09.05.2014 12:42, David Cournapeau wrote:
> >> >
> >> >
> >> >
> >> > On Fri, May 9, 2014 at 1:51 AM, Matthew Brett <
> matthew.brett at gmail.com
> >> > <mailto:matthew.brett at gmail.com>> wrote:
> >> >
> >> >     Hi,
> >> >
> >> >     On Mon, Apr 28, 2014 at 3:29 PM, David Cournapeau
> >> >     <cournape at gmail.com <mailto:cournape at gmail.com>> wrote:
> >> >     >
> >> >     >
> >> >     >
> >> >     > On Sun, Apr 27, 2014 at 11:50 PM, Matthew Brett
> >> >     <matthew.brett at gmail.com <mailto:matthew.brett at gmail.com>>
> >> >     > wrote:
> >> >     >>
> >> >     >> Aha,
> >> >     >>
> >> >     >> On Sun, Apr 27, 2014 at 3:19 PM, Matthew Brett
> >> >     <matthew.brett at gmail.com <mailto:matthew.brett at gmail.com>>
> >> >     >> wrote:
> >> >     >> > Hi,
> >> >     >> >
> >> >     >> > On Sun, Apr 27, 2014 at 3:06 PM, Carl Kleffner
> >> >     <cmkleffner at gmail.com <mailto:cmkleffner at gmail.com>>
> >> >     >> > wrote:
> >> >     >> >> A possible option is to install the toolchain inside
> >> >     site-packages and
> >> >     >> >> to
> >> >     >> >> deploy it as PYPI wheel or wininst packages. The PATH to the
> >> >     toolchain
> >> >     >> >> could
> >> >     >> >> be extended during import of the package. But I have no
> idea,
> >> >     whats the
> >> >     >> >> best
> >> >     >> >> strategy to additionaly install ATLAS or other third party
> >> >     libraries.
> >> >     >> >
> >> >     >> > Maybe we could provide ATLAS binaries for 32 / 64 bit as part
> >> >     of the
> >> >     >> > devkit package.  It sounds like OpenBLAS will be much easier
> to
> >> >     build,
> >> >     >> > so we could start with ATLAS binaries as a default, expecting
> >> >     OpenBLAS
> >> >     >> > to be built more often with the toolchain.  I think that's
> how
> >> >     numpy
> >> >     >> > binary installers are built at the moment - using old binary
> >> >     builds of
> >> >     >> > ATLAS.
> >> >     >> >
> >> >     >> > I'm happy to provide the builds of ATLAS - e.g. here:
> >> >     >> >
> >> >     >> > https://nipy.bic.berkeley.edu/scipy_installers/atlas_builds
> >> >     >>
> >> >     >> I just found the official numpy binary builds of ATLAS:
> >> >     >>
> >> >     >> https://github.com/numpy/vendor/tree/master/binaries
> >> >     >>
> >> >     >> But - they are from an old version of ATLAS / Lapack, and only
> >> >     for 32-bit.
> >> >     >>
> >> >     >> David - what say we update these to latest ATLAS stable?
> >> >     >
> >> >     >
> >> >     > Fine by me (not that you need my approval !).
> >> >     >
> >> >     > How easy is it to build ATLAS targetting a specific CPU these
> days
> >> >     ? I think
> >> >     > we need to at least support nosse and sse2 and above.
> >> >
> >> >     I'm getting crashes trying to build SSE2-only ATLAS on 32-bits, I
> >> >     think Clint will have some time to help out next week.
> >> >
> >> >     I did some analysis of SSE2 prevalence here:
> >> >
> >> >     https://github.com/numpy/numpy/wiki/Window-versions
> >> >
> >> >     Firefox crash reports now have about 1 percent of machines without
> >> >     SSE2.  I suspect that people running new installs of numpy will
> have
> >> >     slightly better machines on average than Firefox users, but it's
> >> > only
> >> >     a guess.
> >> >
> >> >     I wonder if we could add a CPU check on numpy import to give a
> >> > polite
> >> >     'install from the exe' message for people without SSE2.
> >> >
> >> >
> >> > We could, although you unfortunately can't do it easily from ctypes
> only
> >> > (as you need some ASM).
> >> >
> >> > I can take a quick look at a simple cython extension that could be
> >> > imported before anything else, and would raise an ImportError if the
> >> > wrong arch is detected.
> >> >
> >>
> >> assuming mingw is new enough
> >>
> >> #ifdef __SSE2___
> >> raise_if(!__builtin_cpu_supports("sse"))
> >> #endof
> >
> >
> > We need to support it for VS as well, but it looks like win32 API has a
> > function to do it:
> > http://msdn.microsoft.com/en-us/library/ms724482%28VS.85%29.aspx
> >
> > Makes it even easier.
>
> Nice.  So all we would need is something like:
>
> try:
>     from ctypes import windll, wintypes
> except (ImportError, ValueError):
>     pass
> else:
>    has_feature = windll.kernel32.IsProcessorFeaturePresent
>    has_feature.argtypes = [wintypes.DWORD]
>    if not has_feature(10):
>        msg = ("This version of numpy needs a CPU capable of SSE2, "
>               "but Windows says - not so.\n",
>               "Please reinstall numpy using a superpack installer")
>        raise RuntimeError(msg)
>
> At the top of numpy/__init__.py
>
> What would be the best way of including that code in the 32-bit wheel?
>  (The 64-bit wheel can depend on SSE2).
>

Maybe write a separate file `_check_win32_sse2.py.in`, and ensure that when
you generate `_check_win32_sse2.py` from setup.py you only end up with the
above code when you go through the
        if len(sys.argv) >= 2 and sys.argv[1] == 'bdist_wheel':
branch.

Ralf



>
> Cheers,
>
> Matthew
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20140530/0e28ef7a/attachment.html>