[SciPy-Dev] win32 binaries and scipy 0.17.0 release

[Nathaniel Smith]

On Sat, Jan 2, 2016 at 4:01 PM, Tony Kelman <tony at kelman.net> wrote:
> Nathaniel Smith <njs <at> pobox.com> writes:
>
>> That's all great until the msys2 users type "pip install psycopg2" and
>> pip happily installs a package that was built against the standard
>> windows cpython ABI, and breaks in obscure ways when used from msys2
>> python.
>
> I'll have to check whether the msys2 pip is configured in a way that
> tries to use wheels from pypi, or always falls back to source builds.
> Both could be minefields for different reasons.

The proper solution would be for it to use a different platform tag
than standard cpython -- so instead of looking for wheels tagged win32
/ win64, it would look for wheels tagged win32msys / win64msys (or
whatever). But then you have the problem of convincing every project
to provide builds for both platforms, which ugh.

>> My feeling is that while there are some users who might be
>> happy with treating anaconda or whatever like a walled garden and
>> never going outside a small curated set of packages, for most users
>> what they want is the full python ecosystem. And that means
>> interoperating with that ecosystem.
>
> It's an interesting question. My impression is that anaconda wouldn't
> be as popular as it is if more of its users had strong needs to use
> compiled extensions not yet provided within anaconda.
>
>> Depends on whether you care about python or not :-).
>
> Hence my personal lack of investment on the MinGW-w64 CPython fork...
> The IJulia installation problem that I cared most about was solved by
> downloading and driving miniconda through a Conda.jl package.
>
>> Also one might reasonably want to get off of the 10 year old
>> msvcrt.dll runtime even without caring about MSVC, but YMMV.
>
> MinGW-w64 has shims around most of the trouble spots that have come
> up over the years. Things like the math library are why Julia uses
> its own independent libm (mostly taken from BSDs, but buildable
> standalone): https://github.com/JuliaLang/openlibm - feel free to try
> building that with mingwpy BTW, it might help. There have been issues
> with glibc's libm and I believe on Mac as well, so this isn't uniquely
> a Windows problem.

Unfortunately openlibm currently does not work very well if you're
using any compiler that follows the MSVC ABI, because the code pools
it draws from (like the BSDs) assume that the x87 FPU is configured in
"extended precision" mode (which gives you an extra 16 bits of
precision for intermediate calculations). This is a valid assumption
on every popular x86 ABI *except* MSVC. (This is why the mingwpy page
has a bunch of discussion of numerical precision issues -- mingw-w64's
runtime is similar to openlibm in this respect.) Fortunately, though,
it turns out that the bionic libc has a high-quality libm implemented
using standard precision SSE, though (hat tip to Sergey Maidanov at
Intel for pointing this out to us) -- you might want to check it out.

>> In any case, the point is just that mingw-w64 itself may well switch
>> to an MSVC-compatible configuration as the main supported target over
>> the next while.
>
> This is also likely to happen by way of LLVM before it happens with
> GCC. Clang, LLD, compiler-rt, and libc++ are rapidly gaining in
> Windows functionality, including the mingw-w64 developers working on
> a standalone configuration that builds against the core mingw-w64
> headers rather than needing to piggyback on an MSVC/WinSDK installation.
>
> As far as using the ucrt goes, has there been any more recent activity
> than http://sourceforge.net/p/mingw-w64/mailman/message/34124287/ ?

Not in public. The picture here should hopefully become much clearer
within the next few weeks. (Sorry for vagueness, it's just that I've
been pushing on various dominoes and don't know yet where they'll fall
:-).)

> Are the existing mingwpy patches public somewhere?

Not yet. That's "phase 1" in the proposal that was just funded
  https://mingwpy.github.io/proposal_december2015.html
and so I think Carl's going to start working on it, like, tomorrow or
something like that. Again, patience is needed, but hopefully not
*too* much patience :-).

-n

-- 
Nathaniel J. Smith -- http://vorpus.org