[Distutils] Handling the binary dependency management problem

[Chris Barker]

On Wed, Dec 4, 2013 at 12:56 PM, Ralf Gommers <ralf.gommers at gmail.com>wrote:

> The problem is explaining to people what they want - no one reads docs
> before grabbing a binary.
>

right -- so we want a default "pip install" install that will work for most
people. And I think "works for most people" is far more important than
"optimized for your system"

 How many non-sse machines are there still out there? How many non-sse2?
>>
>
> Hard to tell. Probably <2%, but that's still too much.
>

I have no idea how to tell, but I agree 2% is too much, however, 0.2% would
not be too much (IMHO) -- anyway, I'm just wondering how much we are making
this hard for very little return.

Anyway, best would be a select-at-runtime option -- I think that's what MKL
does. IF someone can figure that out, great, but I still think a numpy
wheel that works for most would still be worth doing ,and we can do it now.


Some older Athlon XPs don't have it for example. And what if someone
> submits performance optimizations (there has been a focus on those
> recently) to numpy that use SSE4 or AVX for example? You don't want to
> reject those based on the limitations of your distribution process.
>

No, but we also don't want to distribute nothing because we can't
distribute the best thing.

 And how big is the performance boost anyway?
>>
>
> Large. For a long time we've put a non-SSE installer for numpy on pypi so
> that people would stop complaining that ``easy_install numpy`` didn't work.
> Then there were regular complaints about dot products being an order of
> magnitude slower than Matlab or R.
>

Does SSE by you that? or do you need a good BLAS? But same point, anyway.
Though  I think we lose more users by people not getting an install at all
then we lose by people installing and then finding out they need a to
install an optimized version to a get a good "dot".


>
>> Yes, 64-bit MinGW + gfortran doesn't yet work (no place to install dlls
> from the binary, long story). A few people including David C are working on
> this issue right now. Visual Studio + Intel Fortran would work, but going
> with only an expensive toolset like that is kind of a no-go -
>

too bad there is no MS-fortran-express...

On the other hand, saying "no one can have a 64 bit scipy, because people
that want to build fortran extensions that are compatible with it are out
of luck" is less than ideal. Right now, we are giving the majority of
potential scipy users nothing for Win64.

You know what they say "done is better than perfect"

[Side note: scipy really shouldn't be a monolithic package with everything
and the kitchen sink in it -- this would all be a lot easier if it was a
namespace package and people could get the non-Fortran stuff by
itself...but I digress.]

 Note on OS-X :  how long has it been since Apple shipped a 32 bit machine?
>> Can we dump default 32 bit support? I'm pretty sure we don't need to do PPC
>> anymore...
>>
>
> I'd like to, but we decided to ship the exact same set of binaries as
> python.org - which means compiling on OS X 10.5/10.6 and including PPC +
> 32-bit Intel.
>

no it doesn't -- if we decide not to ship the 3.9, PPC + 32-bit Intel.
binary -- why should that mean that we can't ship the Intel32+64 bit one?

And as for that -- if someone gets a binary with only 64 bit in it, it will
run fine with the 32+64 bit build, as long as it's run on a 64 bit machine.
So if, in fact, no one has a 32 bit Mac anymore (I'm not saying that's the
case) we don't need to build for it.

And maybe the next python.org builds could be 64 bit Intel only. Probably
not yet, but we shouldn't be locked in forever....

-Chris



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20131204/a3951228/attachment.html>