[SciPy-dev] scipy and ATLAS (in)dependency

[eric jones]

Perry Greenfield wrote:

>Pearu Peterson wrote:
>
>  
>
>>Hi,
>>
>>Today I was reading "Special Session: Making Python attractive to
>>General Scientists (Harrington, Greenfield)" notes in
>>
>>   http://www.scipy.org/wikis/scipy04/ConferenceSchedule
>>
>>and I was a bit surprised on one of the conclusions that maybe ATLAS
>>optimization should be undone due to difficulties in building ATLAS.
>>
>>Though, IMHO, building recent versions of ATLAS libraries is not
>>difficult
>>at all on Linux platforms and not even on MS Windows (there are
>>step-by-step instructions available on Scipy site), it just may be a very
>>time consuming process ;).
>>It's not even an issue for Mac as Scipy uses its vecLib framework.
>>I can't say much on the situation on other unix platforms such as irix,
>>sun, etc due to the lack of access to such platforms. But most of current
>>and potential Scipy users are either on MS Windows, Linux, or Mac anyway..
>>
>>But that was not the point I was surprised on. It was acctually the fact
>>that people seem to be unaware of the possibility to build Scipy without
>>ATLAS dependency by using Fortran sources of BLAS and LAPACK libraries.
>>Let me stress that nothing in Scipy requires specifically ATLAS
>>libraries,
>>the corresponding interface in scipy.linalg is smart enough to pick up
>>ATLAS optimized routines when available and use Fortran BLAS/LAPACK
>>routines when they are not.
>>
>>My point is that there is (almost) nothing to do to "undo ATLAS
>>optimization" in Scipy. ATLAS is optional already. However, when ATLAS is
>>not available then Scipy needs BLAS/LAPACK libraries that currently must
>>be provided by the system or users must download them from
>>Netlib. I think
>>that BLAS/LAPACK libraries are the only external libraries that Scipy
>>currently depends on.
>>
>>To get rid of this dependency, I'd suggest include the sources of
>>BLAS/LAPACK libraries to Scipy, and use them silently when optimized
>>BLAS/LAPACK libraries are not available. This would be very similar to
>>scipy.fftpack that silently uses Fortran fftpack sources when FFTW
>>libraries are not found.
>>
>>Just wanted to clear some things up..
>>Pearu
>>
>>    
>>
>As one of the notetakers I'll admit I don't have much personal
>experience with the issue so I could have gotten the conversation
>wrong. But I think the gist was that some felt that there should be
>easy-to-install binary packaging for all popular platforms, not
>easy-to-build packaging (though of course one hopes that is also
>available). If having an optimized ATLAS got in the way of that
>goal, then it was argued that it would be better to have a slow version
>of the linear algebra libraries in the easy-to-install binaries.
>Those that needed the optimized versions could build it themselves.
>The stress was on making it very painless for people to install
>the general setup on their machines and that meant binary installers.
>If the conclusion that optimized atlas made that harder (either
>on the people doing the binary packaging--thus it never happens--
>or the people doing the installing) is wrong, then I should correct
>the notes. But I think that was the point of the comments.
>  
>
I remember that the discussion mainly focused on binary packages also.  
It isn't that we should remove ATLAS from the options available to 
people building from CVS, but that ATLAS makes building RPMs harder to 
do cleanly (in an automated way) and also calls for many more versions 
of installers for win32.  I think our approach has been to provide 
Pentium III optimized ATLAS in the Python -- Enthought Edition that we 
make available.

Joe Harrington took a pole to see how many people would object to have a 
single generic version of SciPy on the web site for each primary 
platform instead of multiple optimized binaries.  80-90% of the room 
thought this was a good idea to reduce the pain on the people 
maintaining packages.

So:
1.  I don't think we should remove that ATLAS from the system, but it 
might be worth while to make it more obvious (on web pages or whatever)  
to people building from CVS that ATLAS isn't required. 

2. Adding LAPACK/BLAS to the repository would add a lot of code to the 
repository.  I'm not all together opposed to this, but I'm not sure it 
is that great of an idea either. 90% of serious users will link to a 
more optimized version (even if it is a generic version of ATLAS).  This 
pushes me to -0 on the idea.

3. Building a single binary per platform for download from SciPy using a 
generic version of ATLAS is a reasonable trade-off between maintenance 
headache and performance (with "reasonable" here defined by a community 
pole).

thanks,
eric

>Perry
>
>
>_______________________________________________
>Scipy-dev mailing list
>Scipy-dev at scipy.net
>http://www.scipy.net/mailman/listinfo/scipy-dev
>  
>