[Numpy-discussion] Newbie Question, Probability

[Robert Kern]

Travis Oliphant wrote:
> I'm always confused about how to distribute something like SciPy for the 
> MAC.  What exactly should be distributed?   Is it possible to use 
> distutils to get it done? 

To get a package format that is actually useful (bdist_dumb just doesn't cut it
on any platform, really), you need to install something else. I prefer building
eggs instead of mpkgs. So this is what I do:

Install setuptools. I then create a script (I usually call it ./be) in the
numpy/ and scipy/ directories to hold all of the options I use for building:


#!/bin/sh
pythonw2.5 -c "import setuptools; execfile('setup.py')" build_src build_clib
--fcompiler=gnu95 build_ext --fcompiler=gnu95 build "$@"


Then, to build an egg:

  $ ./be bdist_egg

You can then upload it to the Package Index (maybe. I had trouble uploading the
Windows scipy binary that Gary Pajer sent me. I suspect that the Index rejected
it because it was too large).

Here are the outstanding issues as I see them:

* Using scipy requires that the FORTRAN runtime libraries that you compiled
against be installed in the appropriate place, i.e. /usr/local/lib. This is
annoying, since there are currently only tarballs available, so the user needs
root access to install them. If an enterprising individual wants to make this
situation better, he might try to make a framework out of the necessary
libraries such that we can simply link against those. Frameworks are easier to
install to different locations with less hassle.

  http://hpc.sourceforge.net/

* g77 does not work with the Universal Python build process, so we are stuck
with gfortran.

* The GNU FORTRAN compilers that are available are architecture-specific. For
us, that means that we cannot build Universal scipy binaries. If you build on an
Intel Mac, the binaries will only work on Intel Macs; if you build on a PPC Mac,
likewise, the resulting binaries only work on PPC Macs.

* In a related problem, I cannot link against ATLAS on Intels, and possibly not
on PPCs, either (I haven't tried building with a Universal Python on PPC). The
Universal compile flags (notably "-arch ppc -arch intel") are used when
compiling the numpy.distutils test programs for discovering ATLAS's version.
Using a single-architecture ATLAS library causes those test programs to not link
(since they are missing the _ATL_buildinfo symbol for the missing architecture).
I've tried using lipo(1) to assemble a Universal ATLAS library from a PPC-built
library and an Intel-built library, but this did not change the symptomology.

Fortunately, part of ATLAS is already built into the Accelerate.framework
provided with the OS and is automatically recognized by numpy.distutils. It's
missing the C versions of LAPACK functions, so scipy.linalg.clapack will be
empty. Also, I think numpy.distutils won't recognize that it is otherwise an
ATLAS (_ATL_buildinfo is also missing from the framework), so it may not try to
compile the C BLAS interfaces, either.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco