[SciPy-dev] PATCH: scipy_distutils/system_info.py

[Pearu Peterson]

David,

On Fri, 24 May 2002, David M. Cooke wrote:

<snip>

> Note that the static_first=1 (which should be 'static libraries are
> preferred') would generate a link line like '-L/usr/lib/3dnow
> -L/usr/lib/3dnow/atlas -llapack -lf77blas -lcblas -latlas'. However,
> this will link in the *shared* versions of these libraries, as the
> linker, if there are static and shared libraries available that
> satisfy the link line, will preferentially select the shared library.
> 
> The static_first=0 situation will have a link line like
> '/usr/lib/3dnow/atlas/liblapack.so /usr/lib/3dnow/libf77blas.so ...'
> which (I just tried it out) works for linking in shared libraries, but
> looks weird and isn't the way it's usually done.

I agree that static_first=1 did not work as expected but it worked as
designed (see below).

> Now, with my patch, static_first=1 will use for the link line
> '/usr/lib/3dnow/atlas/liblapack.a /usr/lib/3dnow/libf77blas.a ...'
> which links in static versions, and static_first=0 will use
> '-L/usr/lib/3dnow/atlas -L/usr/lib/3dnow -llapack -lf77blas -lcblas
> -latlas'
> which will link in shared versions (or static versions, if the shared
> versions don't exist).

Since I don't have shared versions of atlas libraries (in fact, ATLAS
provides no support for that and users are in their own in building shared
versions of atlas), I get with static_first=0 and your patch:

atlas_info:
  FOUND:
    extra_objects = ['/usr/local/lib/atlas/liblapack.a',
'/usr/local/lib/atlas/libf77blas.a', '/usr/local/lib/atlas/libcblas.a',
'/usr/local/lib/atlas/libatlas.a']
    include_dirs = ['/usr/include']

which is not desired either because blas and lapack extension modules will
be very large, they will contain symbols that they do not need. Even if
there are no .so libraries, the proper inclusion of .a libraries would be
to use '-llapack -lf77blas -lcblas  -latlas' so that only needed
symbols get into extension modules.

> If you want to use a static library instead of a shared one (for speed
> reasons, maybe), you will need to specify the filename of the static
> library as an 'extra object file', so it will have to go in
> 'extra_objects'. Either that or specify an option to the linker to
> prefer static to shared libraries.
> 
> Be as it may, I prefer to use shared libraries, as only one copy of
> the library is in memory at one time, and my python modules don't
> suddenly become 5 megs in size...

I agree with you completely.

Initially static_first hooks were meant to be used only to affect the
search path of libraries when static and shared libriaries are in
different directories. Obviously this is not always the case and then
the use of 'extra_objects' was introduced. IMO this was a mistake because
1) it is hackish and 2) it does not work neither with or without the
patch. Note also that one cannot use linker flags to force using static
libraries because python extension modules _must_ be shared.

My suggestion is to stop using 'extra_objects' in static_first hooks and
even if static_first=1 is specified, the shared libraries are preferred
(by the system, not by scipy_distutils [*]) in the case .so and .a files
are in the same directory. If people really want to force static
libraries, they can collect (the links of) static libraries into a
separate directory and add it to site.cfg. I don't think that scipy should
do that for users that do not like default behaviour of their system. What
do you think?

[*] Fighting with the default behaviours of systems that cannot be tested
directly is difficult. IMHO, by default scipy_distutils should use default
settings of the system and let the users fight with the system if they
insist, scipy_distutils can only provide some weapons (mainly site.cfg) to
ease their task.

Pearu