[SciPy-user] Newbie question on execution time of import scipy

[Pearu Peterson]

Hi,

On a Debian Linux with PII (400 MHz) I see similar behaviour to Travis's
report:

pearu at localhost:~$ python
import time
Python 2.2.1 (#1, Jul 29 2002, 23:15:49) 
[GCC 2.95.4 20011002 (Debian prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> s=time.time();import scipy;print time.time()-s
13.2945020199
>>> 
pearu at localhost:~$ python
Python 2.2.1 (#1, Jul 29 2002, 23:15:49) 
[GCC 2.95.4 20011002 (Debian prerelease)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> s=time.time();import scipy;print time.time()-s
2.07088291645
>>> 

In the first import I also hear lots of harddisk activity while in the
second try none of that, probably all libraries are in memory cache
then. So, these excessive scipy import times actually measure the speed of
harddisks, not CPUs. And the import time may vary depending on
available RAM, whether shared libraries were loaded before (e.g. debian
provides shared ATLAS libraries), etc.

Most of the time for scipy import is taken probably by scipy.linalg
which contains the largest extension modules. If your applications require 
mostly double precision routines then the size of linalg extension modules
can be reduced by setting 

skip_single_routines = 1

in the header of linalg/setup_linalg.py (notice also comments there).

However, now I that have tested building scipy.linalg against shared and
static atlas libraries, with and without skip_single_routines being set,
I am surprised that I still cannot draw conclusions which combination
is most efficient: the results vary too much on subsequent tests
and in average the timing results seem to be the same.

It seems the states of memory and cache as well as the speed of
harddisk are the key factors that determine actual scipy import time,
the CPU speed has a secondary importance.

In any case, the scipy import time 2-13 seconds is a killer for certain
tasks, this is also one of the reasons why I have commiting so much 
patches to get scipy modules usable as standalone.

The only solution to get a speed up in scipy import would be to start
using lazy import techniques. However, such techniques are usually highly
experimental and 'from .. import ..' type of imports (that scipy modules
extensively use, unfortunately) make implementing lazy import hooks
harder.

Regards,
	Pearu