[SciPy-dev] attacking import failure from scipy source tree

[Pearu Peterson]

On Mon, 7 Oct 2002, Skip Montanaro wrote:

> 
>     Travis> It sounds like you may be trying to import scipy from a python
>     Travis> session started in your build directory.  Try switching to
>     Travis> another dir (like ~/ or /) and begin the python session and try
>     Travis> an import.  I won't tell you how many times I have to force
>     Travis> myself to remember this each week.
> 
> I get bitten by this occasionally as well.  It's mildly irksome that I need
> to install scipy to test it.  Has anyone debugged this problem?

Yes, I have also thought about this since I started to use scipy.

Possible solutions:

1) If all scipy modules (at least those modules that contain extension
modules) would be in some subdirectory, say lib or src, of scipy source
tree, then the problem would be solved. But this would require extensive
file moving in scipy CVS repository. I am not sure if scipy CVS server can
handle this; limited disk space could be one obstacle.

2) Importing scipy_base is first that will fail when importing scipy from
its source tree. So, we could catch
  import fastumath
failure and give some additional information why it failed. This
solution can be quickly applied but it would be still nicer if scipy tests
could be run without installing scipy first.

3) Extend the local module testing hooks. The scipy modules that I have
been working with can be build locally (meaning: inside its source
directory) and also tested locally without the need to install scipy or
this particular module. Currently, only scipy_test and scipy_distutils
needs to be installed for these hooks to work. But I think that also this
requirement can be removed.
As for an example, currently linalg module can be tested as follows:
  cd scipy/linalg
  python setup_linalg.py build
  python tests/test_basic.py
  python tests/test_decomp.py
  python tests/test_blas.py
(last three commands can have also optional level argument)

We could extend these hooks so that something as follows would work:
  cd scipy/
  python setup_scipy.py build
  python setup_scipy.py test [level]  # run all tests using specified
                                      # level, default level is 1 
  python <modulename>/tests/test_<submodulename> [level]

On the other hand, I am not sure that the following should work be
default:
  cd scipy/
  python
  >>> import scipy
  >>> scipy.test()
because of possible complexity for required hooks. Note that inserting,
for example, build/lib.linux-i686-2.2/ to sys.path may not be
enough. I tried that, but it seems that python picks up modules first from
the current directory (if they exist there) without looking at sys.path
first. I also tried removing '' from sys.path but still "import scipy"
tries to import ./scipy_base instead of 
build/lib.linux-i686-2.2/scipy_base.

However, "import scipy" might work already if

  python setup.py build build_ext --inplace

is used for building scipy. But I am not sure that something will not go
wrong with this (I haven't tried it yet).

Pearu