[SciPy-dev] Scipy for Numarray

[Pearu Peterson]

On Wed, 22 Sep 2004, Charles Harris wrote:

> Hi All,
> 
> I've been thinking about getting started on a version of scipy for
> numarray. It's going to be a big job, so I have questions to put to the
> community.
>
> 1. How should it be set up in cvs? There is currently a scipy module to
> be checked out, should we have a numarray_scipy module at the same
> level?
> 
> 2. Where should it install? To avoid name conflicts, I think it might be
> reasonable to install it, or its components, in the numarray packages
> directory.
> 
> 3. There is stuff like Pycrust and ipython that is not directly related
> to scipy, should it be taken over also, or perhaps broken out into a
> scipy addons module.
> 
> 4. There is a lot of infrastructure for the installation and testing of
> scipy. I am not very familiar with that, so how does it need to be
> modified to install a numarray version of scipy.
> 
> 5. How can we keep both versions updated?
> 

Let me not give direct answers to your all specific questions but give
comments on the issue that hopefully will answer the questions.

First, there have been some off-list discussion about numarray integration
with scipy among developers of both projects and at the moment the plan
seems to be to provide scipy installation that has two versions of
extension modules that are built against Numeric and numarray,
respectively. When importing scipy then only one version of extension
modules will be imported based on the state of some external flag,
e.g. environment variable (imho, using env. variable is reasonable under
unix system but how this sounds for Windows users? Alternative ideas?).

Second, we would like to avoid creating numarray branch of scipy if
possible. Otherwise, merging the Numeric and numarray branches in future
might turn out to be quite a painful task.

On the implementation of the plan:

1) An extension module, say foo, will be built into three files:
   _foo_na.so - foo built against numarray
   _foo_np.so - foo built against Numeric
   foo.py     - Python module that imports one of the above modules
This task should be doable on the top level of the scipy sources
(in setup.py and scipy_distutils) without touching scipy subpackages
at all. And f2py already supports numarray.

2) All scipy subpackages should explicitely import and use
scipy_base functions instead of Numeric. Currently there are number of
scipy subpackages that need to be fixed in this respect. The task should
be trivial to do.

3) The most tricky part of getting numarray+scipy to work seems to
be building scipy_base against numarray as scipy_base extension modules
are very much Numeric specific.

As I see it, the order of implementation should be 3), 2), and 1).
Though, 2) is an independent step from others but without 3), the step 1)
makes little sense.

Pearu