[SciPy-dev] Re: [Numpy-discussion] Trying out Numeric3

[Pearu Peterson]

On Sun, 27 Mar 2005, Michiel Jan Laurens de Hoon wrote:

> Having a separate scipy_distutils that fixes some bugs in Python's distutils 
> is a design mistake in SciPy that we should not repeat in Numeric3. Not that 
> I don't think the code in scipy_distutils is not useful -- I think it would 
> be very useful. But the fact that it is not integrated with the existing 
> Python distutils makes me wonder if this package really has been thought out 
> that well.

I don't think that part of scipy_distutils design was to fix Python's 
distutils bugs. As we found a bug, its fix was added to scipy_distutils as 
well as reported to distutils bug tracker. The main reason for adding bug 
fixes to scipy_distutils was to continue the work with scipy instead of 
waiting for the next distutils release (i.e. Python release), nor we could 
expect that SciPy users would use CVS version of Python's distutils. Also, 
SciPy was meant to support Python 2.1 and up, so the bug fixes remained 
relevant even when the bugs were fixed in Python 2.2 or 2.3 distutils.
So much of history..

> As far as I can tell, scipy_distutils now fulfills four functions:
> 1) Bug fixes for Python's distutils for older Python versions. As Numeric3 
> will require Python 2.3 or up, these are no longer relevant.
> 2) Bug fixes for current Python's distutils. These should be integrated with 
> Python's distutils. Writing your own package instead of contributing to 
> Python gives you bad karma.
> 3) Fortran support. Very useful, and I'd like to see them in Python's 
> distutils. Another option would be to put this in SciPy.fortran or something 
> similar. But since Python's distutils already has a language= option for C++ 
> and Objective-C, the cleanest way would be to add this to Python's distutils 
> and enable language="fortran".
> 4) Stuff particular to SciPy, for example finding Atlas/Lapack/Blas 
> libraries. These we can decide on a case-by-case basis if it's useful for 
> Numeric3.

Plus I would add the scipy_distutils ability to build sources on-fly 
feature (build_src command). That's a very fundamental feature useful
whenever swig or f2py is used, or when building sources from templates or 
dynamically during a build process.

Btw, I have started scipy_core clean up. The plan is to create the 
following package tree under Numeric3 source tree:

   scipy.distutils - contains cpuinfo, exec_command, system_info, etc
   scipy.distutils.fcompiler - contains Fortran compiler support
   scipy.distutils.command - contains build_src and config_compiler
     commands plus few enhancements to build_ext, build_clib, etc
     commands
   scipy.base - useful modules from scipy_base
   scipy.testing - enhancements to unittest module, actually
     current scipy_test contains one useful module (testing.py) that
     could also go under scipy.base and so getting rid of scipy.testing
   scipy.weave -
   scipy.f2py  - not sure yet how to incorporate f2py2e or weave sources
     here. As a first instance people are assumed to download them to
     Numeric3/scipy/ directory but in future their sources could be added
     to Numeric3 repository. For Numeric3 f2py and weave are optional.
   scipy.lib.lapack - wrappers to Atlas/Lapack libraries, by default
     f2c generated wrappers are used as in current Numeric.

For backwards compatibility, there will be
   Packages/{FFT,MA,RNG,dotblas+packages from numarray}/
and
   Lib/{LinearAlgebra,..}.py
under Numeric3 that will use modules from scipy.

Pearu