[SciPy-dev] Ideas for scipy.sparse?

[Robert Kern]

On Fri, Apr 11, 2008 at 4:41 PM, Brian Granger <ellisonbg.net at gmail.com> wrote:
> > >  2) I need these things to be in numpy.  I hate to start another
>  >  >  "should this go into numpy or scipy" thread, but I actually do think
>  >  >  there is a decent case for moving the core sparse arrays into numpy
>  >  >  (not the solvers though).  Please hear me out:
>  >  >
>  >  >  a) Numpy at its core is about arrays.  Conceptually, sparse arrays fit
>  >  >  into this narrow vision of Numpy.
>  >
>  >  Previously, Travis has stated a desire to get some form of standard
>  >  sparse array (or possibly just matrix) support into numpy 1.1 for
>  >  precisely this reason. I happen to agree. However, I have to address
>  >  the following points.
>
>  I did not know that.  It would definitely have to wait until 1.1.
>
>  >  >  b) Sparse arrays are just as foundational as dense arrays in many
>  >  >  areas of computing/science (I would argue, that they are more
>  >  >  foundational than ffts and random numbers).
>  >
>  >  The parenthetical is not a relevant argument.
>  >  numpy.{linalg,fft,random} exist because of history, not design. In
>  >  order to convince people to move from Numeric to numpy, we *had* to
>  >  support a transition from the LinearAlgebra, FFT, and RandomArray
>  >  modules that were distributed with Numeric.
>
>  I understand this, but that doesn't mean we can't make future
>  decisions about what goes into NumnPy based on more principled ideas.

My point *was* that the decision to include them wasn't based on a
principled stance. Their presence is not evidence for any particular
principle about what should be in numpy. You cannot use them as a
reference point.

>  >  >  d) It would not make numpy more difficult to build.
>  >
>  >  A fair amount of the current sparse code uses C++ templates, so I will
>  >  have to say that this statement needs qualification. The impact may be
>  >  low, but it is not negligible. If we rewrite it not to use C++, then
>  >  we don't know how much more difficult it will be. We will need to
>  >  evaluate that when the code is written. The only thing that we can be
>  >  sure won't make numpy more difficult to build is pure Python code.
>
>  At some level this is true.  C++ template are nice in principle, but
>  not always so in practice because of build issues.  We would just have
>  to see, but I am hopeful.  On the other hand, if the build problems
>  were really that much of a problem, they would be equally problematic
>  in scipy.

C++ alone makes it more difficult. For a long time we have held firm
on only including C in numpy. Yes, the C++ does make scipy more
difficult to build, but the standards for acceptable difficulty are
different for scipy than numpy.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco