[SciPy-user] Re: Future directions for SciPy in light of meeting at Berkeley

[Joe Harrington]

These were exactly the issues we addressed at SciPy04, and which led
to the ASP project.  All of the issues brought up in the current
discussion have already been discussed there, and with largely the
same conclusions.  The basic gist is this:

THERE ARE THOUSANDS OF PEOPLE WAITING FOR SCIPY TO REACH CRITICAL MASS!

SciPy will reach the open-source jumping-off point when an outsider
has the following experience: They google, find us, visit us, learn
what they'll be getting, install it trivially, and read a tutorial
that in less than 15 minutes has them plotting their own data.  In
that process, which will take less than 45 minutes total, they must
also gain confidence in the solidity and longevity of the software and
find a supportive community.  We don't meet all the elements of this
test now.  Once we do, people will be ready to jump on and work the
open-source magic.

The goal of ASP (Accessible SciPy) is to meet that test.  Some of what
we need is being done already, but by a very small number of people.
We need everyone's help to reach a meaningful rate of progress.  The
main points and their status:

1. Resolve the numeric/numarray split and get at least stubs for the
   basic routines in the Python core.  Nothing scares new users more
   than instability and uncertainty.  Travis O. is now attempting to
   incorporate numarray's added features (including much of the code
   that implements them) into numeric, and has made a lot of headway.
   Perry G. has said that he would switch back to numeric if it did
   the things numarray does.  I think we can forsee a resolution to
   this split in the calendar year IF that effort stays the course.

2. Package it so that it's straightforward to install on all the
   popular architectures.  Joe Cooper has done a lot here, as have
   others.  The basic stuff installs trivially on Red Hat versions of
   Linux, Windows, and several others (including Debian, I think, and
   Mac, modulo the inherent problems people report with the Mac
   package managers, which we can do nothing about).  Optimized
   installs are also available and not all that difficult,
   particularly if you're willing to issue a one-line command to
   rebuild a source package.  For Linux, it was decided to stick with
   a core and add-on packages, and to offer umbrella packages that
   install common groups of packages through the dependency mechanism
   (e.g., for astronomy or biology).  The main issue here is not the
   packaging, but the documentation, which is trivial to write at this
   point.  I was able to do a "yum install scipy" at SciPy04, once I
   knew where the repository was.  It's:
   http://www.enthought.com/python/fedora/$releasever
   We need someone to write installation notes for each package
   manager.  We also need umbrella packages.

3. Document it thoroughly for both new and experienced users.  Right
   now what we have doesn't scratch the surface.  I mean no offense to
   those who have written what we do have.  We need to update that and
   to write a lot more and a lot else.  Janet Swisher and several
   others are ready to dig into this, but we're waiting for the
   numeric/numarray split to resolve.  A list of needed documents is
   in the ASP proposal.

4. Focus new users on a single selection of packages.  The variety of
   packages available to do a particular task is both a strength and a
   weakness.  While experienced people will want choice, new users
   need simplicity.  We will select a single package each application
   (like plotting), and will mainly describe those in the
   tutorial-level docs.  We will not be afraid to change the selection
   of packages.  You're only a new user once, so it will not affect
   you if we switch the docs after you've become experienced.  For
   example, Matplotlib was selected at the SciPy04 BoF, but if Chaco
   ever reaches that level of new-user friendliness, we might switch.
   Both packages will of course always be available.  Neither needs to
   be in the core on Linux and other systems that have package
   management.  New users will be steered to the "starter" umbrella
   package, which will pull in any components that are not in the
   core.  Enthon will continue to include all the packages in the
   world, I'm sure!

5. Provide a web site that is easy to use and that communicates to
   each client audience.  We (me, Perry, Janet, Jon-Eric) were
   actually gearing up to solicit proposals for improving the site and
   making it the go-to place for all things numerical in python when
   Travis started his work on problem #1.  This is the next step, but
   we're waiting for item 1 to finish so that we don't distract
   everyone's attention from its resolution.  Many developers are
   interested in contributing here, too.  If people feel it's time, we
   can begin this process.  I just don't want to slow Travis and his
   helpers one tiny bit!

6. Catalog all the add-ons and external web sites so that scipy.org
   becomes the portal for all things numeric in python.  This, at
   least, is done, thanks to Fernando Perez.  See:

	http://www.scipy.org/wikis/topical_software/TopicalSoftware

I'll add one more issue:

7. Do something so people who use SciPy, numeric, and numarray
   remember that these issues are being worked, and where, and how to
   contribute.  To that end, all I can do is post periodically about
   ASP and encourage you to remember it whenever someone wonders why
   we haven't hit critical mass yet.  Please visit the ASP wiki.  Read
   the ASP proposal if you haven't, sign up to do something, and do
   it!  Right now, a paltry 6 people have signed up to help out.

	http://www.scipy.org/wikis/accessible_scipy/AccessibleSciPy

   The ASP proposal is linked in the first paragraph of the wiki.

After giving it some thought, we decided to use scipy-dev at scipy.net as
our mailing list, to avoid cross-posted discussions on the 4 mailing
lists.  Please carry on any further discussion there.

Thanks,

--jh--