[SciPy-dev] [SciPy-User] SciPy Foundation

[David Cournapeau]

Hi Joe,

On Sat, Aug 1, 2009 at 2:06 AM, Joe Harrington<jh at physics.ucf.edu> wrote:

>
> I define success as popular adoption in preference to commercial
> packages.  I believe in vote-with-your-feet: this goal will not be
> reached until all aspects of the package and its presentation to the
> world exceed those of our commercial competition.  Scipy is now a
> grass roots effort, but that takes it only so far.  Other projects,
> such as OpenOffice and Sage, don't follow this model and do produce
> quality products that compete with commercial offerings, at least on
> open-source platforms.

I am not sure openoffice is a good example, but I share the sentiment
that something is missing in the organization of the community.

I think it is very important to keep in mind that in any open source
project, telling people what to do does not work well. Not everybody
will share the same goals, are interested in scipy in the same way,
etc...  So any structure should help people doing what they want for
scipy's sake, but above all, should not alienate anyone who would have
worked on scipy otherwise. It may just be rhetoric, but saying that
"it would be nice for scipy to have this goal" instead of "we should
do this" matters IMHO.

Some of the things I am missing:
 - no quantifiable feedback from users: if we want to work on a set of
features, we cannot prioritize.  Likewise, we have very little
statistics on usage, platforms, etc... OTOH, this is often hard to
obtain for open source projects.
 - a scipy foundation: several times already, I have been asked
privately to do add some feature to scipy, generally things which
takes a few hours max, in exchange for some money. It is too much of a
hassle to set up things to get money for a few hours work, and
frankly, for a few hours, I would prefer to ask people to give money
to a scipy foundation instead. Something like the R foundation
(http://www.r-project.org/foundation/main.html). A foundation with a
legal status would make the situation much easier w.r.t donations I
believe. It should not be that hard to set up.
 - website: I think the root of the problem is lack of a dedicated
person for it, a person with design skills ideally, to design a
coherent graphic "chart" (not sure about the exact English word),
etc... I don't know how to get volunteers for this: it seems like many
projects manage to have such volunteers.

About the more particular points you raised:

> - Packaging
>  - Personal Package Archive or equivalent for every release of every
>    OS for the full toolstack (There are tools that do this but we
>    don't use them.  NSF requires Metronome - http://nmi.cs.wisc.edu/
>    - for funding most development grants, so right now we're not even
>    on NSF's radar.)
>  - Track record of having the whole toolstack installation "just
>    work" in a few command lines or clicks for *everyone*
>  - Regular, scheduled releases of numpy and scipy
>  - Coordinated releases of numpy, scipy, and stable scikits into PPA system

The problem of packaging is that it is hard to do well, but has no
technically challenging part in it. And it usually does not fall into
the "scratching ones' itch", because once you know how to build the
software, you are done and usually want to start using the damn thing.
Worse, it needs to be done every-time (every release). So this is
fundamentally different than doc: having done a great packaging work
for version N is useless after N+1 is out. It does not make sense to
pay someone to do it once.

Having some infrastructure would help: for example, something which
automatically builds packages on a set of supported platforms. It has
to be 100 % automatic, so that pushing one button get you the sources,
build the package, install it, and test it. This costs money and time
to set up.

> - Public communication
>  - A real marketing plan
>  - Executing on that plan
>  - Web site geared toward multiple audiences, run by experts at that
>    kind of communication
>  - More webinars, conference booths, training, aimed at all levels
>  - Demos, testimonials, topical forums, all showcased

Concerning communication with users, I think that the mailing lists do
not work well. It is ok for development, but it kinda sucks for
helping average users. Since I have been working on the dark side for
numpy/scipy- windows, I have been regularly using stackoverflow to ask
for some obscure windows stuff. stackoverflow is a a mix between a FAQ
and wikipedia. It works extremely well, and the user experience is way
above anything I have seen in this vein. Something like this to use
for scipy/numpy would be extremely useful I believe. It is vastly
superior to ML or wiki for focused problems ("how to do this in
matlab", "how to install on this linux distribution", etc...).

As an example of usage, R has recently used the main website so that
the most upvoted N R questions would be answered by R core developers
(during a R conference I believe). This all feels much better than ML
to me (again, as far as average user usage is concerned, not for
developer communication).

One website to handle all the user community, no need for complicated
forum rules and all (everything works with search and tags).
Stackoverflow works without any fixed hierarchy for many times more
participants that we will ever have, and much broader topics than us.

They will have soon a dedicated solution for custom websites using the
same stack - maybe something can be worked on as a open source
project.

David