[SciPy-dev] SciPy Foundation

Tue Aug 4 14:53:20 EDT 2009

At this point I think the question becomes: do we let the (clear) fact that there is not a single set of priorities for where SciPy should be headed (which I do not see as a bad thing at this stage) get in the way of the community moving on *some* proposal (e.g., Joe's, with mods) for *some* "not-for-profit entity" (e.g., a "SciPy Foundation," the original topic of this thread) that will function as an institutional resource for furthering whichever priorities for SciPy should bubble to the surface?  In other words, this thread is diverging (into territory necessary to discuss, yes), but can we at least agree (a semi-rhetorical question because I think the answer is clearly "yes") that something along the lines of a "SciPy Foundation" would be useful, certainly for helping us move SciPy where we want it to go, but perhaps also for helping us decide where as well?

DG

--- On Tue, 8/4/09, Sebastian Walter <sebastian.walter at gmail.com> wrote:

> From: Sebastian Walter <sebastian.walter at gmail.com>
> Subject: Re: [SciPy-dev] SciPy Foundation
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Tuesday, August 4, 2009, 2:25 AM
> On Tue, Aug 4, 2009 at 10:35 AM,
> David
> Cournapeau<david at ar.media.kyoto-u.ac.jp>
> wrote:
> > Sebastian Walter wrote:
> >> 2 cents from an outsider who thought about
> contributing to
> >> scipy/scikits (but didn't (yet)):
> >>
> >> I think it is a good idea to make scipy easy to
> use for beginners.
> >> However, after reading this thread, I have the
> impression that it is
> >> not the goal to provide state of the art
> algorithms but rather making
> >> Scipy as popular as possible by putting money and
> effort into the
> >> "marketing" of Scipy.
> >> Don't get me wrong, I think there are some good
> reasons why a project
> >> should thrive for a large user base. Some of the
> best projects are
> >> popular.
> >> Alas, correlation does not imply causality.
> >>
> >> Me for instance, would rather like to see more
> efforts to get state of
> >> the art algorithms to be implemented in Scipy
> because that's something
> >> that would make a real difference in my research
> work. Of course,
> >> targeting the "clueless Matlab" users is quite
> pointless if it is that
> >> what you are after.
> >>
> >
> > One point which has not been mentioned concerning
> matlab-like
> > environment - maybe it is obvious and everyone
> implicitly acknowledges
> > it, but Mathworks is a 30 years old company, with >
> 1000 people today.
> >
> > Building something like matlab, with a good GUI and
> top notch
> > documentation takes a huge amount of resources, of
> which the 'useful'
> > code is only a fraction. I of course don't know the
> details of matlab
> > implementation, but I know that for music oriented
> softwares (which need
> > good UI to sell well, and have non trivial
> computational requirements,
> > so the comparison is not totally stupid), the
> graphical code is 80 % of
> > the code. This ratio is consistent with the big open
> source audio
> > softwares as well (ardour, rosegarden). Worse, being
> cross platform
> > makes the problem much more difficult. For music
> softwares market, mac
> > os x is rarely ignored (~ 40-50% of the market I
> believe), so people
> > need to support two platforms, and that's really a lot
> of work. For
> > scientific software, I think you can go the non native
> route for the
> > graphical toolkit, though.
> >
> > Also, very few open source software are successful as
> far as good GUI
> > are concerned (I don't want to enter into a debate
> here, but there are
> > good documents/studies on this topic). You need
> financial incentive for
> > this, so only projects backed up by big companies
> managed to pull it of.
> >
> > IOW, I am pretty pessimistic about being a 'matlab'
> clone. We should
> > rather shoot for what makes numpy/scipy better
> (extensibility, cross
> > platform, actual language, etc...), because really,
> matlab will always
> > be a much better matlab than us. Price and licensing
> are not good enough
> > to justify migration - if what you want is a free
> matlab clone, why not
> > using octave or scilab anyway.
> >
> > That does NOT mean that we should not aim at making
> the software more
> > accessible. I (and I guess other developers) are
> definitely interested
> > in a more product-like, integrated stack, to make the
> barrier of entry
> > lower. I for example am really tired of the
> installation problems
> > consistently reported. I feel like we cover mac os x
> and windows pretty
> > well now, but the linux situation is still dreadful. I
> have a few ideas
> > on how to improve the situation, but they all requires
> quite a bit of
> > work/infrastructure. I hope that soon, the scenario "I
> see this cool
> > python script on the internet, it requires this
> numpy/scipy thing, can I
> > try it in 2 minutes ?" will be a reality.
> >
> >> Then you really get some "killer applications". I
> could name a few
> >> people who are coding some cool state of the art
> algorithms but waste
> >> so much time because they started coding directly
> in C++. In the
> >> meantime, they could have implemented the
> algorithms in Python _and_
> >> in C++. If scipy had something really good that
> Matlab etc. do not
> >> have: guess what ppl would do....
> >>
> >
> > Yes, there are a lot of people who still don't know
> that there are
> > languages outside Fortran, C and C++. In my field, I
> still see some
> > people who implement parsers in C...
> >
> >> 1) an easy, modular and flexible build system
> (fortran, c, c++, D,
> >> swig, boost:python, cython,...)
> >>
> >
> > you mean like numscons :) ? Adding D support to
> numscons should be easy.
> > For example, I added initial cython support in a
> couple of minutes
> > during the cython talk at SciPy08, adding new
> languages is relatively
> > easy thanks to scons.
> >
> >> 2) very low entry barrier for possible
> contributors:
> >>   a simple checkout, then 
> ./manage.py startapp  mycoolmodule
> >>   and everything is ready to go (
> "Start coding in 5 minutes!")
> >>
> >
> > there are various pieces to enable this (in place
> build, develop command
> > of setuptools, virtualenv/pip/easy_install), but yes,
> the situation is
> > kind of messy. For scikits, that's not so
> difficult  - you should be
> > able to implement a trivial scikit by copying the
> scikits.example
> > package and starting from there.
> >
> > One problem is that it is technically impossible to
> build in place and
> > test in one go because of a nose limitation ATM (for
> some reason, nose
> > fails to import a package if it is in the current
> directory).
> >
> >> 3) a distributed version control system (e.g.
> git). SVN really scares me off...
> >>
> >
> > That's a sensitive issue, I think we should avoid
> starting this one here
> > :) Needless to say, you can use git-svn - several core
> developers use it
> > for numpy/scipy dev, and we distribute an official
> import:
> >
> > http://projects.scipy.org/numpy/browse_git
> >
> > At least I have not touched svn for numpy/scipy
> development for > 6
> > months now, except to check releases when I tag them.
> >
> >> 4) standardized unit tests
> >>
> >
> > What do you mean exactly here ? We use nose for
> testing, what do you
> > consider "non standard".
> >
> >> 5) automated documentation generation
> >>
> >
> > It is almost automated now - but an example for
> scikits is missing in
> > the example package :)
> >
> 
> Just enumerating what I think would be useful to attract
> high quality
> contributors.  I'm aware that scipy has already 
> a lot of the features
> (which is nice).
> But it would be even nicer to have a really low entry
> barrier and have
> a framework that guides you to write good (and documented)
> code with
> extensive unit tests, just like the big web frameworks
> (Django, RoR,
> ...)
> It has to be a win-win situation for both the community and
> the developer.
> 
> 
> > cheers,
> >
> > David
> > _______________________________________________
> > Scipy-dev mailing list
> > Scipy-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 