[SciPy-dev] Technology Previews and Sphinx Docstring Annotations

[Damian Eads]

I'm on the train so I'll be short. I agree with Anne's concerns.

There are lots of proposals being circulated for fairly extensive
community review panels, paperwork, and refactoring without any real
evidence of who is going to do the work. Many of us start writing new
SciPy packages simply because they are related to our own research.
Expecting huge participation toward a 1.0 release with a potentially
unpassable barrier to entry for new code is hard sell to me as a
developer.

I do want to remark I perused Anne's code on the branch, and was quite
impressed with it. The code was documented according to our RST
standards, the documentation was well-written, regression tests
covered many corner cases of the API, the code was written with what I
consider fine practice (I'm known at work for being a stickler for
cleanliness), and the API had a good design. I think it is a model
case for inclusion in SciPy as a tech preview.

I will say that crafting good API design, writing clean code,
developing good tests is damn hard and very time consuming. Expecting
us to follow an extensive review and voting period may just cause many
to give up and withdraw our packages from consideration. After all,
for many of us, this isn't our day job: we have research to do, papers
to submit, Ph.D.'s to defend, and grant proposals to write.

A package inclusion process that is more supportive than defeating
will encourage developers to make more general contributions such as
documentation clean-up, code refactoring, API restructuring, and bug
fixing. But I'm not so sure we have the resources at this time for a
massive revamp as has been proposed.

Damian

On 11/4/08, Anne Archibald <aarchiba at physics.mcgill.ca> wrote:
> 2008/11/4 Travis E. Oliphant <oliphant at enthought.com>:
>> Jarrod Millman wrote:
>>> I absolutely agree with the ideas presented about scikits and look
>>> forward to seeing the numerous scikits improvements.  I feel that I
>>> have gotten into a discussion where the counter argument to what I am
>>> proposing is something I strongly support.  I also feel that the
>>> counterargument doesn't directly address my concern; but it may be
>>> that I am simply perceiving a problem that no one else believes
>>> exists.
>>>
>> Let me make my point again.   I'm arguing that instead of scipy.preview,
>> let's just make a *single* scikit called scikit.preview or
>> scikit.forscipy or scikit.future_scipy or whatever.    This will create
>> some incentive to make scikits easier to install generally as we want to
>> get the future_scipy out there being used.
>>
>> I'm very interested, though, to hear what developers of modules that
>> they would like to see in SciPy but have not made it there yet, think.
>>
>> I'm very interested in the question of how do we make it easier to
>> contribute to SciPy.
>
> As a developer who has written the module that is sparking this
> discussion, if the route to inclusion in scipy were "make a scikit,
> maintain and distribute it until you get enough user feedback to judge
> whether the API is optimal, then move it fully-formed into scipy" my
> code would simply gather dust and never be included. I don't have the
> time and energy to maintain a scikit.
>
> If I were a user I wouldn't bother downloading and installing a scikit
> in the six different computing environments I have used this week,
> particularly since it uses compiled code (but no additional
> dependencies). Why should I expect others to be different?
>
> The question is really, how do we take tested, apparently
> production-ready code, and get it out there so users can get at it?
> The current approach is "put it in scipy and live with the API". One
> proposal is, put it in scipy - which is still quite unstable, API-wise
> - but marked as "new, will be stabilizing until the next release". The
> other proposal is organize a community - nobody has volunteered to do
> any actual work yet - to build, maintain, publicize and distribute
> scikits so that they are as accessible as scipy itself.
>
> Frankly, I think the effect of agreeing on the scikits proposal -
> whatever its theoretical merits - will be to leave things at the
> status quo, except possibly for a higher barrier to getting good code
> into scipy: "put it in a scikit", we will say, and into a scikit it
> will go (or not), where it will molder and rot, never to be included.
>
> If people want to make the scikits option reasonable, I suggest,
> instead of arguing on the mailing list, going out and getting things
> set up so it really is easy to add packages. *Then* come back and tell
> us how it's a better option than what we have now. In the meantime, we
> can either go on with the approach we have - live with occasional API
> breakage for new components of scipy - or spend an evening getting
> scipy.preview set up.
>
>
> Anne
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://projects.scipy.org/mailman/listinfo/scipy-dev
>

-- 
Sent from my mobile device

-----------------------------------------------------
Damian Eads                             Ph.D. Student
Jack Baskin School of Engineering, UCSC        E2-489
1156 High Street                 Machine Learning Lab
Santa Cruz, CA 95064    http://www.soe.ucsc.edu/~eads