[SciPy-dev] Scipy Tutorial (and updating it)

[Pauli Virtanen]

Thu, 04 Dec 2008 20:16:24 -0500, jh wrote:
> Jarrod wrote:
> 
>> I would also prefer changing the name from "Tutorial" to "User Guide".
> 
> I disagree, for two reasons.  First, we are badly diluting our doc
> effort and will fail to complete *any* documents to a user's
> satisfaction with this approach.  We need to close the scipy doc wiki

I do not believe that preventing people from editing scipy docstrings 
will improve numpy documentation in any way; it will only prevent point 
fixes to Scipy docstrings (eg. people becoming annoyed with a specific 
bad docstring, and fixing it). We can say instead that "concentrate on 
Numpy", strongly, on the front page, and remove scipy links from the 
table on the front page.

What in my opinion could help Numpy documentation now is coordination. 
For example via milestones:

- Compile a list of functions; sort it appropriately, and parcel it out
  to manageable chunks (say, tens of items).

- One possible way to parcel these could be to work in the order the
  functions currently appear in the current reference guide.

- I'd include a couple of "unimportant" functions to each milestone,
  so that we don't end up with dull milestones at the end of the run.

- Ask editors to work from top to bottom, one parcel at a time. Ditto for
  reviewers.

- I wouldn't necessarily include any deadlines, as these will slip,
  which is demotivating.

This should be fairly easy to compose, just requires putting up a new 
wiki page with appropriate links and sections for this. This could also 
speed up the review and QA that I feel is now not really progressing.

> Second, a (good) user manual takes a level of planning and commitment
> that has not gone into any of our documents so far.

I don't think it's fair to say that no planning has been done. The 
content of the Numpy reference guide is based on Travis's work, and I 
streamlined it by editing out the user-guidish parts.

Criticism is of course welcome, but it's more helpful to be specific.
I have seen little to no feedback on this.

> This tutorial
> document cannot grow into a good user manual, because documents *never*
> improve with organic growth.  They must be planned and written with
> scope in mind from the outset.  We should reserve the name "User Manual"
> for something we plan and execute to be good with that scope.
>
> To expand (only a little, I promise!) on the User Guide, it needs
> chapters on specific numpy topics, starting with the text in the new
> numpy.doc module, and it needs chapters on general topics like plotting,
> 3D visualization, handling datasets, scientific data formats, etc. 
> Heavy-duty programming chapters belong in a separate document (NumPy
> Programmer's Guide?).  Advanced tricks need to be limited in the User
> Guide or it will be far too long.  It could go in its own guide, or
> might better go in a web cookbook.

I believe, like Gaël, that the "Scipy User Guide" (or Tutorial, whatever 
we choose to call it) can be more restricted in scope than what you 
describe above. Basically, we need only narrative documentation that 
explains background on what each of the parts does, gives motivation why 
and where it is useful, and gives examples how the functionality can be 
used that are more complete than those in the function docstrings.

It is probably much easier to write this type of a document for Scipy 
than for Numpy, since the functionality in Scipy splits naturally to 
parts along submodule boundaries, and each of the submodules concentrates 
on a single topic. What you say above about planning of course applies 
inside each of these topics, but the problem is already less difficult, 
at least for the more narrowly focused pieces (e.g. `scipy.odr`). For the 
larger modules, like interpolate, the problem seems to split according to 
different groups of functionality.

But yes, for each of the parts of Scipy, someone needs to have a vision 
of how the part of the document should be organized, and lay out the 
section structure. The current push for Scipy was to make use of material 
that was already written, in time for 0.7.0. We can maybe put off the 
push for tackling general documentation for the present, but I wouldn't 
object if someone already wants to stub out some of the Scipy submodules 
-- the fact that the writer must know the module well already limits the 
number of people to a small team, I'd believe.

> We are a grass-roots community, but good books are not, and have never
> been, written by a haphazard grass-roots effort, with everyone writing
> whatever they feel like writing.  The best you can hope for is to scope
> and plan as a community, set goals for specific docs and sections of
> docs, form small groups of writers, parcel out pieces with specific
> goals to different groups, and then have a small editing team work with
> the chapter teams to unify the pieces.  This is a standard process in
> academic publishing and it works well.  Other than the
> small-team-does-it-all model, I'm not aware of any other successful
> authoring modes.  Grass roots works on the doc wiki only because we've
> provided strict guidelines and process, and each page is done by just
> one or a few people.

I completely agree with this. Adding random snippets one by one to the 
wiki will not work well for the narrative documentation, so let's not do 
it like that. Maybe we can ask people to first stub out the complete 
structure for a part, and recommend point edits only after the structure 
is in place.

-- 
Pauli Virtanen