[SciPy-dev] [Numpy-discussion] Unifying numpy, scipy, and matplotlib docstring formats

[Keir Mierle]

On 2/15/07, John Hunter <jdh2358 at gmail.com> wrote:
> On 2/15/07, Keir Mierle <mierle at gmail.com> wrote:
> > I would like to re-open this discussion to be sure there is consensus among the
> > numpy, scipy, and matplotlib folk before I invest signifcant time into
> > massaging the docstrings into the right form.
>
> I don't have a lot to add here because I did not follow the previous
> docstring discussions closely.  I generally accept that if Robert says
> something should be done a certain way, it's probably right and I
> don't want to step in front of a freight train.  That said, I think
> from a plain text perspective, your format and example look better
> than the markup for ReST ( I really liked the contour example).  Since
> I'll usually be reading docs through ipython or in the source code
> directly, I prefer documentation with as little explicit markup as
> possible, but respect that a significant chunk of users will be
> reading docs in html or PDF and the markup certainly helps there.
>
> I find
>
> :Parameter
>
> and friends ugly though I'm sure they look splendid after pushing them
> through ReST.   But I'm a softie and don't care enough about the
> intricacies of docstring formatting to make a nuanced argument which
> weighs all the details.  I'm probably happy to defer to whatever the
> numpy/scipy folks decide on along with you and the other developers
> who are pushing this.  Ie, I won't get in your way.

I will put my keyboard where my mouth is, i.e. write an epydoc parser.

> My guess is that you'll have to bear the brunt of the work, because we
> have a lot of priorities in terms of development of matplotlib to work
> on and a revamp of the docstrings is not at the top of the list.

If the entire documentation revamp must be done by me, then it will take some
time (I should be finishing my thesis!) Hopefully I can get some non-core
developers to particpate in a sprint, but maybe I am being too optimistic.

> There are two things I'd like to draw your attention to to see how
> they fit into the proposed scheme(s).
>
>   1) mpl uses a docstring formatting convention "ACCEPTS:' in all
> set_* matplotlib artist methods.  This supports object introspection,
> eg when you do
>
>   >>> line, = plot(...)
>   >>> setp(line)
>
>   We could do it another way, and I'm open to it, but right now we are
> using docstrings

I will have to think about this and read the code more before I can comment,
though I think this should be fine.

>   2) we use string interpolation to auto-generate repetitive parts of
> docstrings, eg the optional keywords that customize lines appear in
> many parts of the code.  See the kwdocd stuff in matplotlib.artist and
> matplotlib.lines and matplotlib.axes.Axes.plot for an example.  On
> first glance, this seems compatible with what you all are proposing,
> with some reformatting of course, but I could be missing something so
> please take a look at this.

This should be entirely compatible with the current scheme.

> As for your enhanced  pylab proposals, this is something that I think
> is very helpful and I would be happy to return the pylab
> name/functionality to the ipython/numpy/scipy/mpl/enthought community.

I have the pylab.net domain too.

> Travis originally owned this but at some point it wasn't doing
> anything more than  serving as a web page for a collection of his
> modules and I asked him if we could use it for the matlab-like
> environment incorporating matplotlib and Numeric.  At the time,
> matplotlib was pure python and trivial to insstall, and scipy was a
> bear to install, so I did not want a dependency on scipy.  Now things
> are very different.  matplotlib can be a bear to install, and the
> scipy install has gotten much better, and so a single unified package
> that brings in all three, rationalizes and factors out their common
> functionality etc, publicizes and maintains it, supports installs
> across platforms, etc.... would be a wonderful and major achievement.
> It is certainly doable, but plan to spend a significant part of next
> year working on it if you want to take this on.  We all have good
> intentions, but none of the core developers of any of the packages
> seems to have the time right now that such an integration effort
> entails.  We've talked about it many times at the last scipy meetings.

Unfortunatley the only platform I can do this on is Linux, because I don't have
access to windows or mac machines. I hear there is now a native GTK port for
mac which doesn't depend on X11, though it is not ready yet. GTK has worked
well on windows for some time.  What about switching to GTK as the prefered
multiplatform backend?

Ideally I would like to take all the work required to, for example, rationalize
and factor common functionality out, and push it all upstream. Ideall PyLab
would just be a single script which pulls down python/numpy/scipy/matplotlib,
and compiles into the final product, because the packages are inherently
designed to work together. This is probably impossible, but I can dream.

> Supporting easy installs of a comprehensive set of tools across
> platforms is a big job.  Not insurmountable, but make sure you
> understand what you are signing up for.  If you want to cut your teeth
> at cross platform installation and support, you are welcome to take on
> the limited task of supporting the matplotlib wx backend across
> platforms; this has been a thorn in our side for some time.

Personally I do not like wx at all. I am not (yet) volunteering to take this
on, because I certainly do not have the experience. The reason I set up the
pylab page is so that the community (or at the moment, myself) can identify not
just the sore points, but very specific next-physical-action type lists for how
to fix the issues.

For new contributors, it's hard to do something useful when the list of things
in need of improvement is, as someone once put it, an amorphous blob of
undoability.

> So your proposals look really nice, but the devil will be in the
> implementation.  Getting the core developers to agree on standards and
> a unified environment is a good and useful first step, but the harder
> part is getting the work done.

Certainly! This is why I am starting with the small initial step of deciding on
an excellent docstring standard (which I feel is extremely important), and will
forget about the future until that is at least mostly underway.

I hope the PyLab vision is possible, but it is a huge amount of work, and I
definitely will not be able to do it myself.

Keir