[SciPy-Dev] Design of scipy.stats

Tue May 25 17:39:01 EDT 2010

On Tue, May 25, 2010 at 1:53 PM, <josef.pktd at gmail.com> wrote:

> On Tue, May 25, 2010 at 4:26 PM, David Goldsmith
> <d.l.goldsmith at gmail.com> wrote:
> > On Tue, May 25, 2010 at 1:08 PM, <josef.pktd at gmail.com> wrote:
> >>
> >> On Tue, May 25, 2010 at 3:52 PM, David Goldsmith
> >> <d.l.goldsmith at gmail.com> wrote:
> >> > On Tue, May 25, 2010 at 11:04 AM, Robert Kern <robert.kern at gmail.com>
> >> > wrote:
> >> >>
> >> >> On Tue, May 25, 2010 at 12:47, David Goldsmith
> >> >> <d.l.goldsmith at gmail.com>
> >> >> wrote:
> >> >> > On Tue, May 25, 2010 at 8:54 AM, Ralf Gommers
> >> >> > <ralf.gommers at googlemail.com>
> >> >> > wrote:
> >> >>
> >> >> >>>  Plus they're not really unimportant.
> >> >> >>>
> >> >> >>> Why not?  They're not returned by Python's help system.  (See
> >> >> >>> earlier
> >> >> >>> this same thread.)
> >> >> >>
> >> >> >> Compare "help(alpha)" with "print alpha.__doc__" - help() is
> broken
> >> >> >> imho.
> >> >> >
> >> >> > Care to elaborate? ("Where I come from" help is the proper idiom -
> >> >> > I've
> >> >> > never heard of anyone using, nor being told to use, print .__doc__;
> >> >> > your
> >> >> > "ho" - that help is broken - strikes me as pretty strange).
> >> >>
> >> >> help() is broken in this case because it does not display the
> .__doc__
> >> >> attribute that is present. Yes, help() is idiomatic. That doesn't
> mean
> >> >> its behavior in this case is not broken. Nor does it mean that the
> >> >> text in the .__doc__ attributes are Unimportant for the docstring
> >> >> editing workflow. They just happen to be unavailable to the web
> >> >> docstring editor for technical reasons.
> >> >
> >> > Perhaps the origin of this thread has been forgotten:
> >> >
> >> > help(alpha) returns:
> >> >
> >> >>>> help(alpha)
> >> > Help on alpha_gen in module scipy.stats.distributions object:
> >> >
> >> > class alpha_gen(rv_continuous)
> >> >  |  ## Alpha distribution
> >> > :
> >> > :
> >> >
> >> > help(alpha_gen) returns:
> >> >
> >> >>>> help(alpha_gen)
> >> > Traceback (most recent call last):
> >> >   File "<stdin>", line 1, in <module>
> >> > NameError: name 'alpha_gen' is not defined
> >>
> >> do you have alpha_gen in your namespace?
> >> >>> from scipy import stats
> >> >>> help(stats.distributions.alpha_gen)
> >>
> >> >
> >> > But (in the Wiki) scipy.stats.distributions.alpha is "full," whereas
> >> > scipy.stats.distributions.alpha_gen is empty (but for automatically
> >> > generated content): help and the doc Wiki appear to exhibit the
> >> > "transpose"
> >> > of one another - I don't know "which one" is "broken" in this
> situation,
> >> > but
> >> > obviously something is.
> >>
> >> Most _gen don't have docstrings in the source, but are fully
> >> automatically generated docs
> >> An example for the new template based docstring is
> >> http://docs.scipy.org/scipy/docs/scipy.stats.distributions.maxwell_gen/
> >>
> >> I'd rather not have wholesale editing of distribution docstrings,
> >
> > So basically, "hands off" scipy.stats.distributions?  All of scipy.stats?
> > Would you like me to go through and mark everything as unimportant (I
> don't
> > think you can rely on an email thread - be it this one or a new one - as
> an
> > adequate safeguard)?  At the end of this road, is
> > help(scipy.stats[.*[.*[.*]]]) going to behave the same as
> > help(scipy.<X>[.*[.*[.*]]]), <X> != stats?
>
> this is only for the distributions, I just worry about reviewing 90
> cut and paste docstrings.
>

I understand your concern, but I think a "natural" extension of "Document a
single concept once<http://docs.scipy.org/numpy/Front%20Page/#document-a-single-concept-once>"
should suffice, i.e., keep the doc of each individual class to the minimum
necessary to comply w/ the Standard and note any peculiarities, and put all
the shared content in the docstring for stats.distributions (with a See also
reference pointing there in each class docstring).

> The instances of the distributions should never be edited,

Good, I'm glad we cleared that up.

> and I hope
> eventually we can block this.

Even better would be to have pydocweb not "see" them at all.

> I would consider the distribution
> classes as very low priority until the dust settles.

What "dust"?  Are these classes "moving targets"?

> So as
> preliminary measure all distributions instances and classes (except
> rv_continuous and rv_discrete) could be marked as unimportant (that's
> roughly 180 docstrings in the editor).
>

Whew, I thought I was going to have to revert all those Unimportants I
bestowed on all the instances; I'm going to defer doing that to the classes
until I hear what this "dust" is.

> The descriptions of the distributions in the tutorial could be
> reorganized, e.g. broken up into smaller pieces, as we discussed
> before on the mailing lists. But I would consider this also lower
> priority, compared to the main documentation in scipy.
>

Agreed: IMHO, as long as there are deficient (but "important") docstrings in
any sub-package, everything else is a second-tier priority.  (Of course, if
all that remains is outside one's comfort zone, then one can proceed to the
second-tier stuff within their comfort zone.)

>
> Scipy.stats, outside of the distributions, can be treated like any
> other scipy sub-package and I appreciate any contribution in improving
> those docstrings. I'm pretty slow in working my way through those.
>

Good to know, thanks!

Sometime within the next couple of days I'll "codify" the results of this
thread in the Wiki's "Q + A" section.

Thanks for your patience everyone - I (for one) feel satisfied w/ where
we've arrived at. :-)

DG

> Thanks,
>
> Josef
>
>
>
>
> >
> > DG
> >
> >>
> >> because it is too much to maintain, until we have established that any
> >> new pattern really works. The recipe how distributions work, can be
> >> explained generically without repeating variations on it in 90
> >> different ways.
> >>
> >> The pdf, cdf, ... formulas are attached to the scipy.stats tutorial,
> >> based on the original documentation by Travis.
> >>
> >> Josef
> >>
> >>
> >> >
> >> > At what level is help broken: the scipy level or the Python level?  If
> >> > help
> >> > is simply an alias for print .__doc__, why precisely is this
> happening?
> >> > Is
> >> > this an undiagnosed breakage (in the sense that only the symptom(s),
> but
> >> > neither the cause(s) nor the cure(s) are known)?
> >> >
> >> > Looking back on this thread, since <X> is the thing users are intended
> >> > to
> >> > use, it sounds like the docstring _should_ be associated with <X>, not
> >> > <X>_gen, so I guess the Wiki is showing the docstring in the "right"
> >> > place,
> >> > but why does help associate the docstring with <X>_gen?  And if the
> >> > docstring for <X> is consequently important, is the docstring for
> >> > <X>_gen
> >> > Unimportant (since its object "should" never be used explicitly)?  If
> >> > its
> >> > docstring is also important, please explain why.
> >> >
> >> > Thanks.
> >> >
> >> > DG
> >> >
> >> >>
> >> >> If you want to mark them
> >> >> Unimportant just as a workaround, that's fine. But if you are keeping
> >> >> track of items to work on this summer, these are important things to
> >> >> do. You will just have to do it manually.
> >> >>
> >> >> --
> >> >> Robert Kern
> >> >>
> >> >> "I have come to believe that the whole world is an enigma, a harmless
> >> >> enigma that is made terrible by our own mad attempt to interpret it
> as
> >> >> though it had an underlying truth."
> >> >>  -- Umberto Eco
> >> >> _______________________________________________
> >> >> SciPy-Dev mailing list
> >> >> SciPy-Dev at scipy.org
> >> >> http://mail.scipy.org/mailman/listinfo/scipy-dev
> >> >
> >> >
> >> >
> >> > --
> >> > Mathematician: noun, someone who disavows certainty when their
> >> > uncertainty
> >> > set is non-empty, even if that set has measure zero.
> >> >
> >> > Hope: noun, that delusive spirit which escaped Pandora's jar and, with
> >> > her
> >> > lies, prevents mankind from committing a general suicide.  (As
> >> > interpreted
> >> > by Robert Graves)
> >> >
> >> > _______________________________________________
> >> > SciPy-Dev mailing list
> >> > SciPy-Dev at scipy.org
> >> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >> >
> >> >
> >> _______________________________________________
> >> SciPy-Dev mailing list
> >> SciPy-Dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> >
> >
> > --
> > Mathematician: noun, someone who disavows certainty when their
> uncertainty
> > set is non-empty, even if that set has measure zero.
> >
> > Hope: noun, that delusive spirit which escaped Pandora's jar and, with
> her
> > lies, prevents mankind from committing a general suicide.  (As
> interpreted
> > by Robert Graves)
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> >
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>

-- 
Mathematician: noun, someone who disavows certainty when their uncertainty
set is non-empty, even if that set has measure zero.

Hope: noun, that delusive spirit which escaped Pandora's jar and, with her
lies, prevents mankind from committing a general suicide.  (As interpreted
by Robert Graves)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20100525/12f411d1/attachment.html>