[Tutor] Documentation concerns.

[Magnus Lyckå]

First of all: It might be that the best Python documentation is
a printed book like Beazley's Python Essential Reference or
Martelli's Python in a Nutshell. I think that's fine, and not
really a problem. Buying even half a dozen of Python books is
cheap if you do serious development.

At 20:01 2003-05-23 -0300, Jorge Godoy wrote:
>So, if there's the logical place, why some documentation is on the
>module itself?

The standard library has always (well, for at least seven years,
I don't know beyond that) been the main documentation for Python
modules and packages (since they appeared).

Due to the introspection facilities in python, stuff like doc-strings
and tools like pydoc and happydoc etc, it's become more common to see
a lot of information that can be extracted from the source code.

It's quite possible that we will se a union between the library
reference text and the module doc strings in the future. The Python
docutils project is developing a standard for writing plain text
in a format that will look pretty in HTML or PDF etc format. This
is used for PEPs today, and might be used both for Python manuals
and code comments in the future, thus making it possible to create
the library reference manual from the source code of the modules.

Several people are even writing entire books in the Docutils text
format reStructuredText today.

It will take yet some time before this transition can start though,
and it will take years before it's completed, so please get used to
the library reference. I will be the best source of information for
some years.

>I think that the module is the logical place and the Python Library
>Reference is the *second* logical place to look at. I already have the
>module.

You should have the library reference as well! It's only when you
sleep it should be under your pillow. In the day time it should
be on your desk! ;)

>If I have some application, I certainly have the module but it's not
>guaranteed that I'll have net access or a copy of the Python Library
>Reference with me.

Only if you have a broken Python installation.

>My IDE is Emacs + several xterms. :-) Using pydoc is really great. As
>is perldoc, texdoc, etc.

You can view your library reference in Emacs or in the xterms
as well. It's probably a good habit to get used to that.

>We were discussing ways to improve the documentation --- at least
>that's what I understood from prior messages --- and having complete
>docs with the module including explanations on simple use --- maybe
>even complex use --- of it would greatly enhance the productivity.

Sure. As I said, we might see this in a few years. At least for
new modules, but in the cases where there are mature modules with
the currently best docs only in the library ref, I'm not sure it
will be converted to the source anytime soon.

But perhaps someone will write a script to convert the current Latex
documentation source to reStructuredText. In that case, that could
just be pasted into the source code.

There are problems here though:

Unless you run fully optimized python (python -O -O) you will carry
the bulk of this extensive documentation in your code. It will both
affect startup time and memory usage. And if you do run -O -O, you
will obviously lose the online help.

This is of course a difference from docs in Java or C++ source code
that never gets through the compiler.

>I was not --- this time ;-) --- looking for the docs (you see, even
>having read --- actually I just passed my eyes on everything, but I
>read whole parts of it --- he whole Python Library Reference I still
>have to look at or question for the places where docs are).

Always have the Global Module Index available in a browser window!
(Perhaps you should have it as start page in your browser? :)

And as I say now and then in this mailing list: Reread chapter 2
sometimes. It's the most important part of the library ref.

> > It's possible that we'll see more and more of the documentation
> > being generated from the source code, but often, the ideal structure
> > of the implementation is not the ideal structure of documentation of
> > a module, so it's probably good to have separate documents.
>
>Standardizing places to find docs is something every project
>needs. Python has done that.
>
>Unfortunately, putting every single modules documentation in a website
>is not what I think is ideal.

In a website? Surely you have the docs included in your Python
installation? Otherwise I think it's broken. I certainly have all
the docs in both Linux and Windows installations. At least in my
Mandrake 9 it's in /usr/share/doc/python-docs-2.2.1/

If you *do* have web access, you might also find
http://starship.python.net/crew/theller/pyhelp.cgi useful.

>Having the docs with the module makes
>them look like part of the code --- IMHO they are as important as the
>code itself --- and makes the author of the module update them with
>every change he does in the code.

If that was true we would never see comments that are out of
sync with the code. ;) And as I mentioned before, I'm not at all
sure that the module author is always the right person to write
the documentation. Besides, only a fraction of the code changes
need to be reflected in the documentation needed to use the
module.

Actually, a problem with this kind of internal documentation
is that it's important to differentiate between the external
interface of a module and how that's documented, and the
implementation and the documentation of that. It's important
that programs that use a module only rely on it's formal
interface.

>I'm reading a lot of the docs I could find. There are lots of things
>at Python's website but there are lots of things spread all around.

Well, there are a lot of third party modules, and they are certainly
documented in various ways. Obviously, there is additinal docs about
all aspects of Python in web sites, books, mailing lists, etc, but I
don't see that that differes from any other language.

Perl has CPAN, and the Python Package Index is far from a CPAN yet.
See http://www.python.org/pypi

>And, you can bet that if I could I'd change several
>standard/non-standard modules documentation to add examples and make
>some information available with the source code. If the documentation
>needed is too big, adding a message like 'For complete documentation
>on this module look at http://somewhere.over.the.net/modules-doc'.

Most 3rd party modules come with docs in HTML format. Not all though.
I hope Docutils / reStructuredText will help make this more uniform.

>Sometimes it takes a body of new people to point out something that
>people are already used to see and use can be improved. I wouldn't
>discard any opinions even if I had to point them out the docs that say
>why the way it is now is better than the proposed way. All have to
>learn with such discussions: the ones wanting to change and the ones
>saying that changing is not needed.

:) I know. I'm currently trying to push for some changes in both
documentation and implementation of the Python DB-API, and there
are many people who don't want to change the tings that they have
gotten used to, even if I suggest changes that will make all the
current code work as before, while new code would be more uniform
and the learning curve smoother for new developers.

In general I think the Python developers are open to improvements,
but they are not able to put a lot of time into this. They do depend
on help from the community.

Right now, the Docutils project is working with the tools, and it's
coming along rather fine, but I don't know how much of the work is
directly focused on fixing the python code comments and manuals though.
Personally I'm more interested in reStructuredText and Docutils for
web site texts and article production.

>You certainly have more python experience than I do. I was just
>looking at it in the last two years, I haven't written much code in it
>and then, when I started doing such a thing, I found out that there's
>a HUGE difference in the documentation style used in modules. I don't
>know, yet, if Perl's way is much better than Python's way on that
>issue, but it certainly is more practical to work with, after all, as
>you pointed out, typing 'help(module)' is much easier than opening a
>browser, going to the python library reference or doing a search on
>the web...

Well, if I remember correctly, my first email correspondance
with Guido, about six years ago or so, was my complaints that
the docs weren't as convenient as Perls... :)

I think my main complaint was with the language reference
manual, and I eventually realized than you're not supposed
to read that! ;)

A lot has changed in both Perl and Python since then. I
guess it was somewhat faster for me to get used to the Perl
docs, and most of the time I just used Johan Vromans quick
reference guide, but I realize that the things I needed that
guide for are things that was soon completely obvious in
Python, and the areas where I felt the docs could be better
in Python were often in areas that I never managed to do
master in Perl... The big difference for me, is that with
Perl I need the docs much more... :)

We also have to remember that there is a lot of important
Python documentation that don't fit in a a source module.

I guess the library reference content could have it's
source with the respective packages and modules.

The tutorial, extending and embedding and python/C api
will probably be separate entities like today though.
I guess the Python/C API guide could be integrated with
the C source, but I don't know if that would make any
practical difference.

The Language Reference manual is perhaps the real problem
child. I remember having problems with it initially. It's
labeled as "for language lawyers" and is largely a spec,
written partly in BNF, but at the same time, this *is* the
reference guide for non-module things like operators and
statements of various sorts. If you want something which
is more reference than the tutorial, you have to look here,
and end up with a very formal text.

Perhaps a separate specification and users lanugage reference
would be an improvement. Although, as I started, books like
Beazley's or Martelli's cover these things nicely, and maybe
we don't need everything for free. After all, it's good if
there is a market for printed books! :)

If nothing else, exposure in book stores and at places like
Amazon is probably important for the growth of Python. God
knows there aren't a lot of big Python advertisments in the
trade rags.


--
Magnus Lycka (It's really Lyckå), magnus@thinkware.se
Thinkware AB, Sweden, www.thinkware.se
I code Python ~ The shortest path from thought to working program