[Tutor] Documentation concerns.

[Jorge Godoy]

Magnus Lyck=E5 <magnus@thinkware.se> writes:

> First of all: It might be that the best Python documentation is a
> printed book like Beazley's Python Essential Reference or Martelli's
> Python in a Nutshell. I think that's fine, and not really a
> problem. Buying even half a dozen of Python books is cheap if you do
> serious development.

I agree with you on how good it is to have books to do serious
development. We in the southe hemisphere, with rare exceptions, have a
'small' problem: our money is a lot devalued when compared to
yours. Spending US$70,00 on a book is like spending a worker full
month paying. And books I'd like to have --- such as Python Bible ---
are more expensive than that...=20

I first must have some results --- small ones, of course, since I
don't have the 'library' yet --- then I can spend money to have bigger
results. We're a small company, yet. :-)

> At 20:01 2003-05-23 -0300, Jorge Godoy wrote:
>>So, if there's the logical place, why some documentation is on the
>>module itself?
>
> The standard library has always (well, for at least seven years, I
> don't know beyond that) been the main documentation for Python
> modules and packages (since they appeared).

It's been there since I first saw python, about three years ago --- my
first installation of it was two years ago. So, knowing where the
standard modules' docs are is not a real problem.=20

> Due to the introspection facilities in python, stuff like
> doc-strings and tools like pydoc and happydoc etc, it's become more
> common to see a lot of information that can be extracted from the
> source code.

And this is a big improvement in the documentation area. You can share
everything with your code and it is enough to have working
implementations. People can grab the source code, run 'pydoc
source-code' and see how to use it for their work. (And without ads as
in free websites ;-)).

> It's quite possible that we will se a union between the library
> reference text and the module doc strings in the future. The Python
> docutils project is developing a standard for writing plain text in
> a format that will look pretty in HTML or PDF etc format. This is
> used for PEPs today, and might be used both for Python manuals and
> code comments in the future, thus making it possible to create the
> library reference manual from the source code of the modules.

It would be great. And would allow us to have the documentation
whenever the source is available.

Please, note that I'm not willing to change the standard modules
documentation. I'm just willing to improve documentation by adding
aditional hints in the source code. It isn't needed to do that now.=20

Instead of having to do 'pydoc cgi' to look at some method
declaration/syntax and having to open a web browser to see how to use
it, we could just use 'pydoc' to have both. The thing is: "there's
always a Python interpreter running on your machine". There isn't
always a web browser and an Internet connection.

> Several people are even writing entire books in the Docutils text
> format reStructuredText today.

I've read a big argument about using ReST for the standard modules
documentation... :-) Coincidently it was started with a PEP suggesting
its use.

> It will take yet some time before this transition can start though,
> and it will take years before it's completed, so please get used to
> the library reference. I will be the best source of information for
                         ^
> some years.

I be you will :-) Just kidding. Yes, it wil be the best source of
information. But it can't prevent us from enhancing it. As I said, if
we had access and authorization to change standard libraries
documentation, I'd volunteer to add some things to the output I get
when I run 'pydoc'.=20

>>I think that the module is the logical place and the Python Library
>>Reference is the *second* logical place to look at. I already have the
>>module.
>
> You should have the library reference as well! It's only when you
> sleep it should be under your pillow. In the day time it should be
> on your desk! ;)

Hmmm... This might be the reason why it wasn't helping more! :-)))

>>If I have some application, I certainly have the module but it's not
>>guaranteed that I'll have net access or a copy of the Python Library
>>Reference with me.
>
> Only if you have a broken Python installation.

Some Linux distributions out there split their packages a lot and
allow the installation of packages without their docs. This is also
true on small systems with a limited amount of disk (where this issue
becomes critical in both ways: too much docs =3D too much disk used, too
little docs =3D too hard to have the docs when needed).=20

>>My IDE is Emacs + several xterms. :-) Using pydoc is really great. As
>>is perldoc, texdoc, etc.
>
> You can view your library reference in Emacs or in the xterms
> as well. It's probably a good habit to get used to that.

Yep. 'info python', 'info python-lib' and 'info python-api' are very
good and well known friends. I can access them with C-h i on Emacs.

Unfortunately, third part modules don't insert themselves on info's
index or at the standard library. They are spreaded on the system
(there's a logic on their installation, though).=20

> Sure. As I said, we might see this in a few years. At least for new
> modules, but in the cases where there are mature modules with the
> currently best docs only in the library ref, I'm not sure it will be
> converted to the source anytime soon.

I hope they are... But, for standard modules, I can live with that.=20

> But perhaps someone will write a script to convert the current Latex
> documentation source to reStructuredText. In that case, that could
> just be pasted into the source code.

LaTeX is, as pointed out by Guido on the thread I've spoken above,
much less verbose than XML and gives a much better output when
generating printed or screen reading documents. For screen reading,
HTML is enough, though. LaTeX -> HTML converters make a good job on
the Python documentation.=20

> There are problems here though:
>
> Unless you run fully optimized python (python -O -O) you will carry
> the bulk of this extensive documentation in your code. It will both
> affect startup time and memory usage. And if you do run -O -O, you
> will obviously lose the online help.

When developing an application it isn't a problem. When in production,
you can run fully optimized... This won't, them, affect production
systems except that they'll have '#!/usr/bin/python -O -O -tt' as the
first line of their code ('-tt' is an addition of mine it is already
present in the development stage ;-)).

> This is of course a difference from docs in Java or C++ source code
> that never gets through the compiler.

Maybe it's something we should inherit from their experience... I
think that Perl's docs have the same behaviour: they don't get through
the compiler.

> Always have the Global Module Index available in a browser window!
> (Perhaps you should have it as start page in your browser? :)

I have the info system open with them.

> And as I say now and then in this mailing list: Reread chapter 2
> sometimes. It's the most important part of the library ref.

OK. Except that 'chapter 2' isn't meaningful in info, where chapters
aren't numbered... Oh, yes! "2. Built-in Functions, Types, and
Exceptions". This is the one I thought when I looked at info...=20

> In a website? Surely you have the docs included in your Python
> installation? Otherwise I think it's broken. I certainly have all
> the docs in both Linux and Windows installations. At least in my
> Mandrake 9 it's in /usr/share/doc/python-docs-2.2.1/

I do have it here, but I can choose not to have them on machines with
a low capacity -- such as handhelds -- or on machines that are in
production. It saves --- at least my python-doc package do --- almost
2.6 MB of disk... Exactly 2612985 bytes.

> If you *do* have web access, you might also find
> http://starship.python.net/crew/theller/pyhelp.cgi useful.

I also like http://www.pythonware.com/library/

> If that was true we would never see comments that are out of sync
> with the code. ;) And as I mentioned before, I'm not at all sure
> that the module author is always the right person to write the
> documentation. Besides, only a fraction of the code changes need to
> be reflected in the documentation needed to use the module.

Comments out of sync with code reflect either a development stage or a
lack of attention from the coder himself.=20

I don't think the author would be the best writer, but he surely can
document the module and give one simple working example. And, if the
theory or documentation is too dense, he can put a note pointing
somewhere else.=20

> Actually, a problem with this kind of internal documentation is that
> it's important to differentiate between the external interface of a
> module and how that's documented, and the implementation and the
> documentation of that. It's important that programs that use a
> module only rely on it's formal interface.

And I'm not suggesting to have the module's documentation as comments
in the source. I'm suggesting having them as part of the
code. Something that can be extracted and printed on screen with
happydoc or pydoc.=20

> Well, there are a lot of third party modules, and they are certainly
> documented in various ways. Obviously, there is additinal docs about
> all aspects of Python in web sites, books, mailing lists, etc, but I
> don't see that that differes from any other language.

Neither do I. (see below)

> Perl has CPAN, and the Python Package Index is far from a CPAN yet.
> See http://www.python.org/pypi

I've already seen that.

CPAN is a great thing.=20

But, one additional thing here is that Perl include the doc with the
module. And it is also documented at CPAN: http://search.cpan.org/ ---
using the same example (the CGI module), take a look at
http://search.cpan.org/author/JHI/perl-5.8.0/lib/CGI.pm (this is
extracted from the module).

> Most 3rd party modules come with docs in HTML format. Not all
> though.  I hope Docutils / reStructuredText will help make this more
> uniform.

Me too. I'd rather have text documentation since I can use it
virtually anywhere without installing a browser or stripping the HTML
output (what isn't really hard...).=20

> :) I know. I'm currently trying to push for some changes in both
> documentation and implementation of the Python DB-API, and there are
> many people who don't want to change the tings that they have gotten
> used to, even if I suggest changes that will make all the current
> code work as before, while new code would be more uniform and the
> learning curve smoother for new developers.

God bless you. :-) I hope you can get changes --- if they are for
better --- approved.=20

> In general I think the Python developers are open to improvements,
> but they are not able to put a lot of time into this. They do depend
> on help from the community.

:-)
And then they should offer the community more resources.=20

I am proud of code I write, but I know other developers will make
improvements and/or additions to my code if they think it will help. I
just have to decide if I will accept them or not. This is why my email
is at each and every source file I write.

> Right now, the Docutils project is working with the tools, and it's
> coming along rather fine, but I don't know how much of the work is
> directly focused on fixing the python code comments and manuals
> though.  Personally I'm more interested in reStructuredText and
> Docutils for web site texts and article production.

:-) You're not a newbie, this is why. For website and articles I'd
take a look at
HTMLgen(http://starship.python.net/crew/friedrich/HTMLgen/html/main.html).
It's like Perl's Template Toolkit (http://www.tt2.org) and allow an
easy way to standardize your website look'n'feel without too much
trouble. I bet that getting PDFs with it is also easy and can be done
on the fly without much trouble (of course caching would make things
better, but files can be generated on demand and stored for some time
in the cache).

> Well, if I remember correctly, my first email correspondance with
> Guido, about six years ago or so, was my complaints that the docs
> weren't as convenient as Perls... :)

Heh...=20

> I think my main complaint was with the language reference manual,
> and I eventually realized than you're not supposed to read that! ;)

I use that as a reference, as the name says... I just looked at it and
read some parts from top to bottom. But, I haven't read it all.

> A lot has changed in both Perl and Python since then. I guess it was
> somewhat faster for me to get used to the Perl docs, and most of the
> time I just used Johan Vromans quick reference guide, but I realize
> that the things I needed that guide for are things that was soon
> completely obvious in Python, and the areas where I felt the docs
> could be better in Python were often in areas that I never managed
> to do master in Perl... The big difference for me, is that with Perl
> I need the docs much more... :)

I solved my problems very fast with it... And with maintenable and
readable code :-)=20

Python is easier, though, to have something working right from the
design table.=20

> We also have to remember that there is a lot of important Python
> documentation that don't fit in a a source module.

Of course! As I said, I'm not willing to put everything inside some
module. Just some examples and a more verbose explanation on how to
use the module.=20

> I guess the library reference content could have it's source with
> the respective packages and modules.

Hmmm... Dunno. It might be hard to make searches on specific parts of
it that way. The '-k' option should be enhanced too... maybe an index
of keywords, dunno.

> The tutorial, extending and embedding and python/C api will probably
> be separate entities like today though.  I guess the Python/C API
> guide could be integrated with the C source, but I don't know if
> that would make any practical difference.

I think it would cause more trouble since you'd have to have a tool
for extracting Python documentation from Python's source and Python
documentation from C source...=20

> Perhaps a separate specification and users lanugage reference would
> be an improvement. Although, as I started, books like Beazley's or
> Martelli's cover these things nicely, and maybe we don't need
> everything for free. After all, it's good if there is a market for
> printed books! :)

Indeed.

> If nothing else, exposure in book stores and at places like Amazon
> is probably important for the growth of Python. God knows there
> aren't a lot of big Python advertisments in the trade rags.

Maybe this is documentations fault or the lack of a centralized place
(such as CPAN) to search things... (There is a module list at python's
website, but I couldn't find it fast on the hyperlinks... maybe
because I'm not used to go there since the list is very small.)

--=20
Godoy.    <godoy@metalab.unc.edu>