[Doc-SIG] Python docs in reST

[Fred L. Drake, Jr.]

On Wednesday 25 May 2005 17:08, Felix Wiemann wrote:
 > I'm not sure what you mean.  Basically, the "include" *literally*
 > inserts the contents of the included file into the input stream.

I just mean that this should be a node insertion; header identification should 
be completely independent of what underline characters are used in the 
context document.  This may already be the case; I've simply not checked 
that.

 > If you want to load not only chapters but the *whole* Python
 > documentation into a document, you'd end up with unacceptable parse

The Library Reference is currently parsed all at once; that's the bulk of the 
documentation by far.  The time to generate HTML for that is already long.  
If a ReST version of the document could be processed in approximately the 
same length of time, that's good enough.  A better system would be welcome, 
and likely not too hard.

 > times.  To fix this, you'd need to parse all input files (without
 > resolving references etc.) and store the results (namely Docutils node
 > trees) in XML files.  Later, when creating the big document, the XML

Maybe XML, maybe not.  Depends on how open the system should be internally.  
I'd certainly like it to be open, and keeping it open to extensibility is not 
a bad thing in my book.

 > files can be read and assembled, which should probably not take very
 > much time.  Disadvantages:
 >
 > * Need to use "make" or implement make-like functionality to regenerate
 >   XML files only when necessary.

If you're trying to avoid processing effort, you need some way to determine 
what can't be avoided.

If we have a system that "cooks" each input document into a node tree and 
references to included documents, it shouldn't be hard to knit things 
together.  This only matters if the actual parse is slow; anything else will 
be relatively quick.  Loading structure takes time too, regardless of whether 
the structure is encoded in XML or as a Python pickle.  If there's a 
significant parse time (and I think there is), we need to work on the 
performance of the parser.

(Has anyone specifically looked at this yet?  If so, it would be interesting 
to hear of any specific issues that were found and still need attention.)

 > > but needs to be done before other transformations.
 >
 > What other transformations?

Things like generating page-local Tables of Contents come after, similar to 
things like generating the final form of the links.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>