[Doc-SIG] Python docs in reST

[Felix Wiemann]

Fred L. Drake, Jr. wrote:

> Felix Wiemann wrote:
>
>> Basically, the "include" *literally* inserts the contents of the
>> included file into the input stream.
>
> I just mean that this should be a node insertion; header identification should 
> be completely independent of what underline characters are used in the 
> context document.

Then you need a new "subdocument" directive.  The "include" directive
can't do that.

>> times.  To fix this, you'd need to parse all input files (without
>> resolving references etc.) and store the results (namely Docutils node
>> trees) in XML files.  Later, when creating the big document, the XML
>
> Maybe XML, maybe not.  Depends on how open the system should be
> internally.

If you want to store Docutils' node tree, XML is the most natural
choice.  That wouldn't change the openness of the system, and neither is
anybody expected to fiddle with XML files.  It's just a serialization of
an intermediate processing stage.

> This only matters if the actual parse is slow; anything else will be
> relatively quick.  Loading structure takes time too, regardless of
> whether the structure is encoded in XML or as a Python pickle.

Yes, but it's probably still much master than the reST parser, which
processes about 10 KB/sec at 1.4 GHz.

> If there's a significant parse time (and I think there is), we need to
> work on the performance of the parser.
>
> (Has anyone specifically looked at this yet?  If so, it would be interesting 
> to hear of any specific issues that were found and still need attention.)

>From my profiling results, much time is spent in the statemachine and in
the regular expressions.  Dunno how much this can be optimized, though.

>>> [output-splitting] needs to be done before other transformations.
>>
>> What other transformations?
>
> Things like generating page-local Tables of Contents come after, similar to 
> things like generating the final form of the links.

Thanks; we'll have to keep this in mind.

-- 
For private mail please ensure that the header contains 'Felix Wiemann'.

http://www.ososo.de/