[Doc-SIG] suggestions for a PEP

[Tavis Rudd]

--------------Boundary-00=_OHJ3M3E2ZQZ6GAK7DZGH
Content-Type: text/plain;
  charset="iso-8859-1"
Content-Transfer-Encoding: 8bit

Good morning,
I like Ka-Ping's suggestion of a formal PEP and have some made some notes 
summarizing the ideas I like from recent postings to doc-sig.  They could be 
written up as a PEP, along with a formal spec for Structured Text.  There's 
nothing truly original here, just a sythesis of other people's ideas:

1- module API documentation should be in the same file as source

2- a FORMALIZED version of structured text should be used for inline
     formatting.  There's no need to repeat the justifications here.
     The final version of structured text should include a facility for 
      storing meta-data in a field format that is easily identifiable to both 
      the human eye and the parsing tool.
      (e.g. authors, version, keywords, spam)

3- no changes should be required to the python parser

4- the module's namespace should not be polluted and it's memory 
     requirements should not be inflated by use of inline documentation

5- therefore, the existing __doc__ docstrings should be used for very short
     synopyses, and extended documentation that is discarded at the 
     the byte-compile stage should be written in string literals that appear
     immediately after the existing docstrings. These extra string literals 
     would be written in ST, while the __doc__strings would be in plain text.
     These two forms of API docs should complement and not duplicate each 
     other.

     See the example module attached to this message.

6- the documentation parsing tools should be capable of producing output in 
     many formats (manpages, plain text, html, latex, for a start), 

7- the doc parsing tools should not need to import and run the module to 
      produce it's documentation (for security reasons alone)

8- module Library Reference documentation should also be kept in the same
     file as the module source.  It should compliment the API docs with 
     examples, extended discussions of usage, tutorials, test code, etc., 
     but should not duplicate the API reference material.

9- the Library Reference docs should be written in string literals, as with   
     the extended API docs proposed in pt. 5, but there should be a prefix    
     token such as """LIBREF:  at the start of each chunk to signal to the 
     doc tools that the following text is not part of the API ref.  The token 
     would allow this documentation to be split up into chunks that can appear
     anywhere in the source file (a la perl's POD).

10- the Library Reference documentation should also be written in ST as 
      using LaTeX here would force the module author to learn yet another 
      mark-up language, require the documentation user to install yet another 
      processing tool (although this isn't an issue on Linux), and would place
      too much emphasis on the separation between the API and library   
      reference docs and discourage synchronization as the module evolves!
      The same argument applies to maintaining the status quo of external doc 
      files.

      Any extra meta-data that is needed for proper indexing, etc. 
      (to meet Guido's concerns) should be included as fields in the string 
      literals as is done in JavaDoc (but not neccessarily with that syntax).


What do you think?

Cheers,
Tavis Rudd

p.s. Other issues to consider:
- caching of documentation so it doesn't have to be regenerated 
   every time it's used
- documenting Packages
- inheriting documentation (Edward Loper's idea)
- hiding API docs for __privateInternals (ditto)
- documenting extensions in other languages
- comments within the markup language

--------------Boundary-00=_OHJ3M3E2ZQZ6GAK7DZGH
Content-Type: text/plain;
  charset="iso-8859-1";
  name="test.py"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="test.py"

IwojIE11bHRpbGluZSBjb21tZW50IGF0IHN0YXJ0IG9mIG1vZHVsZSAKIyAtIGNvdWxkIGJlIGlu
Y2x1ZGVkIGFzIHRoZSBtb2R1bGUgZGVzY3JpcHRpb24gaWYKIyAgIHRoZXJlIGlzIG5vIGRvY3N0
cmluZyAtIGUuZy4gYXMgcHlkb2MgY3VycmVudGx5IGRvZXMKIwoiIiJNb2R1bGUgRG9jc3RyaW5n
IiIiCiIiIgpleHRyYURvY3N0cmluZyBmb3IgdGhlIG1vZHVsZQoKTXkgcHJvcG9zYWwgaXMgdG8g
dXNlIHRoZSBleGlzdGluZyBfX2RvY19fIGxvY2F0aW9uIGZvciBhIG9uZS1saW5lIHN5bm9wc2lz
LCBhbmQKd2hlcmUgbmVlZGVkIHBsYWNlIGEgc2Vjb25kIHN0cmluZyBsaXRlcmFsIGFmdGVyIHRo
ZSBfX2RvY19fIGxvY2F0aW9uIGZvciBtb3JlCmRldGFpbGVkIEFQSSBkb2N1bWVudGF0aW9uIHRo
YXQgaXMgZm9ybWF0dGVkIGluIFNUcHkuICBSYXRoZXIgdGhhbiBpbnRyb2R1Y2luZyBhIG5ldwpf
X2Zkb2NfXyBhdHRyaWJ1dGUgaW50byBweXRob24sIGFzIEVkd2FyZCBMb3BlciBzdWdnZXN0ZWQs
IHRoZXNlIHN0cmluZyBsaXRlcmFscwpzaG91bGQgYmUgZGlzY2FyZGVkIGF0IHRoZSBweXRob24g
Ynl0ZS1jb21waWxlIHN0YWdlLiAgVGhlcmUncyBubyBuZWVkIGZvciB0aGlzCmRvY3VtZW50YXRp
b24gYXQgcnVuLXRpbWUgYW5kIHRoZXJlJ3MgY2VydGFpbmx5IG5vIG5lZWQgdG8gcmVxdWlyZSBj
aGFuZ2VzIHRvCnB5dGhvbidzIHBhcnNlci4KCkFzIHlvdSdsbCBzZWUgYmVsb3csIHRoaXMgd291
bGQgYWxzbyBhbGxvdyBmb3IgYXR0cmlidXRlIGRvY3N0cmluZ3MsIGEgbGEKTWFyYy1BbmRyZSdz
IFBFUCAoSSd2ZSBmb3Jnb3R0ZW4gdGhlIG51bWJlcikuCgoiIiIKIyBkb24ndCB1c2UgX19hdXRo
b3JfXywgX192ZXJzaW9uX18sIGV0Yy4gYXMgbW9kdWxlIGF0dHJpYnV0ZXMuCiMgVGhlcmUncyBu
byBuZWVkIHRvIHBvbGx1dGUgdGhlIG5hbWVzcGFjZS4gUmF0aGVyIGluY2x1ZGUgdGhlbQojIGFz
IGZpZWxkcyBpbiB0aGUgZXh0cmFEb2NzdHJpbmcKCgpNT0RVTEVfQ09OU1RBTlRfMSA9IDAKIiIi
ZXh0cmFEb2NzdHJpbmcgZm9yIGEgbW9kdWxlIGNvbnN0YW50IiIiCgpNT0RVTEVfQ09OU1RBTlRf
MiA9IDEKIiIiZXh0cmFEb2NzdHJpbmcgZm9yIGFub3RoZXIgbW9kdWxlIGNvbnN0YW50IiIiCgoK
ZGVmIGdsb2JhbEZ1bmMoYXJnKToKICAgICIiIkZ1bmN0aW9uIERvY3N0cmluZyIiIgogICAgIiIi
ZXh0cmFEb2NzdHJpbmcgZm9yIGdsb2JhbCBmdW5jdGlvbiIiIgoKICAgIGFWYXIgPSBhcmcqMiAK
ICAgICIiImV4dHJhRG9jc3RyaW5nIGZvciBhbiBleHByZXNzaW9uIHdpdGhpbiBhIGdsb2JhbCBm
dW5jdGlvbiAoaXMgdGhpcyByZWFsbHkgbmVlZGVkPz8pIiIiCiAgICBwcmludCBhVmFyIAogICAg
CgpjbGFzcyBTYW1wbGVDbGFzczoKICAgICIiIkNsYXNzIERvY3N0cmluZyIiIgogICAgIiIiZXh0
cmFEb2NzdHJpbmcgZm9yIGEgY2xhc3MKCiAgICAtIHRlc3Qgc3RyaW5nIHdpdGggYSB3aG9sZSBi
dW5jaCBvZiB0ZXh0LCB3aGljaCBjb3VsZCBiZSBmb3JtYXR0ZWQgd2l0aCBTVHB5LCBldGMuCiAg
ICAKICAgIFNOSVAgLS0tIFNldmVyYWwgdG9vbHMgaGF2ZSBiZWVuIHdyaXR0ZW4gb3IgcHJvcG9z
ZWQgZm9yIHByb2Nlc3NpbmcgUHl0aG9uIGRvY3VtZW50YXRpb24KICAgIHN0cmluZ3Mgd2l0aCBz
cGVjaWZpYyBmb3JtYXR0aW5nIGNvbnZlbnRpb25zIFsxXSwgWzJdLCBbM10sIFs0XS4gRXZlbnR1
YWxseSwgaXQKICAgIHdvdWxkIGJlIG5pY2UgaWYgc3VjaCBhIHRvb2wgYmVjYW1lIHBhcnQgb2Yg
dGhlIFB5dGhvbiBiYXNlIGxpYnJhcnkuIEhvd2V2ZXIsIGZvcgogICAgdGhpcyB0byBoYXBwZW4s
IHRoZSBjb21tdW5pdHkgbmVlZHMgdG8gY29tZSB0byBhIGNvbnNlbnN1cyBvbiB3aGF0IGZvcm1h
dHRpbmcKICAgIGNvbnZlbnRpb25zIGRvY3VtZW50YXRpb24gc3RyaW5ncyBzaG91bGQgdXNlLiBJ
biB0aGlzIGRvY3VtZW50LCBJIGRpc2N1c3Mgc29tZSBvZgogICAgdGhlIGlzc3VlcyB0aGF0IHN1
Y2ggY29udmVudGlvbnMgbXVzdCBkZWFsIHdpdGguIEkgYXR0ZW1wdCB0byBwcmVzZW50IGEgZmFp
cmx5CiAgICBjb21wcmVoZW5zaXZlIGxpc3Qgb2YgaXNzdWVzLCBidXQgSSB3aWxsIGJlIGdsYWQg
dG8gYW1lbmQgdGhpcyBkb2N1bWVudCBhcyBtb3JlCiAgICBpc3N1ZXMgYXJlIHBvaW50ZWQgb3V0
IHRvIG1lLiAgQmVjYXVzZSBJIHdhcyBpbnRlcmVzdGVkIGluIGV4cGxvcmluZyBzb21lIG9mCiAg
ICB0aGVzZSBpc3N1ZXMsIEkgd3JvdGUgbXkgb3duIGRvY3VtZW50YXRpb24gZXh0cmFjdGlvbiB0
b29sLCBFcHlkb2MgKGVkbG9wZXIncwogICAgcHlkb2MpLiBUaGlzIHRvb2wgd2FzIHByaW1hcmls
eSBpbnRlbmRlZCBhcyBhIHByb3RvdHlwZSwgdG8gbGV0IG1lIHBsYXkgd2l0aAogICAgZGlmZmVy
ZW50IHdheXMgb2YgcHJvY2Vzc2luZyBkb2N1bWVudGF0aW9uIHN0cmluZ3MuIEZvciBtb3JlIGlu
Zm9ybWF0aW9uLCBvciB0bwogICAgc2VlIHdoYXQgdHlwZSBvZiBvdXRwdXQgRXB5ZG9jIHByb2R1
Y2VzLCBzZWUgdGhlIGRvY3VtZW50YXRpb24gZm9yIEVweWRvYwogICAgKGtlZXBpbmcgaW4gbWlu
ZCB0aGF0IGl0J3MganVzdCBhIHByb3RvdHlwZSA7LSkgKS4gTWFueSBvZiB0aGUgaWRlYXMgY29u
dGFpbmVkIGluCiAgICB0aGlzIGRvY3VtZW50IGFyZSBub3QgbWluZS4gSSBkb24ndCB0YWtlIGNy
ZWRpdCBmb3IgYW55IG9mIHRoZW0uIEhvd2V2ZXIsIEknbSBub3QKICAgIG9yZ2FuaXplZCBlbm91
Z2ggdG8gZmlndXJlIG91dCB3aGVyZSBtb3N0IG9mIHRoZSBpZGVhcyBkaWQgY29tZSBmcm9tLgog
ICAgIiIiCgogICAgCiAgICBhbkF0dHJpYnV0ZSA9IDEyMzQKICAgICIiImV4dHJhRG9jc3RyaW5n
IGZvciBhIGNsYXNzIGRhdGEgYXR0cmlidXRlIiIiCiAgICBhbm90aGVyQXR0cmlidXRlID0gJ2Fi
Y2RlZmcnCiAgICAiIiJleHRyYURvY3N0cmluZyBmb3IgYW5vdGhlciBjbGFzcyBkYXRhIGF0dHJp
YnV0ZSIiIgogICAgCiAgICBkZWYgbWV0aG9kKHNlbGYsIGFyZyk6CiAgICAgICAgIiIiQSB0ZXN0
IGZ1bmMgZG9jc3RyaW5nICIiIgogICAgICAgICIiImV4dHJhRG9jc3RyaW5nIGZvciBhIGNsYXNz
IG1lbWJlciBmdW5jdGlvbiIiIgogICAgICAgIHByaW50IGFyZwogICAgICAgIHJldHVybiAxCgoK
CiMgYW5kIG9mIGNvdXJzZSB1c2UgY29tbWVudHMgdG8gZG9jdW1lbnQgYW55dGhpbmcgdGhhdCBv
bmx5IG5lZWRzIHRvIGJlCiMgcG9pbnRlZCBvdXQgdG8gdGhvc2Ugd29ya2luZyB3aXRoIG1vZHVs
ZSdzIHNvdXJjZSBkaXJlY3RseSBlLmcuCmdsb2JhbEZ1bmMoJ0JpZ2dsZXMgRGljdGF0ZXMgYSBM
ZXR0ZXIuICcpICMgdGhpcyBjb3VsZCBkb2VzIGJ1Z2dlciBhbGwgLSBkb24ndCBib3RoZXIgbWFp
bnRhaW5pbmcgaXQuCmVyaWMgPSBTYW1wbGVDbGFzcygpCmVyaWMubWV0aG9kKCdzcGFtJykKCgoK
IyBBbmQgZmluYWxseSBhIHN0cmluZyBsaXRlcmFsIGZvciB0aGUgZm9ybWFsIGRvY3VtZW50YXRp
b24gdG8KIyBpbmNsdWRlIGluIHRoZSBweXRob24gbGlicmFyeSByZWZlcmVuY2UgYXMgS2EtUGlu
ZyBzdWdnZXN0ZWQgaW4gaGlzCiMgcG9zdCBvbiBNYXJjaCAxMS4gIFRoaXMgaXMgY29uY2VwdHVh
bGx5IHNlcGFyYXRlIGZyb20gdGhlCiMgQVBJIHJlZmVyZW5jZSBkb2N1bWVudGF0aW9uIHN0b3Jl
ZCBpbiB0aGUgZG9jc3RyaW5ncyBhbmQgZXh0cmFEb2NzdHJpbmdzIGFib3ZlLgojIEl0IHNob3Vs
ZCBjb21wbGltZW50LCBub3QgZHVwbGljYXRlLCB0aGUgQVBJIHJlZmVyZW5jZSBhbmQgcHJvdmlk
ZSBhbnkKIyBtZXRhLWRhdGEgbmVjZXNzYXJ5IGZvciBwcm9wZXIgaW5kZXhpbmcgaW4gdGhlIHB5
dGhvbiBsaWJyYXJ5IHJlZmVyZW5jZSBwYWdlcy4KCiIiIkxJQlJBUllfUkVGOgoKU3RhcnQgd2l0
aCBhIHRva2VuIHRvIHRlbGwgdGhlIGRvY3VtZW50YXRpb24gcGFyc2luZyB0b29sIHRoYXQgd2hh
dCBmb2xsb3dzIGlzIGZvcgp0aGUgbGlicmFyeSByZWZlcmVuY2UgYW5kIG5vdCB0aGUgQVBJIHJl
Zi4gIFRoaXMgd291bGQgcmVtb3ZlIHRoZSBuZWVkIGZvciB0aGlzCnN0cmluZyBsaXRlcmFsIHRv
IGJlIGluY2x1ZGVkIGFzIHRoZSB2ZXJ5IGxhc3QgdGhpbmcgaW4gdGhlIHNvdXJjZSBmaWxlLCBh
bmQgd291bGQKYWxzbyBhbGxvdyB0aGUgbGlicmFyeSByZWZlcmVuY2UgY29kZSB0byBiZSBzcGxp
dCB1cCBpbnRvIGNodW5rcyB0aHJvdWdob3V0CnRoZSBzb3VyY2UgY29kZSAoYSBsYSBwZXJsJ3Mg
UE9EKS4gIEkgdGhpbmsgdGhpcyBpcyB3aGF0IEthLVBpbmcgbWVhbnQgYnkKIihvciBtYXliZSBj
b2xsZWN0IGRvY3N0cmluZ3MgZnJvbSBhbnl3aGVyZSBpbiB0aGUgZmlsZSkuIgoKVGhpcyBkb2N1
bWVudGF0aW9uIHN0cmluZyBjb3VsZCBiZSB3cml0dGVuIGluIExhVGVYLCBvciBzdHJ1Y3R1cmVk
IHRleHQuICBNeSBwZXJzb25hbApwcmVmZXJlbmNlIGlzIGZvciBhIGZvcm1hbGl6ZWQgU3RydWN0
dXJlZCBUZXh0IGxhbmd1YWdlLiAgSSB1c2UgTGFUZVggZXh0ZW5zaXZlbHkKZm9yIGxhcmdlciBk
b2N1bWVudHMgKGUuZy4gbXkgdGhlc2lzKSwgYnV0IHRoaW5rIGl0J3MgaW5hcHByb3ByaWF0ZSBm
b3IgbW9kdWxlIGRvY3MKZm9yIHRoZXNlIHJlYXNvbnM6Ci0gaXQgZGVtYW5kcyB0aGUgcHJlc2Vu
Y2Ugb2YgZXh0cmEgcHJvY2Vzc2luZyB0b29scwotIGl0IHJlcXVpcmVzIHRoZSBtb2R1bGUgY3Jl
YXRvciB0byBsZWFybiB5ZXQgYW5vdGhlciBtYXJrdXAgbGFuZ3VhZ2UKLSBpdCBvZmZlcnMgbm8g
cmVhbCBhZHZhbnRhZ2Ugb3ZlciBTVCBmb3Igc21hbGwgZG9jdW1lbnRzICg8MTAgcGFnZXMpCi0g
aGF2aW5nIHR3byBtYXJrLXVwIGxhbmd1YWdlcyBpbiB1c2UgZm9yIG9uZSBtb2R1bGUgcGxhY2Vz
IHRvbyBtdWNoIGVtcGhhc2lzCiAgb24gdGhlIHNlcGFyYXRpb24gYmV0d2VlbiB0aGUgQVBJIGFu
ZCBsaWJyYXJ5IHJlZmVyZW5jZSBkb2NzIGFuZCBkaXNjb3VyYWdlcwogIHN5bmNocm9uaXphdGlv
biBhcyB0aGUgbW9kdWxlIGV2b2x2ZXMhCgoiIiIKCgo=

--------------Boundary-00=_OHJ3M3E2ZQZ6GAK7DZGH--