[Python-Dev] patching pydoc?

[Ron Adam]

tomer filiba wrote:
> i have a problem with pydoc in rpyc. i wanted help(obj), where obj
> is a NetProxy object, to work as if it were local.
> 
> i followed the code starting from site.help to pydoc.doc, which is the
> ultimate function that generates and prints the text. i expected there
> would be some function in the middle that prepares the text, and
> another that writes it to the pager, but to my disappointment pydoc.doc
> does both.
> 
> this means i can't transfer the document to my local machine (it's
> printed directly to the remote console).
> 
> therefore, i would like to split this behavior into two parts:
> * render_doc - a function that returns the document text
> * doc - a function that calls render_doc and sends it to the pager
> 
> this way no existing code breaks (no existing function signatures
> are changed) and i gain help on remote objects.
> i hope people would be in favor, as it's not such a big change anyway.
> 
> 
> is it possible to add to 2.5?
> 
> -tomer


Hi Tomer,

This doesn't address your small patch since the following described 
project is, not complete, will be either be a separate module or 
package, (if/when it's completed), and because it's more of a complete 
rewrite than not, would be ready and tested no sooner than 2.6. But you 
might be interested in taking a look at what I've attempted to do so far.

(This is too premature to consider adding to python distribution but 
maybe after completing, and after it is used in the community it could 
be a candidate.)


I am/was working on cleaning up pydoc by trying to separate the various 
formating, server, and output routines, because they are currently very 
much intertwined making enhancing and/or maintaining pydoc more 
difficult than it could be.  This has resulted rewriting most of the top 
level functions, which after having some success, has been put on hold 
because I have other things to do that are more immediate at the moment.

What I have done (so far) is separate the file pydoc.py into multiple 
files in order to begin to better organize the code, And rewriting each 
of those files so they are less interdependent on each other and more 
complete in them selves with the possibility of putting it back together 
in a single file or a package at some point.  (I'm leaning on it being a 
package myself.)

My attempt has focused on having pydoc create an intermediate xml-tree 
data structure that is then be used as the source to generate text, 
html, and other outputs.  This separates the introspective and 
formatting functions of pydoc.  The xml to html function mostly work. 
(some dependency link tags still need to be reimplemented.)

Another improvement I've made is having the html server prepend an html 
search bar to each page which eliminates the need to have a separate TK 
search window.  This makes interactively browsing with pydoc very easy 
and nice. :-)  It also uses a style sheet to format the html pages, 
making it easier to change colors, indents, fonts, and borders, etc...


If you are interested in working on this or would just like to take a 
look, let me know, I can put it in a zip file and send it to you.  It's 
not as well organized as I would like at the moment, but the xml and 
html generation functions work.  No attempt to create plain text output 
from the xml has been made yet.  I intended to write the xml to text 
(and paging help) functions from scratch once the html output is 
complete and fully working. (mostly done) I think it will not be 
difficult to get the output very similar to the existing pydoc help text 
form.


I'm thinking it may be better to share it at this time rather than let 
it sit on my hard drive and possibly get waylayed, or worse, forgotten 
indefinitely.

If this is the "right approach", and is "desirable", maybe it could make 
a good community project.  Possibly someone could post it and/or manage 
it someplace?  It may need a bit more cleaning up before it's ready for 
that though.


Some additional thoughts of why to do this are:

I'm hoping there may be a way to run the html server on a web site to 
generate documentation directly (pydoc-live?) instead of pre-generating 
html files and placing them on the web site.  (Possible?,  Has that 
already been done?)  I know pydoc html files are available on line, but 
I think they are all pregenerated by pydoc first then uploaded.  That 
may be preferred in most cases because it would require less server 
overhead and be more secure, but live pydocs would give more immediate 
and dynamic feedback when a project is being updated and changed often.

Another far off (and more ambitious 3.0 ?) goal may be to use the pydoc 
generated xml as a source to insert doc strings and other information 
directly into the python user manuals.  That may improve both the python 
library manuals as well as the pydoc help information by having them 
more closely tied together.  Inconsistencies would be found sooner. The 
doc strings could be the brief quick ref documents, which it currently 
isn't always brief and/or present, and the python manual could include 
the pydoc output (or selected parts of it) along with the longer 
explanation and discussion.  Having pydoc produce xml as an intermediate 
format makes these types of things easier to do.


Cheers,
    Ron Adam