From goodger at python.org Tue May 3 05:04:27 2005 From: goodger at python.org (David Goodger) Date: Mon, 02 May 2005 23:04:27 -0400 Subject: [Doc-SIG] two subjects: docutils for python docs, and integrating documentation in source code In-Reply-To: <87d5scyycs.fsf@news2.ososo.de> References: <8393fff0504201039723adaf0@mail.gmail.com> <874qdzngom.fsf@news2.ososo.de> <8393fff05042206014362b8fd@mail.gmail.com> <87d5scyycs.fsf@news2.ososo.de> Message-ID: <4276EA3B.30404@python.org> > Martin Blais wrote: >> [...] many methods or functions simply don't need to appear in the >> documentation [Felix Wiemann] > Maybe add a special token to the docstring that the method shouldn't > appear in auto-documentation? A special comment perhaps? Or a function attribute? I don't think such metadata should be put in the docstring; that's for documentation. OTOH, the common convention for "weakly private" or "internal" can be used: a single underscore prefix. E.g.: def _internal(...): >> what i meant is that it is not currently possible to generate >> generic index entries with docutils, e.g. in a paragraph of normal >> text, how do i indicate that a word in the paragraph should appear >> in the index? > > * Write index support for Docutils and use a special role. Some ideas have been documented; see http://docutils.sf.net/docs/dev/rst/alternatives.html#index-entries-indexes -- David Goodger -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 253 bytes Desc: OpenPGP digital signature Url : http://mail.python.org/pipermail/doc-sig/attachments/20050502/72a12d3a/signature.pgp From vlb at ironport.com Thu May 12 19:56:33 2005 From: vlb at ironport.com (Vicki Brown) Date: Thu, 12 May 2005 10:56:33 -0700 Subject: [Doc-SIG] "inverse" documentation generation Message-ID: And now for something (perhaps) completely diifferent. Most "documentation generators" are optimized for someone working _on_ the code rather than _with_ the code. That is, they work from the assumption that you have a method/class/module and want to know how it works - not from the view that you know what you want to do and want to discover the best method/class/module that will let you accomplish your goal in the least amount of time and effort :) I want to be able to generate documentation that aids a client in using a set of Python modules, not in implementing and maintaining those modules. Example: Bob is using an automated test framework written in Python. He's writing a test and he'd like to call a routine that will "be like hitting return over and over until it gets me back out to the top of my test." He asks John (one of the framework authors) who says: "We've got that; it's called restart". Bob then hunts through the pydoc documentation for the 100 modules and packages in the framework, then resorts to a find command on the source tree (searching for for "def restart"), and eventually discovers 5 different "restart" methods scattered throughout the code. Back in pydooc, he sees that one of these has a docstring that looks like "the right one". """ restart(): Resets the session to a base state, ready for more method calls. """ The rest appear to be internal methods, not API at all. If John wasn't available, how would Bob ever discover this method? He doesn't know its name. He doesn't know which class or module it's in. He doesn't want to read through pages of pydoc HTML files or familiarize himself with the entire framework code base. He wants to _write a test_. Even if he started with a"find" on the source tree (or a search through the HTML-ized docs), he may not pick the right keywords for the search. I need a way to enhance Bob's efficiency as a client of the framework, not a developer of the framework. At the very least, I think what I want is a way to extract the method, class, and module names, and their docstrings, and present them in an inverse presentation format: instead of digging down through module to class to method, I want to start from the docstrings. I want to start from the viewpoint of "I know what I want to do. Now tell me how to do it". Comments, suggestions, ideas, and general discussion welcomed. -- - V. Vicki Brown * Internal Technical Documentation http://home.ironport.com/~vlbrown * Jabber: vlbrown at ironport.com * Words are, of course, the most powerful drug used by mankind. * -- Rudyard Kipling From kenneth.m.mcdonald at gmail.com Fri May 13 19:55:27 2005 From: kenneth.m.mcdonald at gmail.com (Kenneth McDonald) Date: Fri, 13 May 2005 12:55:27 -0500 Subject: [Doc-SIG] Current 'standard' documentation generation program/standards for Python? Message-ID: Is there yet an agreed-upon standard (format and/or doc generation prog) for Python documentation? I used epydoc a while back and it was OK, but still not great. Python is by far my fav. language, but the lack of a good doc solution is just driving me nuts! Thanks, Ken From mwh at python.net Sat May 14 13:02:20 2005 From: mwh at python.net (Michael Hudson) Date: Sat, 14 May 2005 12:02:20 +0100 Subject: [Doc-SIG] Current 'standard' documentation generation program/standards for Python? In-Reply-To: (Kenneth McDonald's message of "Fri, 13 May 2005 12:55:27 -0500") References: Message-ID: <2mis1m5av7.fsf@starship.python.net> Kenneth McDonald writes: > Is there yet an agreed-upon standard (format and/or doc generation > prog) for Python documentation? Well, the current docs are written in Latex, but I don't think that's what you're really asking. > I used epydoc a while back and it was OK, but still not > great. I don't know of anything markedly better than epydoc. Maybe you could help improve it? Cheers, mwh -- Java is a WORA language! (Write Once, Run Away) -- James Vandenberg (on progstone at egroups.com) & quoted by David Rush on comp.lang.scheme From skip at pobox.com Mon May 16 05:04:29 2005 From: skip at pobox.com (Skip Montanaro) Date: Sun, 15 May 2005 22:04:29 -0500 Subject: [Doc-SIG] Python docs in reST? Message-ID: <17032.3517.362406.820105@montanaro.dyndns.org> A thread on c.l.py ("Python Documentation (should be better?)") got me to thinking about options for improving/restructuring the online documentation. That led me to thinking about converting the current docs to reST. A scan of the last year's worth of list archives didn't yield any obvious discussion on the topic, but I find it hard to believe the topic hasn't been discussed in the past. Pointers to past threads cheerfully accepted. It seems to me that converting to reST has a couple obvious benefits: * dump latex2html * better potential HTML generation options (like HTML w/ an integrated annotation capability) * possibly enlarge pool of documentation contributors Fred wrote some Latex->ESIS->SGML tools that look like they might possibly be redirected to the more-or-less one-time task of converting to reST. Thoughts? -- Skip Montanaro skip at pobox.com From mike at pcblokes.com Mon May 16 10:05:05 2005 From: mike at pcblokes.com (Michael Foord) Date: Mon, 16 May 2005 09:05:05 +0100 Subject: [Doc-SIG] Python docs in reST? In-Reply-To: <17032.3517.362406.820105@montanaro.dyndns.org> References: <17032.3517.362406.820105@montanaro.dyndns.org> Message-ID: <42885431.2010407@pcblokes.com> Skip Montanaro wrote: >A thread on c.l.py ("Python Documentation (should be better?)") got me to >thinking about options for improving/restructuring the online documentation. >That led me to thinking about converting the current docs to reST. A scan >of the last year's worth of list archives didn't yield any obvious >discussion on the topic, but I find it hard to believe the topic hasn't been >discussed in the past. Pointers to past threads cheerfully accepted. > >It seems to me that converting to reST has a couple obvious benefits: > > * dump latex2html > > * better potential HTML generation options (like HTML w/ an integrated > annotation capability) > > * possibly enlarge pool of documentation contributors > >Fred wrote some Latex->ESIS->SGML tools that look like they might possibly >be redirected to the more-or-less one-time task of converting to reST. > >Thoughts? > > > As it stands the docutils tools are some way off being able to generate Python documentation. They are not yet targeted at multipage documents - this includes the navigation links and multi page contents/indexes. I believe it is a long term goal (in which case Latex -> reST would be a worthy tool) - but not yet possible. You can generate HTML fragments from reST and insert them into templates. This is the approach rest2web__ uses. I'm going to see if I can evolve rest2web to address some of the issues above for my own needs. Eventually a solution within docutils will evolve (hopefully - if I *can* contribute then I will). Regards, Fuzzy http://www.voidspace.org.uk/python __ http://www.voidspace.org.uk/python/rest2web From kenneth.m.mcdonald at gmail.com Tue May 17 23:04:57 2005 From: kenneth.m.mcdonald at gmail.com (Kenneth McDonald) Date: Tue, 17 May 2005 16:04:57 -0500 Subject: [Doc-SIG] Python docs in reST In-Reply-To: References: Message-ID: On 5/16/05, doc-sig-request at python.org wrote: > Send Doc-SIG mailing list submissions to > doc-sig at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.python.org/mailman/listinfo/doc-sig > or, via email, send a message with subject or body 'help' to > doc-sig-request at python.org > > You can reach the person managing the list at > doc-sig-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Doc-SIG digest..." > > > Today's Topics: > > 1. Python docs in reST? (Skip Montanaro) > 2. Re: Python docs in reST? (Michael Foord) > To me, the biggest single problem in Python documentation is that there is no standard inline documentation format which has been "blessed" as _the_ standard by the Python core development team. I like Python much better than Ruby, but take a look at Ruby for a an example of what kind of tools can be built once people have a standard as a base. rubydoc is really, really nice Whatever might be chosen or developed, we need IMHO a doc standard that is part of the core python package, and comes with all of the tools needed to use it. Without this, documentation of python modules is always going to be rather ad hoc. Not that this was useful, but in any case if anyone wants to work towards it... :-) Ken From ianb at colorstudy.com Tue May 17 23:31:05 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Tue, 17 May 2005 16:31:05 -0500 Subject: [Doc-SIG] Python docs in reST In-Reply-To: References: Message-ID: <428A6299.6050005@colorstudy.com> Kenneth McDonald wrote: > To me, the biggest single problem in Python documentation is that > there is no standard inline documentation format which has been > "blessed" as _the_ standard by the Python core development team. I > like Python much better than Ruby, but take a look at Ruby for a an > example of what kind of tools can be built once people have a standard > as a base. rubydoc is really, really nice I think the community reached consensus on ReST_ some time ago -- not completely officially (though there is a PEP out there), but it's certainly the defacto standard. The docutils_ project, which ReST is a part of, is ostensibly larger than just ReST, > Whatever might be chosen or developed, we need IMHO a doc standard > that is part of the core python package, and comes with all of the > tools needed to use it. Without this, documentation of python modules > is always going to be rather ad hoc. Additionally, but conflictingly, there is a documentation tool in the standard library: pydoc_ (conflicting because it doesn't support ReST). But I'd say the primary JavaDoc-like tool would be Epydoc_ (which does support ReST, as well as its own markup language). Personally I've never seen a three-pane autogenerated reference document that I've liked. But eh... I think I'm just expecting too much from one tool. I guess one problem is that Python doesn't have good a formal notion of "public" vs "private", so a lot of cruft and implementation will show up in a autogenerated document. But, I'm feeling a need to autogenerate some documentation for Paste_, so I have to figure out some of this myself. Right now I'm thinking of building a little scraper that uses metadata specifically applicable to the project -- for instance, I want to keep indexes and make nice documentation of all the configuration values, and all the WSGI environment keys different components use. Obviously this kind of metadata is quite structured and (in the specific sense) not widely applicable. If PEAK_ was more like a tool and less like a worldview, I'd probably try to see what Phillip has done there, as I know he worked (or at least thought) about a kind of customizable system like this. But I'm afraid it's too much for me to get my head around right now :( -- anyway, I can't find it in CVS there, so maybe it never got to the point of code. .. _docutils: http://docutils.sourceforge.net/ .. _ReST: http://docutils.sourceforge.net/rst.html .. _pydoc: http://python.org/doc/current/lib/module-pydoc.html .. _Epydoc: http://epydoc.sourceforge.net/ .. _Paste: http://pythonpaste.org .. _PEAK: http://peak.telecommunity.com/ -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From mike at pcblokes.com Wed May 18 09:28:04 2005 From: mike at pcblokes.com (Michael Foord) Date: Wed, 18 May 2005 08:28:04 +0100 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <428A6299.6050005@colorstudy.com> References: <428A6299.6050005@colorstudy.com> Message-ID: <428AEE84.8060208@pcblokes.com> Ian Bicking wrote: > [snip..] > >Personally I've never seen a three-pane autogenerated reference document >that I've liked. But eh... I think I'm just expecting too much from one >tool. I guess one problem is that Python doesn't have good a formal >notion of "public" vs "private", so a lot of cruft and implementation >will show up in a autogenerated document. > > Yup. One of the biggest problems I've had with autogenerating documentation is that there is usually no way to mark a method/function as private other than the double underscore name mangling (which isn't usually what you want to do). There is also, usually, no way of adding documentation to constants within a module - other than in the module docstring itself. I've often thought about implementing something myself to overcome these problems. My favourite solution would allow some documentation in comments as well. Some of the difficulties that people experience with epydoc are because it imports modules to extract docstrings. The advantage this has is that it allows it to link classes to their superclasses by inspecting them. Simple parsing would eliminate some problems that people experience - but make it much harder to obtain class/superclass information. I've never really looked at/used pydoc - I wonder how hard it would be to add a useful synergy with reST ?? It's certainly an area I'd be willing to put some work into. Best Regards, Fuzzy http://www.voidspace.org.uk/python/ >But, I'm feeling a need to autogenerate some documentation for Paste_, >so I have to figure out some of this myself. Right now I'm thinking of >building a little scraper that uses metadata specifically applicable to >the project -- for instance, I want to keep indexes and make nice >documentation of all the configuration values, and all the WSGI >environment keys different components use. Obviously this kind of >metadata is quite structured and (in the specific sense) not widely >applicable. > >If PEAK_ was more like a tool and less like a worldview, I'd probably >try to see what Phillip has done there, as I know he worked (or at least >thought) about a kind of customizable system like this. But I'm afraid >it's too much for me to get my head around right now :( -- anyway, I >can't find it in CVS there, so maybe it never got to the point of code. > >.. _docutils: http://docutils.sourceforge.net/ >.. _ReST: http://docutils.sourceforge.net/rst.html >.. _pydoc: http://python.org/doc/current/lib/module-pydoc.html >.. _Epydoc: http://epydoc.sourceforge.net/ >.. _Paste: http://pythonpaste.org >.. _PEAK: http://peak.telecommunity.com/ > > > From martin.blais at gmail.com Wed May 18 12:51:26 2005 From: martin.blais at gmail.com (Martin Blais) Date: Wed, 18 May 2005 06:51:26 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <428AEE84.8060208@pcblokes.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> Message-ID: <8393fff05051803515b0e2a93@mail.gmail.com> hmm... I have come to the following conclusion: there are two types of documentation, and they are completely separate beasts, and I feel that we should start recognizing this difference. 1. reference docs: its structure mirrors the structure of the code, and it is often or best automatically generated from the source code and comments in the source code. It does not matter very much if the private methods are visible in this documentation; 2. user documentation: a manual written with the specific purpose of educating a person about how to use the specific module. its structure does not mirror that of the source code at all. It does not describe all the source code, just what is needed. There are code examples. There might be different manuals for different types of users written from different angles. Most of the recent discussions I hear about documentation blur this distinction, and I believe that this is making the discussions a little bit unfocused. Fred's approach with Python was to build (2), which is why IMHO that Python docs are so great. We need to recognize that writing documentation is a task in itself, and no amount of intelligent processing over docstrings will ever produce user manuals that are fun to read. My beef with this approach is that the docstrings are not used, and the documentation of functions has to be maintained by hand, in the doc file which is completely separate and far away. I have been proposing a way to share those bits and pieces (the docstrings) so that one could write a user manual independently, but still document the low-level functions in the docstrings and have that information pulled into the manual selectively. I feel that (1) is often not very useful. My experience working on large C++ codebases where the auto-documentation tools were working well and used consistently is that most people still used the source code rather than the generated HTML pages (and certainly this is what I did too). With emacs everything is available C-s or a tag away, or a grep away (or with LXR if you have that). I'd actually prefer to read from the source because other cues gave subtle information about the intent of the programmers (or lack of intent). However, many people want to generate that kind of documentation. We just have to make sure what our intention is. Oftentimes I feel that it is our programmer's instinct that makes us believe we can automatically generate (1), while really, it's not really possible. So anyway, it would be great if we could clarify the context of our intent in the discussions about documentation, whether we want to build (1), or (2), or to state clearly if we want to generate both from a single source, or whatever. Just an idea. cheers, From mike at pcblokes.com Wed May 18 13:20:12 2005 From: mike at pcblokes.com (Michael Foord) Date: Wed, 18 May 2005 12:20:12 +0100 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <8393fff05051803515b0e2a93@mail.gmail.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> Message-ID: <428B24EC.6000408@pcblokes.com> Hello All, I agree with your comments about writing documentation - any moderately large framework *needs* properly written documentation rather than just an API reference. But : Martin Blais wrote: >hmm... I have come to the following conclusion: there are two types >of documentation, and they are completely separate beasts, and I feel >that we should start recognizing this difference. > >1. reference docs: its structure mirrors the structure of the code, >and it is often or best automatically generated from the source code >and comments in the source code. It does not matter very much if the >private methods are visible in this documentation; > > > If I am writing a small (ish) module I can fulfill 1 and 2 with a decent module docstring and proper docstrings for functions and classes etc. In which case I *need* to be able to auto extract the docs and create a reference doc from them. I *do not* want private methods visible in this documentation - and I do want a way of specifying that a function/method should not be included. If the 'user' is a programmer, I don't think there is necessarily a clear distinction between 1 and 2. Even where the documentation is being separately written and maintained it would be *useful* to have an 'extract docstring from source' directive - so that the docs (where appropriate) don't need to be maintained twice. This is different from autogenerating the whole of the documentation. Best Regards, Fuzzyman http://www.voidspace.org.uk/python >[snip..] >cheers, >_______________________________________________ >Doc-SIG maillist - Doc-SIG at python.org >http://mail.python.org/mailman/listinfo/doc-sig > > > > > From martin.blais at gmail.com Wed May 18 14:56:24 2005 From: martin.blais at gmail.com (Martin Blais) Date: Wed, 18 May 2005 08:56:24 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <428B3AE0.70603@pcblokes.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B24EC.6000408@pcblokes.com> <8393fff050518050550e65de9@mail.gmail.com> <428B3AE0.70603@pcblokes.com> Message-ID: <8393fff0505180556781bd8e2@mail.gmail.com> > >>If I am writing a small (ish) module I can fulfill 1 and 2 with a decent > >>module docstring and proper docstrings for functions and classes etc. In > >>which case I *need* to be able to auto extract the docs and create a > >>reference doc from them. > >> > >>I *do not* want private methods visible in this documentation - and I do > >>want a way of specifying that a function/method should not be included. > >> > >>If the 'user' is a programmer, I don't think there is necessarily a > >>clear distinction between 1 and 2. > >> > >> > > > >well, i would argue that what you want to do is (1), > > > Yes :-) > > > but you want it > >in the source file. > > > > > Yes, and be able to generate an API doc from it. > > >i was suggesting earlier that this could be implemented by adding a > >special docstring of your choice (e.g. __manual__) to your source > >code, it could even be at the end if you want it to. whatever tool > >you use to generate documentation would first extract that flie before > >processing. > > > > > > > > > Why not just use the module docstring ? can you imagine scrolling past an entire manual everytime you open a new file? (i'm assuming we're writing real user documentation in that docstring, not just an overview) From skip at pobox.com Wed May 18 15:17:56 2005 From: skip at pobox.com (Skip Montanaro) Date: Wed, 18 May 2005 08:17:56 -0500 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <428AEE84.8060208@pcblokes.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> Message-ID: <17035.16516.937200.902431@montanaro.dyndns.org> Michael> Yup. One of the biggest problems I've had with autogenerating Michael> documentation is that there is usually no way to mark a Michael> method/function as private other than the double underscore Michael> name mangling (which isn't usually what you want to do). Actually, there are two ways: * single leading underscore => this attribute is private (this is simply a convention based on the "we're all adults here" rubric) * double leading underscores without double trailing underscores => the compiler mangles the name, IMHO mostly to prevent accidental stomping on attributes in a subclass, not so much to make it truly private Skip From mike at pcblokes.com Wed May 18 15:22:40 2005 From: mike at pcblokes.com (Michael Foord) Date: Wed, 18 May 2005 14:22:40 +0100 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <8393fff0505180556781bd8e2@mail.gmail.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B24EC.6000408@pcblokes.com> <8393fff050518050550e65de9@mail.gmail.com> <428B3AE0.70603@pcblokes.com> <8393fff0505180556781bd8e2@mail.gmail.com> Message-ID: <428B41A0.3030801@pcblokes.com> Martin Blais wrote: >>>>If I am writing a small (ish) module I can fulfill 1 and 2 with a decent >>>>module docstring and proper docstrings for functions and classes etc. In >>>>which case I *need* to be able to auto extract the docs and create a >>>>reference doc from them. >>>> >>>>I *do not* want private methods visible in this documentation - and I do >>>>want a way of specifying that a function/method should not be included. >>>> >>>>If the 'user' is a programmer, I don't think there is necessarily a >>>>clear distinction between 1 and 2. >>>> >>>> >>>> >>>> >>>well, i would argue that what you want to do is (1), >>> >>> >>> >>Yes :-) >> >> >> >>>but you want it >>>in the source file. >>> >>> >>> >>> >>Yes, and be able to generate an API doc from it. >> >> >> >>>i was suggesting earlier that this could be implemented by adding a >>>special docstring of your choice (e.g. __manual__) to your source >>>code, it could even be at the end if you want it to. whatever tool >>>you use to generate documentation would first extract that flie before >>>processing. >>> >>> >>> >>> >>> >>> >>Why not just use the module docstring ? >> >> > >can you imagine scrolling past an entire manual everytime you open a new file? >(i'm assuming we're writing real user documentation in that docstring, >not just an overview) > > > > > Ha :-) OTOH can you imagine scrolling past an entire module just to get to the docs ;-) If I'm writing something that requires a full doc then I wouldn't put it in a docstring - but for some *modules* [#]_ it is appropriate to include brief documentation in the source code. I think you're right - there are two related but separate use cases. 1) Autogenerating API docs from source code. 2) Writing proper documentation - and included details from the source code. The reasons **I** am unhappy with the existing tools [#]_ (i.e. epydoc) is that they don't allow me to mark constants/functions/classes/methods as private, and they don't allow me to document constants. You could overcome these problems by parsing source code rather than importing and examining objects. You would obviously need to develop appropriate conventions. I don't object to using a magic variable like '__manual__' - but I would want to obtain it by parsing rather than introspection. (In which case you could equally use a docstring at the end of a document). In either case it would need work recommencing on the Pysource reader. I'm working on rest2web at the moment - but will have some time in the future to look at this - I do want to get further into the docutils source code. Best Regards, Fuzzy http://www.voidspace.org.uk/python .. [#] as opposed to a framework or application that will require more than just a description of functions/classes/methods .. [#] For achieving number 1 From nico at tekNico.net Wed May 18 15:36:56 2005 From: nico at tekNico.net (Nicola Larosa) Date: Wed, 18 May 2005 15:36:56 +0200 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <428B41A0.3030801@pcblokes.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B24EC.6000408@pcblokes.com> <8393fff050518050550e65de9@mail.gmail.com> <428B3AE0.70603@pcblokes.com> <8393fff0505180556781bd8e2@mail.gmail.com> <428B41A0.3030801@pcblokes.com> Message-ID: > The reasons **I** am unhappy with the existing tools [#]_ (i.e. epydoc) > is that they don't allow me to mark constants/functions/classes/methods > as private, and they don't allow me to document constants. Epydoc does have the ability to avoid including private objects in the generated docs. Even when you include them, it generates two versions of the docs, one with and one without them. > You could overcome these problems by parsing source code rather than > importing and examining objects. You would obviously need to develop > appropriate conventions. That's how HappyDoc works. Maybe someone could try and merge (or rather, cherry-pick features from) the two tools. ;-) -- Nicola Larosa - nico at tekNico.net In one sense, we're entering an age of high tech. In another sense, it's *always* been high tech. Poster: picture of a nautilus shell. Caption: "High Tech: It's not just where we're heading, it's where we're coming from." -- Kirby Urner, April 2005 From mike at pcblokes.com Wed May 18 16:32:25 2005 From: mike at pcblokes.com (Michael Foord) Date: Wed, 18 May 2005 15:32:25 +0100 Subject: [Doc-SIG] Python docs in reST In-Reply-To: References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B24EC.6000408@pcblokes.com> <8393fff050518050550e65de9@mail.gmail.com> <428B3AE0.70603@pcblokes.com> <8393fff0505180556781bd8e2@mail.gmail.com> <428B41A0.3030801@pcblokes.com> Message-ID: <428B51F9.3020406@pcblokes.com> Nicola Larosa wrote: >>The reasons **I** am unhappy with the existing tools [#]_ (i.e. epydoc) >>is that they don't allow me to mark constants/functions/classes/methods >>as private, and they don't allow me to document constants. >> >> > >Epydoc does have the ability to avoid including private objects in the >generated docs. Even when you include them, it generates two versions of >the docs, one with and one without them. > > > > It's probably me *assuming* that epydoc only recognises double underscores as private. I'd still prefer a way of marking *within* the docstring (or by comments) that a method is private. I'd rather not have to name my objects around my docuemtnation tool. I may be expecting too much though. Regards, Fuzzy http://www.voidspace.org.uk/python >>You could overcome these problems by parsing source code rather than >>importing and examining objects. You would obviously need to develop >>appropriate conventions. >> >> > >That's how HappyDoc works. Maybe someone could try and merge (or rather, >cherry-pick features from) the two tools. ;-) > > > > From ianb at colorstudy.com Wed May 18 18:08:00 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 18 May 2005 11:08:00 -0500 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <8393fff05051803515b0e2a93@mail.gmail.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> Message-ID: <428B6860.9020900@colorstudy.com> Martin Blais wrote: > hmm... I have come to the following conclusion: there are two types > of documentation, and they are completely separate beasts, and I feel > that we should start recognizing this difference. > > 1. reference docs: its structure mirrors the structure of the code, > and it is often or best automatically generated from the source code > and comments in the source code. It does not matter very much if the > private methods are visible in this documentation; > > 2. user documentation: a manual written with the specific purpose of > educating a person about how to use the specific module. its > structure does not mirror that of the source code at all. It does not > describe all the source code, just what is needed. There are code > examples. There might be different manuals for different types of > users written from different angles. I think that's a good distinction. And like several people I'm not very interested in 1. Right now I'm comfortable using stand-alone text files for some kinds of documentation, like tutorials. These often bring together code that isn't closely related, cover topics only briefly, and hopefully can be doctested to maintain accuracy. But when it comes to reference documentation I'm struggling. For Paste in particular, automatically generated documentation would be unnecessarily intimidating and hard to use, as it would expose all sorts of details about the implementation that aren't important from the outside. Short of prefixing 90% of my symbols with _ (which obviously I don't want to do ;) there's a lot of stuff something like epydoc couldn't recognize as private -- because they aren't really "private" at all, just not that "public" either. (Well, in theory on the global level I can use __all__) As I've been thinking about this, one solution is interfaces. I'm pretty happy with the documentation value of the Wareweb Servlet interface, for instance: http://svn.pythonpaste.org/Paste/trunk/paste/wareweb/interfaces.py And I'm happy to maintain that even though it's not a formal interface, I'm not using it for anything, and I don't expect any alternate implementations. (Though I'm reluctant to make Paste depend on Zope interfaces until they get into core because they include C code; though of course the presence of that C code doesn't mean anything to me because I still wouldn't *use* those interfaces for anything, so maybe I just have to become comfortable with the installation issues.) Formal ways of noting what interfaces are required for certain arguments would also be useful. But that only covers objects. And while the source of an interface is highly readable, converting that to HTML would be useful. But I have a lot of middling documentation. It's not doctestable, it's not structured so that an interface is useful, the code is in modules I don't expect the user to import specifically, and I am wary of just creating a separate text file because of API drift; I've found it hard to keep track of the information in two places. My thinking now is that I put these docs in the modules directly -- maybe with docstrings with special markings, maybe in a special variable (e.g., __manual__) -- and then manually maintain documents that import those bits of documentation into a more structured form. This would give me space to include notes between modules (providing continuity and notes on the relations between code), and order them the way I prefer. If the documentation is able to have intelligent interlinking, that'd also be great; my experience with ReST is not great here, as it makes strong distinctions between same-page links and external links. Though maybe that could be easily fixed with a two-phase build process; ReST identifies the links, outputs HTML, and then I resolve all the links in a separate phase. This mean I wouldn't have to recompile the ReST as often too, which is nice since ReST is slow enough that it matters. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From ianb at colorstudy.com Wed May 18 18:20:44 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 18 May 2005 11:20:44 -0500 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <428B41A0.3030801@pcblokes.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B24EC.6000408@pcblokes.com> <8393fff050518050550e65de9@mail.gmail.com> <428B3AE0.70603@pcblokes.com> <8393fff0505180556781bd8e2@mail.gmail.com> <428B41A0.3030801@pcblokes.com> Message-ID: <428B6B5C.7040700@colorstudy.com> Michael Foord wrote: > You could overcome these problems by parsing source code rather than > importing and examining objects. You would obviously need to develop > appropriate conventions. > > I don't object to using a magic variable like '__manual__' - but I would > want to obtain it by parsing rather than introspection. (In which case > you could equally use a docstring at the end of a document). > > In either case it would need work recommencing on the Pysource reader. > I'm working on rest2web at the moment - but will have some time in the > future to look at this - I do want to get further into the docutils > source code. Personally I thought source parsing was the right thing too -- it makes some things easier, like detecting top-level strings, which can be treated like interstitial docstrings. But since then I've changed my mind, and prefer introspection. It allows some things that source parsing doesn't allow at all, like generating documentation programmatically. I think this is important for code that uses frameworks, as there's often very structured information about the code that is best expressed in Python data structures. It also allows code that effectively does domain-specific introspection to create documentation. Another issue is that Python source is becoming increasingly dynamic (both because of extensions to Python and because the Python community has generally been using more metaprogramming techniques). For instance, you can't really figure out what a decorator will do until you actually run it. We don't have good conventions right now for how a decorator actually could add information to the documentation, but any convention we could make will have to rely on object introspection instead of source parsing. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From aahz at pythoncraft.com Wed May 18 18:42:15 2005 From: aahz at pythoncraft.com (Aahz) Date: Wed, 18 May 2005 09:42:15 -0700 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <17035.16516.937200.902431@montanaro.dyndns.org> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <17035.16516.937200.902431@montanaro.dyndns.org> Message-ID: <20050518164215.GB22594@panix.com> On Wed, May 18, 2005, Skip Montanaro wrote: > > Michael> Yup. One of the biggest problems I've had with autogenerating > Michael> documentation is that there is usually no way to mark a > Michael> method/function as private other than the double underscore > Michael> name mangling (which isn't usually what you want to do). > > Actually, there are two ways: > > * single leading underscore => this attribute is private (this is simply > a convention based on the "we're all adults here" rubric) That's two-thirds true: the convention is based on module attributes, where a single underscore is in fact significatn. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "And if that makes me an elitist...I couldn't be happier." --JMS From mwh at python.net Mon May 16 11:24:27 2005 From: mwh at python.net (Michael Hudson) Date: Mon, 16 May 2005 10:24:27 +0100 Subject: [Doc-SIG] Python docs in reST? In-Reply-To: <17032.3517.362406.820105@montanaro.dyndns.org> (Skip Montanaro's message of "Sun, 15 May 2005 22:04:29 -0500") References: <17032.3517.362406.820105@montanaro.dyndns.org> Message-ID: <2msm0n4j78.fsf@starship.python.net> Skip Montanaro writes: > A thread on c.l.py ("Python Documentation (should be better?)") got me to > thinking about options for improving/restructuring the online documentation. > That led me to thinking about converting the current docs to reST. A scan > of the last year's worth of list archives didn't yield any obvious > discussion on the topic, but I find it hard to believe the topic hasn't been > discussed in the past. Pointers to past threads cheerfully accepted. I thought it had too, but maybe only on python-list. I'm not so sure it would make much difference, but maybe that's just my cynicism. > It seems to me that converting to reST has a couple obvious benefits: > > * dump latex2html It works most of the time these days! > * better potential HTML generation options (like HTML w/ an integrated > annotation capability) But only, very much "potential". > * possibly enlarge pool of documentation contributors Yeah, right. This is where my lack of faith in humanity comes out. I believe there has been a long-ish term standing offer from assorted people to turn plain text docs into suitable latex for the docs (if not, let me make one). The main reason that Python's documentation is patchy in places is because writing good content is hard! Dealing with *any* format is minor in comparison. > Fred wrote some Latex->ESIS->SGML tools that look like they might possibly > be redirected to the more-or-less one-time task of converting to reST. I don't know if the task of generating reST has been really solved, either. Cheers, mwh -- > I wouldn't want to live without readline, but some of the > things it does call for the application of thumbscrews. -- me on python-dev From skip at pobox.com Wed May 18 19:29:48 2005 From: skip at pobox.com (Skip Montanaro) Date: Wed, 18 May 2005 12:29:48 -0500 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <428B6860.9020900@colorstudy.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> Message-ID: <17035.31628.509536.426587@montanaro.dyndns.org> >> 1. reference docs: ... >> >> 2. user documentation: ... Ian> I think that's a good distinction. And like several people I'm not Ian> very interested in 1. Ah, but I started this thread thinking about #1. FYI, while #2 is important, I'm still interested in this. I'm particularly interested in being able to replace latex2html with something more flexible that would allow relatively easy experimentation with the online presentation of the core documentation. Skip From skip at pobox.com Wed May 18 19:36:53 2005 From: skip at pobox.com (Skip Montanaro) Date: Wed, 18 May 2005 12:36:53 -0500 Subject: [Doc-SIG] Python docs in reST? In-Reply-To: <2msm0n4j78.fsf@starship.python.net> References: <17032.3517.362406.820105@montanaro.dyndns.org> <2msm0n4j78.fsf@starship.python.net> Message-ID: <17035.32053.771431.733525@montanaro.dyndns.org> >> * dump latex2html Michael> It works most of the time these days! I was under the impression that Fred was still dealing with a forked version of the code and that it was still fairly fragile. >> * better potential HTML generation options (like HTML w/ an integrated >> annotation capability) Michael> But only, very much "potential". Sure, but how much experimentation do you think is practical using latex2html? >> * possibly enlarge pool of documentation contributors Michael> Yeah, right. This is where my lack of faith in humanity comes Michael> out. Understood. Michael> I believe there has been a long-ish term standing offer from Michael> assorted people to turn plain text docs into suitable latex for Michael> the docs (if not, let me make one). A number of people (myself included) have made this offer on multiple occsions, generally in response to some variant of "Python's documentation sucks". (Which we all know it doesn't, but they complain just the same.) Michael> The main reason that Python's documentation is patchy in places Michael> is because writing good content is hard! Dealing with *any* Michael> format is minor in comparison. I would be happy if people contributed examples or submitted enhancement requests that referred to cookbook recipes. >> Fred wrote some Latex->ESIS->SGML tools that look like they might >> possibly be redirected to the more-or-less one-time task of >> converting to reST. Michael> I don't know if the task of generating reST has been really Michael> solved, either. Yeah, that seems to be a stumbling block. Skip From martin.blais at gmail.com Wed May 18 19:43:19 2005 From: martin.blais at gmail.com (Martin Blais) Date: Wed, 18 May 2005 13:43:19 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <428B6860.9020900@colorstudy.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> Message-ID: <8393fff050518104323989f74@mail.gmail.com> On 5/18/05, Ian Bicking wrote: > But I have a lot of middling documentation. It's not doctestable, it's > not structured so that an interface is useful, the code is in modules I > don't expect the user to import specifically, and I am wary of just > creating a separate text file because of API drift; I've found it hard > to keep track of the information in two places. i'm going to sound like a sucker here, but why not just put it "next" to the module file, in a .txt file?, e.g. module.py module.txt won't hurt the code, and it's right there a C-xC-f away... (I don't like to have docs there somehow (and i suppose you don't either because nobody does it), but i really cannot say why...) From martin.blais at gmail.com Wed May 18 19:51:20 2005 From: martin.blais at gmail.com (Martin Blais) Date: Wed, 18 May 2005 13:51:20 -0400 Subject: [Doc-SIG] Python docs in reST? In-Reply-To: <2msm0n4j78.fsf@starship.python.net> References: <17032.3517.362406.820105@montanaro.dyndns.org> <2msm0n4j78.fsf@starship.python.net> Message-ID: <8393fff05051810511ac527fe@mail.gmail.com> On 5/16/05, Michael Hudson wrote: > Skip Montanaro writes: > > * possibly enlarge pool of documentation contributors > > Yeah, right. This is where my lack of faith in humanity comes out. > > I believe there has been a long-ish term standing offer from assorted > people to turn plain text docs into suitable latex for the docs (if > not, let me make one). > > The main reason that Python's documentation is patchy in places is > because writing good content is hard! Dealing with *any* format is > minor in comparison. i agree! however, we could make it just a little bit easier: i only recently started using the Docs directory to document some code, and I was bothered by a few things: 1. you have to keep a copy of the Python soruce code lying around to make it work; 2. it doesn't "just work", you have to rip a lot of stuff out of the given makefile to make it do the minimal thing for a new documentation set, and it has a lot of stuff that I didn't need, like isilo, custom bits for the python docs, etc. It took me a little while to get it going, it would have been nicer to do something like "emerge python-doc-system" and copy a few clear rules from somewhere else. If we could somehow separate the code that is needed for generating the Python docs (and not taking along all the custom bits that are specific to that/those documents), and package that separately, that would help a tiny little bit. just an idea. cheers, From bronger at physik.rwth-aachen.de Wed May 18 19:41:43 2005 From: bronger at physik.rwth-aachen.de (Torsten Bronger) Date: Wed, 18 May 2005 19:41:43 +0200 Subject: [Doc-SIG] Python docs in reST? References: <17032.3517.362406.820105@montanaro.dyndns.org> <2msm0n4j78.fsf@starship.python.net> Message-ID: <87is1g4ejs.fsf@wilson.rwth-aachen.de> Hall?chen! Michael Hudson writes: > [...] > > I believe there has been a long-ish term standing offer from > assorted people to turn plain text docs into suitable latex for > the docs (if not, let me make one). > > The main reason that Python's documentation is patchy in places is > because writing good content is hard! Dealing with *any* format > is minor in comparison. The willingness to contribute good content is an important premise, no doubt. However, it is also important that the community has agreed on one format. I'm not prepared to write in (or worse, convert to) a certain format that's abandonned later. I came to Python three months ago, and it's still not clear to me which format one should use for the documentation (which is very important to me and, in contrast to most developers, I enjoy to create). LaTeX seems to be a we-want-to-get-rid-of-it, and reST is still not really ready. (I used the latter, nevertheless. My main reason was that LaTeX cannot be autoconverted reliably.) So all in all, I do think that Python would benefit from a clear homogeneous documentation policy, combined with strong reST tools. Tsch?, Torsten. -- Torsten Bronger, aquisgrana, europa vetus From ianb at colorstudy.com Wed May 18 20:21:14 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 18 May 2005 13:21:14 -0500 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <17035.31628.509536.426587@montanaro.dyndns.org> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <17035.31628.509536.426587@montanaro.dyndns.org> Message-ID: <428B879A.7000700@colorstudy.com> Skip Montanaro wrote: > >> 1. reference docs: ... > >> > >> 2. user documentation: ... > > Ian> I think that's a good distinction. And like several people I'm not > Ian> very interested in 1. > > Ah, but I started this thread thinking about #1. Well, not the #1 that Martin described (epydoc-like). The Python documentation is a perfect example of #2 -- documentation written completely by hand that provides a structured and concise description of a module. I shudder to think about what #1 would look like for the standard library. Well, that's not fair, I can see exactly what it looks like: http://epydoc.sourceforge.net/stdlib/ -- maybe "shuddering" isn't really appropriate (it's not that bad), but it's not at all like the reference docs we know and love. > FYI, while #2 is important, I'm still interested in this. I'm particularly > interested in being able to replace latex2html with something more flexible > that would allow relatively easy experimentation with the online > presentation of the core documentation. Well, getting back to your original question about ReST (I've also copied David Goodger and Felix Wiemann, as they might have more to say about this and I'm not sure if they are tracking this list)... * I believe ReST markup is a superset of the conventional Python Latex. I know a couple people worked on this at PyCon 2004, and they seemed to feel that local roles were the only missing part (to specifically mark a word as being a class name, for instance), and that was added to docutils a while ago in response. This doesn't address linking, indexing, or the build process -- just the local markup. * ReST is a superset, so there's a bunch of things that I think are much more easily expressed there than in the Latex. (Note: I have no idea what the actual Latex looks like, I only say this from familiarity with its output ;) The problem with this is that it may be hard to mix documents generated from the two sources and keep a consistent style. * There's no build process that would apply. This doesn't seem like a huge task, nor one that can really be addressed until someone actually tries it. * Interlinking documents may be difficult. ReST documents can't easily be merged or separated (from what I can tell), and there's no support for abstract links (you can only link internally or to concrete files). These are acknowledged problems with ReST, so there might ideas or even code to address this. * I'm not sure about indexing -- there's markup to express indexable terms (I believe), but I don't know if there's tools that actually make use of that. * That said, some of its internal-to-the-document indexing is better. It creates lots of anchors allowing deeper links and the contents headers are often useful. It's readily extensible on the local level by adding new directives and inline roles. * There's no tools that I know of to generate ReST. I think it would have to be based on heuristics, with manual editing. I think the warnings are pretty good, though, so I think this is doable without having to carefully proofread all of the output. * I personally find writing ReST pleasant, but I think Michael is right that it's not the Latex that keeps people from contributing. Personally, I think it has a lot to do with an intimidation factor; standard library docs are implicitly authoritative, so you don't write docs on a whim. Well, does anyone write docs on a whim? Anyway, if I was to analyze my own non-contribution, that'd probably be the reason. Anyway, I think commenting and annotation systems -- while often annoying and messy -- do offer a middleground for contributors. For the most part these could be applied to the current system; but ReST would make it easier, and has (some) more potential for streamlining the integration process, integrating comments and annotations into the actual documentation. * OTOH, complicated build tools are death to building a group of contributors -- it makes a huge difference, because contributors are grown over time, most don't start with the intention to be serious contributors. The offer to convert documents doesn't necessarily make its way to the people who would take you up on it. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From aahz at pythoncraft.com Wed May 18 20:46:06 2005 From: aahz at pythoncraft.com (Aahz) Date: Wed, 18 May 2005 11:46:06 -0700 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <428B879A.7000700@colorstudy.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <17035.31628.509536.426587@montanaro.dyndns.org> <428B879A.7000700@colorstudy.com> Message-ID: <20050518184605.GC142@panix.com> On Wed, May 18, 2005, Ian Bicking wrote: > > * I'm not sure about indexing -- there's markup to express indexable > terms (I believe), but I don't know if there's tools that actually make > use of that. I've written code for this, but I don't think it has been merged into the tree. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "And if that makes me an elitist...I couldn't be happier." --JMS From aahz at pythoncraft.com Wed May 18 20:49:24 2005 From: aahz at pythoncraft.com (Aahz) Date: Wed, 18 May 2005 11:49:24 -0700 Subject: [Doc-SIG] Python docs in reST? In-Reply-To: <2msm0n4j78.fsf@starship.python.net> References: <17032.3517.362406.820105@montanaro.dyndns.org> <2msm0n4j78.fsf@starship.python.net> Message-ID: <20050518184924.GD142@panix.com> On Mon, May 16, 2005, Michael Hudson wrote: > > I believe there has been a long-ish term standing offer from assorted > people to turn plain text docs into suitable latex for the docs (if > not, let me make one). That's half-true. What isn't true about it is that there is still some missing publicity on this offer IMO. Only recently was a change committed to the docs making this offer clear in the docs, and even after that, there were people on c.l.py who thought that LaTeX was a requirement. I think that if you want to coordinate a group of people working on this, posting the offer to c.l.py.announce might be a good idea. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "And if that makes me an elitist...I couldn't be happier." --JMS From ianb at colorstudy.com Wed May 18 23:52:18 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 18 May 2005 16:52:18 -0500 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <8393fff050518104323989f74@mail.gmail.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <8393fff050518104323989f74@mail.gmail.com> Message-ID: <428BB912.2070409@colorstudy.com> Martin Blais wrote: > On 5/18/05, Ian Bicking wrote: >>But I have a lot of middling documentation. It's not doctestable, it's >>not structured so that an interface is useful, the code is in modules I >>don't expect the user to import specifically, and I am wary of just >>creating a separate text file because of API drift; I've found it hard >>to keep track of the information in two places. > > > i'm going to sound like a sucker here, but why not just put it "next" > to the module file, in a .txt file?, e.g. > > module.py > module.txt > > won't hurt the code, and it's right there a C-xC-f away... (I don't > like to have docs there somehow (and i suppose you don't either > because nobody does it), but i really cannot say why...) I don't know why either, but somehow it seems Wrong. Because it breaks tab completion? Because it's annoying to use with distutils? Eh, those probably aren't the reasons. In part I guess it's because I like docstrings. I don't want to leave my module bare because it's all in a separate .txt file. And I definitely don't want to duplicate the documentation in both places. At the same time, I also want people to be able to read docs without browsing through the source, and people look in docs/ first for that stuff. And there are docs that definitely don't belong besides the source. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From simon at arrowtheory.com Thu May 19 02:57:45 2005 From: simon at arrowtheory.com (Simon Burton) Date: Thu, 19 May 2005 10:57:45 +1000 Subject: [Doc-SIG] Python docs in reST? In-Reply-To: <2msm0n4j78.fsf@starship.python.net> References: <17032.3517.362406.820105@montanaro.dyndns.org> <2msm0n4j78.fsf@starship.python.net> Message-ID: <20050519105745.45e58dbf.simon@arrowtheory.com> On Mon, 16 May 2005 10:24:27 +0100 Michael Hudson wrote: > > Skip Montanaro writes: > > > A thread on c.l.py ("Python Documentation (should be better?)") got me to > > thinking about options for improving/restructuring the online documentation. > > That led me to thinking about converting the current docs to reST. A scan > > of the last year's worth of list archives didn't yield any obvious > > discussion on the topic, but I find it hard to believe the topic hasn't been > > discussed in the past. Pointers to past threads cheerfully accepted. > > I thought it had too, but maybe only on python-list. > > I'm not so sure it would make much difference, but maybe that's just > my cynicism. > > > It seems to me that converting to reST has a couple obvious benefits: > > > > * dump latex2html > > It works most of the time these days! ... I should chime in here with another viewpoint. I do a lot of scientific/engineering python, and have been using python's latex2html system for one good reason: we need math expressions in the docs. Originally I cooked up a script to rip docstrings, introspect function signatures, and write the latex source out for the API docs. It quickly got out of hand: maybe because my little script was too much of a hack, or maybe because the latex2html system does not really lend itself well to the "three pane"-navigatable documentation style, especially for deeply nested packages. Perhaps this could be improved with some more latex macros; I don't know (latex scares the willies out of me). Where I work now, they recently moved to python from a proprietary system. Mostly I think everyone is enjoying the change, but my boss is shocked at the lack of a "standard" python documentation system, and that the python docs are maintained separately, by hand. We looked at epydoc. I've had one (lame) attempt at integrating latex-math, no luck so far. Now the boss is willing to pay a bounty for this feature, though I think we need someone local (to Canberra, Australia). Simon. -- Simon Burton, B.Sc. Licensed PO Box 8066 ANU Canberra 2601 Australia Ph. 61 02 6249 6940 http://arrowtheory.com From fdrake at acm.org Thu May 19 04:46:29 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Wed, 18 May 2005 22:46:29 -0400 Subject: [Doc-SIG] Python docs in reST? In-Reply-To: <17032.3517.362406.820105@montanaro.dyndns.org> References: <17032.3517.362406.820105@montanaro.dyndns.org> Message-ID: <200505182246.29887.fdrake@acm.org> On Sunday 15 May 2005 23:04, Skip Montanaro wrote: > Fred wrote some Latex->ESIS->SGML tools that look like they might possibly > be redirected to the more-or-less one-time task of converting to reST. Actually, I think they generate XML at this point, but I've not looked at them in a long time. Probably the best approach would be to simply finish that task, and then write XSLT transforms to take it the rest of the way. But it really depends on who has time to work on it; I'm afraid I've been a bit busy lately. -Fred -- Fred L. Drake, Jr. From bronger at physik.rwth-aachen.de Thu May 19 06:19:18 2005 From: bronger at physik.rwth-aachen.de (Torsten Bronger) Date: Thu, 19 May 2005 06:19:18 +0200 Subject: [Doc-SIG] Python docs in reST? References: <17032.3517.362406.820105@montanaro.dyndns.org> <2msm0n4j78.fsf@starship.python.net> <20050519105745.45e58dbf.simon@arrowtheory.com> Message-ID: <87fywjn8zd.fsf@wilson.rwth-aachen.de> Hall?chen! Simon Burton writes: > [...] > > Where I work now, they recently moved to python from a proprietary > system. Mostly I think everyone is enjoying the change, but my > boss is shocked at the lack of a "standard" python documentation > system, and that the python docs are maintained separately, by > hand. Here again I see this unfortunate ambiguity of "docs", menitoned in 8393fff05051803515b0e2a93 at mail.gmail.com by Martin Blais earlier: What kind of documentation do you mean? Tsch?, Torsten. -- Torsten Bronger, aquisgrana, europa vetus From bronger at physik.rwth-aachen.de Thu May 19 06:52:08 2005 From: bronger at physik.rwth-aachen.de (Torsten Bronger) Date: Thu, 19 May 2005 06:52:08 +0200 Subject: [Doc-SIG] Python docs in reST References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> Message-ID: <87acmrn7gn.fsf@wilson.rwth-aachen.de> Hall?chen! Martin Blais writes: > [...] > > 1. reference docs: its structure mirrors the structure of the > code, and it is often or best automatically generated from the > source code and comments in the source code. It does not matter > very much if the private methods are visible in this > documentation; > > 2. user documentation: a manual written with the specific purpose > of educating a person about how to use the specific module. its > structure does not mirror that of the source code at all. It does > not describe all the source code, just what is needed. There are > code examples. There might be different manuals for different > types of users written from different angles. I agree almost fully. Maybe I just didn't get you totally correct, nevertheless, I'd like to comment on it. I've never been involved in large software projects, but I consider (1) as mere source code documentation, rather than real reference (not a useful reference, at least). I think "TeX -- The Program" and "The TeXbook" reflect very well how (1) and (2) should be in my opinion: Good documentation of the source code itself (needn't be 'literate') for people who wish to contribute/bugfix/maintain, and a real human-generated user documentation. The latter must be complete (it may contain a tutorial, though). Granted, you have to maintain both, but there is no silver bullet, and I've found it the lesser evil so far. If you try to force the user to have to look at a reference generated from the source, it's either quite awkward documentation, or you end up having two files in one, effectively. Tsch?, Torsten. -- Torsten Bronger, aquisgrana, europa vetus From fdrake at acm.org Thu May 19 06:59:34 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Thu, 19 May 2005 00:59:34 -0400 Subject: [Doc-SIG] two subjects: docutils for python docs, and integrating documentation in source code In-Reply-To: <8393fff05042206014362b8fd@mail.gmail.com> References: <8393fff0504201039723adaf0@mail.gmail.com> <874qdzngom.fsf@news2.ososo.de> <8393fff05042206014362b8fd@mail.gmail.com> Message-ID: <200505190059.35124.fdrake@acm.org> On Friday 22 April 2005 09:01, Martin Blais wrote: > but note that these generate index entries for code declarations (i.e. > function, class, data). these are "easy" since they have their own > syntactic units. what i meant is that it is not currently possible to > generate generic index entries with docutils (not that i know of > anyway), e.g. in a paragraph of normal text, how do i indicate that a > word in the paragraph should appear in the index? We actually don't have a construct for that in the LaTeX, either. Index entries are made in addition to the actual text. It would be easy to add something like that, though, but I don't know that it would be used often. We have a lot of specialized LaTeX macros for index entries now, to support various combinations of nouns and adjectives with up to four words being combined in near-combinatorial patterns. We probably don't need that many; most are not used extensively, and rarely in newer documentation. (Newer documentation is more likely to contain multiple index entries with just some of the combinations, which seems more sane to me.) > we should also see what can be improved. in particular, the following > things should be improved about the current documentatoin input: > > - the optional parameters markup should be grouped rather than nested with > \optional. it makes more sense to declare them this way in Python > because of the keywords syntax for function calls (e.g. foot(bli=4)); The \optional stuff has become very painful; I'm interested in suggestions on how it should be structured. The existing markup is based very heavily in what was in Python from 1995-1998; things have changed a good bit since then. Some issues that have to be considered: - For some functions (especially common with those implemented in C), nesting things using \optional{arg\optional{, arg}} actually reflects the call signature. We have to be able to do this. - Some functions accept keyword-only arguments that can't be provided using positional arguments at all (using **keywords). We don't have a good way of dealing with this now. - Some functions have signatures that are best described using multiple signatures. (Consider type() as an example.) For these, some benefit from separate descriptions for each signature, and others can reasonably share descriptions among signatures. > - there is no way to declare the base classes. The problem is that the > syntax for the constructor (documentation) takes the space of inheritance > (source code); I agree that this is a problem. We also don't have good ways to indicate class methods and static methods. The need to describe things provided by specialized descriptors will only become more complicated as @decorators gain more traction. > - there needs to be a way to write a documentation paragraph for many > methods/functions at the same time. i haven't found how to do this > nicely yet, We need a better way to display these groupings as well; they aren't terribly obvious in the existing LaTeX documentation where they exist. I suspect the right presentation will be very different in the HTML and typeset versions. > apart from breaking the declarations of functions outside of the > classdesc; I don't know what you mean by this. Are you actually describing methods inside the {classdesc} environment using nested {methoddesc}s and {memberdesc}s? > - there should be a formal syntax to describe the role of each of the > arguments, as well as for the return value. Parsing the code for doc > generation should validate > that all the arguments are covered in the documentation string (this > could be an > option, and a condition for merging code). many people have suggested > ways of doing this. Verification can certainly be optional. It would also need have a way to say "skip it for this function" in cases where magical games are being played with *args and **kw. And it would need to undestand that documenting an argument in a base class may be sufficient (that's a policy decision). -Fred -- Fred L. Drake, Jr. From bronger at physik.rwth-aachen.de Thu May 19 09:34:09 2005 From: bronger at physik.rwth-aachen.de (Torsten Bronger) Date: Thu, 19 May 2005 09:34:09 +0200 Subject: [Doc-SIG] Python docs in reST? References: <05May18.144608pdt.58617@synergy1.parc.xerox.com> Message-ID: <87oeb77jpq.fsf@wilson.rwth-aachen.de> Hall?chen! Bill Janssen writes: >> LaTeX seems to be a we-want-to-get-rid-of-it, and reST is still not >> really ready. > > I decided to do the documentation for my latest project, UpLib, in > reST, to see how well it works. My conclusion is that it probably > will never be ready to document Python, because the design goal of > making the reST source read like text conflicts with the goal of > making it functional enough to document a large system. I'm not so pessimistic. I think that only a few constructs are missing to make it suitable for large projects (inclusion and interlinks). But apparently this is already being tackled. For non-Python user guides I use Texinfo. What I really miss is Texinfo's @deffn for explaining functions etc. Yes, I know , but it doesn't solve this issue. reST lacks a standard means of defining function/method signatures, return values, exceptions etc. (Similar structures are needed for classes.) I use field lists at the moment. That's okay, but it's not real logical markup since the reST interpreter has no chance to recognise it as a function definition. Besides, its pdf and HTML results are poor. Other missing things include standardised meta info, i18n (including encodings), formulae (a well-defined subset of LaTeX would be good), and a meta standard for how to use "interpreted text". > [...] > > [...] but next time I'll probably use a simple text markup > language like troff with an appropriate macro package, or better > yet runoff. Do you really think that troff is suitable for large, modern documentation? How many people would have to learn it? How easy is it to install it on Windows? How good are the PDFs generated from it? How is it converted to HTML? (I don't know troff very well, as you can see. ;) Tsch?, Torsten. -- Torsten Bronger, aquisgrana, europa vetus From mwh at python.net Thu May 19 10:08:31 2005 From: mwh at python.net (Michael Hudson) Date: Thu, 19 May 2005 09:08:31 +0100 Subject: [Doc-SIG] Python docs in reST? In-Reply-To: <87is1g4ejs.fsf@wilson.rwth-aachen.de> (Torsten Bronger's message of "Wed, 18 May 2005 19:41:43 +0200") References: <17032.3517.362406.820105@montanaro.dyndns.org> <2msm0n4j78.fsf@starship.python.net> <87is1g4ejs.fsf@wilson.rwth-aachen.de> Message-ID: <2mfywj4ozk.fsf@starship.python.net> Torsten Bronger writes: > Hall?chen! > > Michael Hudson writes: > >> [...] >> >> I believe there has been a long-ish term standing offer from >> assorted people to turn plain text docs into suitable latex for >> the docs (if not, let me make one). >> >> The main reason that Python's documentation is patchy in places is >> because writing good content is hard! Dealing with *any* format >> is minor in comparison. > > The willingness to contribute good content is an important premise, > no doubt. However, it is also important that the community has > agreed on one format. Well, there are two questions here, which possibly -- but don't -- have the same answer: "what format is the documentation for Python in?" and "what format should I use for documenting my Python module?" The answer to the first has been "Latex" for about 10 years (and was "framemaker" before that, I think). The second doesn't really have a canonical answer, and I get the feeling that this is what bothers people. Reasonable answers are "Latex", like CPython, or "ReST". > I'm not prepared to write in (or worse, convert to) a certain format > that's abandonned later. > > I came to Python three months ago, and it's still not clear to me > which format one should use for the documentation (which is very > important to me and, in contrast to most developers, I enjoy to > create). Here you're talking about the second question, right? > LaTeX seems to be a we-want-to-get-rid-of-it, and reST is still not > really ready. (I used the latter, nevertheless. My main reason was > that LaTeX cannot be autoconverted reliably.) I'm not sure what you mean for the latter. > So all in all, I do think that Python would benefit from a clear > homogeneous documentation policy, combined with strong reST tools. I'm not going to argue with that! Cheers, mwh -- Ability to type on a computer terminal is no guarantee of sanity, intelligence, or common sense. -- Gene Spafford's Axiom #2 of Usenet From mwh at python.net Thu May 19 10:10:29 2005 From: mwh at python.net (Michael Hudson) Date: Thu, 19 May 2005 09:10:29 +0100 Subject: [Doc-SIG] Python docs in reST? In-Reply-To: <17035.32053.771431.733525@montanaro.dyndns.org> (Skip Montanaro's message of "Wed, 18 May 2005 12:36:53 -0500") References: <17032.3517.362406.820105@montanaro.dyndns.org> <2msm0n4j78.fsf@starship.python.net> <17035.32053.771431.733525@montanaro.dyndns.org> Message-ID: <2mbr774owa.fsf@starship.python.net> Skip Montanaro writes: > >> * dump latex2html > > Michael> It works most of the time these days! > > I was under the impression that Fred was still dealing with a forked version > of the code and that it was still fairly fragile. No, we use stock latex2html. There a bunch of style files and customizations on top that probably count as a little fragile but they certainly work for me. > >> * better potential HTML generation options (like HTML w/ an integrated > >> annotation capability) > > Michael> But only, very much "potential". > > Sure, but how much experimentation do you think is practical using > latex2html? Not a lot, I'll grant you. More if you're called "Fred" :) > >> * possibly enlarge pool of documentation contributors > > Michael> Yeah, right. This is where my lack of faith in humanity comes > Michael> out. > > Understood. > > Michael> I believe there has been a long-ish term standing offer from > Michael> assorted people to turn plain text docs into suitable latex for > Michael> the docs (if not, let me make one). > > A number of people (myself included) have made this offer on multiple > occsions, generally in response to some variant of "Python's documentation > sucks". (Which we all know it doesn't, but they complain just the same.) Yup. > Michael> The main reason that Python's documentation is patchy in places > Michael> is because writing good content is hard! Dealing with *any* > Michael> format is minor in comparison. > > I would be happy if people contributed examples or submitted enhancement > requests that referred to cookbook recipes. Me too. Cheers, mwh -- I'm okay with intellegent buildings, I'm okay with non-sentient buildings. I have serious reservations about stupid buildings. -- Dan Sheppard, ucam.chat (from Owen Dunn's summary of the year) From mike at pcblokes.com Thu May 19 10:16:14 2005 From: mike at pcblokes.com (Michael Foord) Date: Thu, 19 May 2005 09:16:14 +0100 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <20050518184605.GC142@panix.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <17035.31628.509536.426587@montanaro.dyndns.org> <428B879A.7000700@colorstudy.com> <20050518184605.GC142@panix.com> Message-ID: <428C4B4E.9030304@pcblokes.com> Aahz wrote: >On Wed, May 18, 2005, Ian Bicking wrote: > > >>* I'm not sure about indexing -- there's markup to express indexable >>terms (I believe), but I don't know if there's tools that actually make >>use of that. >> >> > >I've written code for this, but I don't think it has been merged into >the tree. > > I'd really like to see this go into docutils. it's a major step towards having docutils handle multipage documents effectively. Regards, Fuzzy http://www.voidspace.org.uk/python From mwh at python.net Thu May 19 10:18:54 2005 From: mwh at python.net (Michael Hudson) Date: Thu, 19 May 2005 09:18:54 +0100 Subject: [Doc-SIG] Python docs in reST? In-Reply-To: <8393fff05051810511ac527fe@mail.gmail.com> (Martin Blais's message of "Wed, 18 May 2005 13:51:20 -0400") References: <17032.3517.362406.820105@montanaro.dyndns.org> <2msm0n4j78.fsf@starship.python.net> <8393fff05051810511ac527fe@mail.gmail.com> Message-ID: <2m7jhv4oi9.fsf@starship.python.net> Martin Blais writes: [python's documentation system] > 2. it doesn't "just work", you have to rip a lot of stuff out of the > given makefile to make it do the minimal thing for a new documentation > set, and it has a lot of stuff that I didn't need, like isilo, custom > bits for the python docs, etc. It took me a little while to get it > going, it would have been nicer to do something like "emerge > python-doc-system" and copy a few clear rules from somewhere else. That's not a bad idea at all. Note, though, that the easiest way of getting the tools working (for me :) is symlinking ~/bin/mkhowto -> ~/Source/python/dist/src/Doc/tools/mkhowto and then, er, keeping the mkhowto invocation in my shell history. This could probably be streamlined, yes. Cheers, mwh -- All obscurity will buy you is time enough to contract venereal diseases. -- Tim Peters, python-dev From mwh at python.net Thu May 19 10:20:36 2005 From: mwh at python.net (Michael Hudson) Date: Thu, 19 May 2005 09:20:36 +0100 Subject: [Doc-SIG] Python docs in reST? In-Reply-To: <20050519105745.45e58dbf.simon@arrowtheory.com> (Simon Burton's message of "Thu, 19 May 2005 10:57:45 +1000") References: <17032.3517.362406.820105@montanaro.dyndns.org> <2msm0n4j78.fsf@starship.python.net> <20050519105745.45e58dbf.simon@arrowtheory.com> Message-ID: <2m3bsj4off.fsf@starship.python.net> Simon Burton writes: > Where I work now, they recently moved to python from a proprietary > system. Mostly I think everyone is enjoying the change, but my boss > is shocked at the lack of a "standard" python documentation system, > and that the python docs are maintained separately, by hand. Well, that's because the standard Python documentation is "type 2" documentation (i.e. user docs) and any attempt to autogenerate those is doomed... Cheers, mwh -- This makes it possible to pass complex object hierarchies to a C coder who thinks computer science has made no worthwhile advancements since the invention of the pointer. -- Gordon McMillan, 30 Jul 1998 From bronger at physik.rwth-aachen.de Thu May 19 10:59:56 2005 From: bronger at physik.rwth-aachen.de (Torsten Bronger) Date: Thu, 19 May 2005 10:59:56 +0200 Subject: [Doc-SIG] Python docs in reST? References: <17032.3517.362406.820105@montanaro.dyndns.org> <2msm0n4j78.fsf@starship.python.net> <87is1g4ejs.fsf@wilson.rwth-aachen.de> <2mfywj4ozk.fsf@starship.python.net> Message-ID: <87hdgz7fqr.fsf@wilson.rwth-aachen.de> Hall?chen! Michael Hudson writes: > Torsten Bronger writes: > >> [...] > > Well, there are two questions here, which possibly -- but don't -- > have the same answer: "what format is the documentation for Python > in?" and "what format should I use for documenting my Python > module?" In my opinion they should have the same answer. Tsch?, Torsten. -- Torsten Bronger, aquisgrana, europa vetus From mike at pcblokes.com Thu May 19 11:17:20 2005 From: mike at pcblokes.com (Michael Foord) Date: Thu, 19 May 2005 10:17:20 +0100 Subject: [Doc-SIG] Python docs in reST? In-Reply-To: <87hdgz7fqr.fsf@wilson.rwth-aachen.de> References: <17032.3517.362406.820105@montanaro.dyndns.org> <2msm0n4j78.fsf@starship.python.net> <87is1g4ejs.fsf@wilson.rwth-aachen.de> <2mfywj4ozk.fsf@starship.python.net> <87hdgz7fqr.fsf@wilson.rwth-aachen.de> Message-ID: <428C59A0.9060804@pcblokes.com> Torsten Bronger wrote: >Hall?chen! > >Michael Hudson writes: > > > >>Torsten Bronger writes: >> >> >> >>>[...] >>> >>> >>Well, there are two questions here, which possibly -- but don't -- >>have the same answer: "what format is the documentation for Python >>in?" and "what format should I use for documenting my Python >>module?" >> >> > >In my opinion they should have the same answer. > > > Not if the answer to the first question is Latex ;-) Fuzzy http://www.voidspace.org.uk/python >Tsch?, >Torsten. > > > From fdrake at acm.org Thu May 19 15:07:46 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Thu, 19 May 2005 09:07:46 -0400 Subject: [Doc-SIG] Python docs in reST? In-Reply-To: <2m7jhv4oi9.fsf@starship.python.net> References: <17032.3517.362406.820105@montanaro.dyndns.org> <8393fff05051810511ac527fe@mail.gmail.com> <2m7jhv4oi9.fsf@starship.python.net> Message-ID: <200505190907.47070.fdrake@acm.org> On Thursday 19 May 2005 04:18, Michael Hudson wrote: [re: making the doc tools easier to use outside the Python sources] > That's not a bad idea at all. And one I started work on at one point, but never had time to make much progress on. But that was at a time when we were still talking about SGML. > Note, though, that the easiest way of getting the tools working (for > me :) is symlinking > > ~/bin/mkhowto -> ~/Source/python/dist/src/Doc/tools/mkhowto Yes. > and then, er, keeping the mkhowto invocation in my shell history. > This could probably be streamlined, yes. "mkhowto mydocs.tex" is hard, how? Some sort of configuration file would be handy, perhaps, and make it easier to reproduce specific sets of options. I'd be open to patches. ;-) -Fred -- Fred L. Drake, Jr. From mwh at python.net Thu May 19 15:31:10 2005 From: mwh at python.net (Michael Hudson) Date: Thu, 19 May 2005 14:31:10 +0100 Subject: [Doc-SIG] Python docs in reST? In-Reply-To: <200505190907.47070.fdrake@acm.org> (Fred L. Drake, Jr.'s message of "Thu, 19 May 2005 09:07:46 -0400") References: <17032.3517.362406.820105@montanaro.dyndns.org> <8393fff05051810511ac527fe@mail.gmail.com> <2m7jhv4oi9.fsf@starship.python.net> <200505190907.47070.fdrake@acm.org> Message-ID: <2mu0kz2vhd.fsf@starship.python.net> "Fred L. Drake, Jr." writes: > On Thursday 19 May 2005 04:18, Michael Hudson wrote: > [re: making the doc tools easier to use outside the Python sources] > > > and then, er, keeping the mkhowto invocation in my shell history. > > This could probably be streamlined, yes. > > "mkhowto mydocs.tex" is hard, how? I have memories of long "-a" options, at least. > Some sort of configuration file would be handy, perhaps, and make it > easier to reproduce specific sets of options. I'd be open to > patches. ;-) Yeah, that would be nice :) Cheers, mwh -- Considering that this thread is completely on-topic in the way only c.l.py threads can be, I think I can say that you should replace "Oblivion" with "Gravity", and increase your Radiohead quotient. -- Ben Wolfson, comp.lang.python From fdrake at acm.org Thu May 19 15:42:15 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Thu, 19 May 2005 09:42:15 -0400 Subject: [Doc-SIG] two subjects: docutils for python docs, and integrating documentation in source code In-Reply-To: <4276EA3B.30404@python.org> References: <8393fff0504201039723adaf0@mail.gmail.com> <87d5scyycs.fsf@news2.ososo.de> <4276EA3B.30404@python.org> Message-ID: <200505190942.16014.fdrake@acm.org> On Monday 02 May 2005 23:04, David Goodger wrote: > [Felix Wiemann] > > Maybe add a special token to the docstring that the method shouldn't > > appear in auto-documentation? > > A special comment perhaps? Or a function attribute? I don't think > such metadata should be put in the docstring; that's for > documentation. I'd be reasonably happy with the special comment approach, if we generate documentation by parsing. The more dynamic things become in Python, the less convinced I am that parsing alone is actually a workable approach. It might be easier to use special comments if the compiler understood them to set attributes on the objects marked. More realistically, we could very easily use decorators with Python 2.4 and newer. Unless someone has time to work on this, that's probably good enough. It might not be hard to adjust epydoc to use annotations added that way. For example:: @docinfo(hidden=True) def my_really_magical_function(*args, **kw): pass Something like this could also be used to indicate grouping:: @docinfo(group='Magical Things') def my_public_magical_function(*args, **kw): pass > OTOH, the common convention for "weakly private" or "internal" can be > used: a single underscore prefix. E.g.: I think this is a really weak convention. It might be helpful for eliminating non-public things, but it's not unusual for APIs intended for subclasses to include the single underscore prefix. We'd definately want some way to denote these categorizations explicitly. -Fred -- Fred L. Drake, Jr. From fdrake at acm.org Thu May 19 15:43:53 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Thu, 19 May 2005 09:43:53 -0400 Subject: [Doc-SIG] Python docs in reST? In-Reply-To: <2mu0kz2vhd.fsf@starship.python.net> References: <17032.3517.362406.820105@montanaro.dyndns.org> <200505190907.47070.fdrake@acm.org> <2mu0kz2vhd.fsf@starship.python.net> Message-ID: <200505190943.53545.fdrake@acm.org> On Thursday 19 May 2005 09:31, Michael Hudson wrote: > I have memories of long "-a" options, at least. Ah, for the footer. Yes, this is where configuration could help a lot. -Fred -- Fred L. Drake, Jr. From skip at pobox.com Thu May 19 15:47:48 2005 From: skip at pobox.com (Skip Montanaro) Date: Thu, 19 May 2005 08:47:48 -0500 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <428B879A.7000700@colorstudy.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <17035.31628.509536.426587@montanaro.dyndns.org> <428B879A.7000700@colorstudy.com> Message-ID: <17036.39172.175861.627680@montanaro.dyndns.org> >>>>> "Ian" == Ian Bicking writes: Ian> Skip Montanaro wrote: >> >> 1. reference docs: ... >> >> >> >> 2. user documentation: ... >> Ian> I think that's a good distinction. And like several people I'm not Ian> very interested in 1. >> >> Ah, but I started this thread thinking about #1. Ian> Well, not the #1 that Martin described (epydoc-like). The Python Ian> documentation is a perfect example of #2 -- documentation written Ian> completely by hand that provides a structured and concise Ian> description of a module. Yes, mea culpa. I had them flipped. Dunno why. Skip From fdrake at acm.org Thu May 19 15:50:16 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Thu, 19 May 2005 09:50:16 -0400 Subject: [Doc-SIG] two subjects: docutils for python docs, and integrating documentation in source code In-Reply-To: <4268C83C.1050301@pcblokes.com> References: <8393fff0504201039723adaf0@mail.gmail.com> <2mwtqv40xx.fsf@starship.python.net> <4268C83C.1050301@pcblokes.com> Message-ID: <200505190950.16917.fdrake@acm.org> On Friday 22 April 2005 05:47, Michael Foord wrote: > Right - is Python documentation really written straight into LaTex markup > ? Is that so hard to believe? While LaTeX has a messy side, I find it generally quite nice to work with as an author; the markup is fairly clean in that it doesn't introduce much visual clutter at all. Yes, the authoring happens directly in LaTeX. For anyone who wants to contribute content without learning LaTeX, many of us here (including myself!) have offered to add the markup ourselves. The hard part is still coming up with useful content. -Fred -- Fred L. Drake, Jr. From skip at pobox.com Thu May 19 15:59:51 2005 From: skip at pobox.com (Skip Montanaro) Date: Thu, 19 May 2005 08:59:51 -0500 Subject: [Doc-SIG] Python docs in reST? In-Reply-To: <200505190907.47070.fdrake@acm.org> References: <17032.3517.362406.820105@montanaro.dyndns.org> <8393fff05051810511ac527fe@mail.gmail.com> <2m7jhv4oi9.fsf@starship.python.net> <200505190907.47070.fdrake@acm.org> Message-ID: <17036.39895.465459.780262@montanaro.dyndns.org> >> Note, though, that the easiest way of getting the tools working (for >> me :) is symlinking >> >> ~/bin/mkhowto -> ~/Source/python/dist/src/Doc/tools/mkhowto Fred> Yes. Shouldn't we modify the Python installation Makefile to install mkhowto? Skip From fdrake at acm.org Thu May 19 16:07:31 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Thu, 19 May 2005 10:07:31 -0400 Subject: [Doc-SIG] Python docs in reST? In-Reply-To: <17036.39895.465459.780262@montanaro.dyndns.org> References: <17032.3517.362406.820105@montanaro.dyndns.org> <200505190907.47070.fdrake@acm.org> <17036.39895.465459.780262@montanaro.dyndns.org> Message-ID: <200505191007.31879.fdrake@acm.org> On Thursday 19 May 2005 09:59, Skip Montanaro wrote: > Shouldn't we modify the Python installation Makefile to install mkhowto? Only if we're going to install all the things it uses; there's actually quite a pile of things that are needed. I need to spend some time figuring out how to install those things so that mkhowto can find it at runtime. I also have a difficult time with the "mkhowto" name; this feels almost presumptiously general. Maybe I should just get over that, since I've not been able to think of anything better. -Fred -- Fred L. Drake, Jr. From skip at pobox.com Thu May 19 16:12:30 2005 From: skip at pobox.com (Skip Montanaro) Date: Thu, 19 May 2005 09:12:30 -0500 Subject: [Doc-SIG] Python docs in reST? In-Reply-To: <200505191007.31879.fdrake@acm.org> References: <17032.3517.362406.820105@montanaro.dyndns.org> <200505190907.47070.fdrake@acm.org> <17036.39895.465459.780262@montanaro.dyndns.org> <200505191007.31879.fdrake@acm.org> Message-ID: <17036.40654.537550.129710@montanaro.dyndns.org> Fred> On Thursday 19 May 2005 09:59, Skip Montanaro wrote: >> Shouldn't we modify the Python installation Makefile to install >> mkhowto? Fred> Only if we're going to install all the things it uses; Understood. Not as simple as a standalone script. Fred> I also have a difficult time with the "mkhowto" name; this feels Fred> almost presumptiously general. Maybe I should just get over that, Fred> since I've not been able to think of anything better. mkpyhowto? Skip From aahz at pythoncraft.com Thu May 19 18:59:52 2005 From: aahz at pythoncraft.com (Aahz) Date: Thu, 19 May 2005 09:59:52 -0700 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <428C4B4E.9030304@pcblokes.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <17035.31628.509536.426587@montanaro.dyndns.org> <428B879A.7000700@colorstudy.com> <20050518184605.GC142@panix.com> <428C4B4E.9030304@pcblokes.com> Message-ID: <20050519165951.GD24084@panix.com> On Thu, May 19, 2005, Michael Foord wrote: > Aahz wrote: >>On Wed, May 18, 2005, Ian Bicking wrote: >>> >>>* I'm not sure about indexing -- there's markup to express indexable >>>terms (I believe), but I don't know if there's tools that actually make >>>use of that. >> >>I've written code for this, but I don't think it has been merged into >>the tree. > > I'd really like to see this go into docutils. it's a major step towards > having docutils handle multipage documents effectively. Feel free to go into my sandbox and upgrade it! ;-) -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "The only problem with Microsoft is they just have no taste." --Steve Jobs From janssen at parc.xerox.com Fri May 20 00:58:50 2005 From: janssen at parc.xerox.com (Bill Janssen) Date: Thu, 19 May 2005 15:58:50 PDT Subject: [Doc-SIG] Python docs in reST? In-Reply-To: Your message of "Thu, 19 May 2005 00:34:09 PDT." <87oeb77jpq.fsf@wilson.rwth-aachen.de> Message-ID: <05May19.155855pdt."58617"@synergy1.parc.xerox.com> > Do you really think that troff is suitable for large, modern > documentation? Yes. Entire operating systems have been documented with it. In addition, the excellent GNU Troff (groff) system by James Clark and a host of others will produce Postscript or HTML output, among other formats. But I like even simpler systems, such as runoff or roff. > How many people would have to learn it? For my purposes, two or three. > How easy is it to install it on Windows? No idea. I try to avoid that particular tarpit. > How good are the PDFs generated from it? Very good. > How is it converted to HTML? The tool "grops" included with groff. Bill From janssen at parc.xerox.com Fri May 20 01:38:07 2005 From: janssen at parc.xerox.com (Bill Janssen) Date: Thu, 19 May 2005 16:38:07 PDT Subject: [Doc-SIG] Python docs in reST? In-Reply-To: Your message of "Thu, 19 May 2005 15:58:50 PDT." <05May19.155855pdt."58617"@synergy1.parc.xerox.com> Message-ID: <05May19.163810pdt."58617"@synergy1.parc.xerox.com> > > How is it converted to HTML? > > The tool "grops" included with groff. Oops. I meant to say "grohtml". Bill From kenneth.m.mcdonald at gmail.com Fri May 20 02:51:45 2005 From: kenneth.m.mcdonald at gmail.com (Kenneth McDonald) Date: Thu, 19 May 2005 19:51:45 -0500 Subject: [Doc-SIG] nroff/troff In-Reply-To: References: Message-ID: I haven't used these systems in quite a while so I don't know what state they are in, but I still like the way they did things far better than any other system I've seen. Much, much easier to learn than LaTeX, but still very capable. Just wanted to comment. Come to think of it, they might make a decent markup language for incode documentation too. And they really are easy to learn, at least to get off the ground with. Cheers, Ken > ---------- Forwarded message ---------- > From: Bill Janssen > To: Torsten Bronger > Date: Thu, 19 May 2005 15:58:50 PDT > Subject: Re: [Doc-SIG] Python docs in reST? > > Do you really think that troff is suitable for large, modern > > documentation? > > Yes. Entire operating systems have been documented with it. In > addition, the excellent GNU Troff (groff) system by James Clark and a > host of others will produce Postscript or HTML output, among other > formats. > > But I like even simpler systems, such as runoff or roff. > From aahz at pythoncraft.com Fri May 20 02:58:22 2005 From: aahz at pythoncraft.com (Aahz) Date: Thu, 19 May 2005 17:58:22 -0700 Subject: [Doc-SIG] nroff/troff In-Reply-To: References: Message-ID: <20050520005822.GB10366@panix.com> On Thu, May 19, 2005, Kenneth McDonald wrote: > > I haven't used these systems in quite a while so I don't know what > state they are in, but I still like the way they did things far better > than any other system I've seen. Much, much easier to learn than > LaTeX, but still very capable. Just wanted to comment. > > Come to think of it, they might make a decent markup language for > incode documentation too. > > And they really are easy to learn, at least to get off the ground with. If we're switching away from LaTeX, we're going to reST. I don't think there's much debate about that. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "The only problem with Microsoft is they just have no taste." --Steve Jobs From mike at pcblokes.com Sat May 21 15:44:56 2005 From: mike at pcblokes.com (mike@pcblokes.com) Date: Sat, 21 May 2005 14:44:56 +0100 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <20050519165951.GD24084@panix.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <17035.31628.509536.426587@montanaro.dyndns.org> <428B879A.7000700@colorstudy.com> <20050518184605.GC142@panix.com> <428C4B4E.9030304@pcblokes.com> <20050519165951.GD24084@panix.com> Message-ID: <20050521144456.hlqaio6yx7kgwcsc@www.pcblokes.com> Quoting Aahz : > On Thu, May 19, 2005, Michael Foord wrote: >> Aahz wrote: >>> On Wed, May 18, 2005, Ian Bicking wrote: >>>> >>>> * I'm not sure about indexing -- there's markup to express indexable >>>> terms (I believe), but I don't know if there's tools that actually make >>>> use of that. >>> >>> I've written code for this, but I don't think it has been merged into >>> the tree. >> >> I'd really like to see this go into docutils. it's a major step towards >> having docutils handle multipage documents effectively. > > Feel free to go into my sandbox and upgrade it! ;-) I might just take you up on that. First I'm going to look at generating contents structures for several pages. This will allow me to generate a single contents page for a multi page document. The step *after* that is an index. I'm only making tentative steps into the docutils source though, so I have no idea of any timescale on this. Best Regards, Fuzzy http://www.voidspace.org.uk/python > -- > Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ > > "The only problem with Microsoft is they just have no taste." --Steve Jobs > _______________________________________________ > Doc-SIG maillist - Doc-SIG at python.org > http://mail.python.org/mailman/listinfo/doc-sig > From Felix.Wiemann at gmx.net Sun May 22 02:03:16 2005 From: Felix.Wiemann at gmx.net (Felix Wiemann) Date: Sun, 22 May 2005 02:03:16 +0200 Subject: [Doc-SIG] Python docs in reST References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> Message-ID: <87k6ls2kl7.fsf@news2.ososo.de> Ian Bicking wrote: > If the documentation is able to have intelligent interlinking, that'd > also be great; my experience with ReST is not great here, as it makes > strong distinctions between same-page links and external links. Yes. It would be nice if Docutils were able to merge several input documents into one, so that you could use internal links to link between the parts (input files). > Though maybe that could be easily fixed with a two-phase build > process; ReST identifies the links, outputs HTML, and then I resolve > all the links in a separate phase. This is more a hack (sorry), because you'd have to mess with HTML code. A clean implementation would have to deal with multiple input files and multiple output files as distinct and independent concepts, i.e. you shouldn't need to have a 1:1 relation between input and output files. I don't have enough time (and interest) to develop such multi-file support at the moment, though. It may become important when we try to convert the standard docs to reST, though. -- For private mail please ensure that the header contains 'Felix Wiemann'. http://www.ososo.de/ From ianb at colorstudy.com Sun May 22 08:44:33 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Sun, 22 May 2005 01:44:33 -0500 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <87k6ls2kl7.fsf@news2.ososo.de> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <87k6ls2kl7.fsf@news2.ososo.de> Message-ID: <42902A51.90703@colorstudy.com> Felix Wiemann wrote: >>Though maybe that could be easily fixed with a two-phase build >>process; ReST identifies the links, outputs HTML, and then I resolve >>all the links in a separate phase. > > > This is more a hack (sorry), because you'd have to mess with HTML code. > > A clean implementation would have to deal with multiple input files and > multiple output files as distinct and independent concepts, i.e. you > shouldn't need to have a 1:1 relation between input and output files. I > don't have enough time (and interest) to develop such multi-file support > at the moment, though. It may become important when we try to convert > the standard docs to reST, though. I think now is a good time for hacks, because we don't actually know what we're trying to create. There's no clear model that we can point to, so we need to be more experimental and less worried about the gracefulness of the underlying implementation. This HTML hack would be relatively easy to process right nwo. And really I think it's justifiable that interlinking and indexing be dealt with somewhat separately from much of the rest of ReST, so it's not architecturally bad, even if the means of communication with ReST are a bit crude. I'd be happier to see ugly code with great input and output right now, even if it was somewhat limited in its scope. That would be a good starting place. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From skip at pobox.com Sun May 22 18:16:09 2005 From: skip at pobox.com (Skip Montanaro) Date: Sun, 22 May 2005 11:16:09 -0500 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <42902A51.90703@colorstudy.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <87k6ls2kl7.fsf@news2.ososo.de> <42902A51.90703@colorstudy.com> Message-ID: <17040.45129.807736.306028@montanaro.dyndns.org> >>>>> "Ian" == Ian Bicking writes: Felix> This is more a hack (sorry), because you'd have to mess with HTML Felix> code. Felix> Felix> A clean implementation would have to deal with multiple input Felix> files and multiple output files as distinct and independent Felix> concepts ... Ian> I think now is a good time for hacks, because we don't actually Ian> know what we're trying to create... It's not like this is a new problem, is it? It seems to me we should be able to come up with a reasonable set of requirements and a design that addresses them. I'd like to avoid hacks that require we deal with HTML as an input format. Skip From martin.blais at gmail.com Sun May 22 20:39:44 2005 From: martin.blais at gmail.com (Martin Blais) Date: Sun, 22 May 2005 14:39:44 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <17040.45129.807736.306028@montanaro.dyndns.org> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <87k6ls2kl7.fsf@news2.ososo.de> <42902A51.90703@colorstudy.com> <17040.45129.807736.306028@montanaro.dyndns.org> Message-ID: <8393fff0505221139292ecd28@mail.gmail.com> On 5/22/05, Skip Montanaro wrote: > >>>>> "Ian" == Ian Bicking writes: > It's not like this is a new problem, is it? It seems to me we should be > able to come up with a reasonable set of requirements and a design that > addresses them. I'd like to avoid hacks that require we deal with HTML as > an input format. I agree.... talking about doing "hacks" when we're talking about replacing something like LaTeX does not sound very appropriate to me... LaTeX is a very powerful program. I think we need to enumerate the problems that we're having with it now that makes people want to move away from it (apart from the "it's not an all-python solution" idea). In other words, to answer these two questions clearly: 1. why exactly do we want to change the documentation system? what are the problems with the current approach? can it just be fixed with the current system? what would be the advantages of a new system? (I voiced some answers of this question in my previous inquiries, namely, that there is currently no way to unify part of the docs from the source, i.e. function descriptions should be in the source and pulled into the manual sources automatically, to have one single source for all "bits" of docs. Other messages seem to suggest that LaTeX is too complicated for contributors. I personally disagree, but it may be a valid reason if this is the case. IMHO if someone can't be bothered to learn the 20 macros being used in the LaTeX sources used for the Python docs (which are themselves reasonably well documented and for which ample examples are available) I have to wonder about their capacity to contribute readable prose to the documentation.) 2. what is the precise list of features that are offered in the current documentation system (excluding all the LaTeX features that it is not using) and how can we replace all of them with the new system (docutils or otherwise)? (Again, just my HO (and i'm just an observer more than anything else, really), but I don't like the idea of replacing a source format to a new system if the new system doesn't have at least all the features of the old one. As much as I enjoy working with docutils, it does not seem to offer all that LaTeX and its multiple conversions has to offer now.) cheers, From mike at pcblokes.com Mon May 23 10:16:43 2005 From: mike at pcblokes.com (Michael Foord) Date: Mon, 23 May 2005 09:16:43 +0100 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <87k6ls2kl7.fsf@news2.ososo.de> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <87k6ls2kl7.fsf@news2.ososo.de> Message-ID: <4291916B.5000707@pcblokes.com> Felix Wiemann wrote: >Ian Bicking wrote: > > > >>If the documentation is able to have intelligent interlinking, that'd >>also be great; my experience with ReST is not great here, as it makes >>strong distinctions between same-page links and external links. >> >> > >Yes. It would be nice if Docutils were able to merge several input >documents into one, so that you could use internal links to link between >the parts (input files). > > Surely merging input files into one output file is relatively straightforward ? What would be nicer (but more complex ?) would be generating several output files and autogenerating the cross links between them. Didn't this discussion start with how to automatically include docstrings from source code - or have we moved on ? Yes resolving a system for docutils that breaks the 1 : 1 dependency on input and output files would be nice, then you could generate contents pages and navigation links for multipage documetns very nicely. But I imagine that is going to take sometime... Hacks in the meantime sound liek a good idea... especially as we don't have a clear idea of what we want the system to even *look* like. Trying *something* sounds like the *only* way of getting there.... Best Regards, Fuzzy http://www.voidspace.org.uk/python > > >>Though maybe that could be easily fixed with a two-phase build >>process; ReST identifies the links, outputs HTML, and then I resolve >>all the links in a separate phase. >> >> > >This is more a hack (sorry), because you'd have to mess with HTML code. > >A clean implementation would have to deal with multiple input files and >multiple output files as distinct and independent concepts, i.e. you >shouldn't need to have a 1:1 relation between input and output files. I >don't have enough time (and interest) to develop such multi-file support >at the moment, though. It may become important when we try to convert >the standard docs to reST, though. > > > From fdrake at acm.org Tue May 24 22:32:23 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Tue, 24 May 2005 16:32:23 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <87k6ls2kl7.fsf@news2.ososo.de> References: <428B6860.9020900@colorstudy.com> <87k6ls2kl7.fsf@news2.ososo.de> Message-ID: <200505241632.24134.fdrake@acm.org> On Saturday 21 May 2005 20:03, Felix Wiemann wrote: > Yes. It would be nice if Docutils were able to merge several input > documents into one, so that you could use internal links to link between > the parts (input files). For most purposes, the standard include: directive seems good enough for me, so long as header-levels are all considered relative to context (probably already the case, but I haven't checked). There may be a few cases where there may be benefit from being able to conditionalize content based on whether the document is standalone or not, but these seem unusual. It may be that these could be better handled by having a component that is always subsidiary and using alternate containing documents. > A clean implementation would have to deal with multiple input files and > multiple output files as distinct and independent concepts, i.e. you > shouldn't need to have a 1:1 relation between input and output files. I > don't have enough time (and interest) to develop such multi-file support > at the moment, though. It may become important when we try to convert > the standard docs to reST, though. I've increasingly come to think that it makes sense to load all the documents as a single collection (one really big document), for the purposes of dealing with linking. Splitting the output into separate pieces is a separate issue in many ways, but needs to be done before other transformations. Which leads me to think it's just another transformation, and any transformation that runs after needs to be able to deal with multi-output structures. -Fred -- Fred L. Drake, Jr. From fdrake at acm.org Tue May 24 22:47:38 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Tue, 24 May 2005 16:47:38 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <8393fff0505221139292ecd28@mail.gmail.com> References: <17040.45129.807736.306028@montanaro.dyndns.org> <8393fff0505221139292ecd28@mail.gmail.com> Message-ID: <200505241647.38715.fdrake@acm.org> On Sunday 22 May 2005 14:39, Martin Blais wrote: > 1. why exactly do we want to change the documentation system? what > are the problems with the current approach? can it just be fixed with > the current system? what would be the advantages of a new system? > > (I voiced some answers of this question in my previous inquiries, > namely, that there is currently no way to unify part of the docs from > the source, i.e. function descriptions should be in the source and > pulled into the manual sources automatically, to have one single > source for all "bits" of docs. This would be very nice to have. For the HTML presentation, there's no reason we can't add this and stick to LaTeX. That'll be harder for the typeset presentations (though the value of that for the library reference is seriously questionable). > Other messages seem to suggest that LaTeX is too complicated for > contributors. I personally disagree, but it may be a valid reason if > this is the case. IMHO if someone can't be bothered to learn the 20 I think there's a strong feeling among many people that writing text is not itself programming, and that explicit (== "visible" for now) tagging is undesirable. However, any community which handles large volumns of structured information eventually finds explicit structure is valuable. Thinking of text as less structured than program source is simply wrong; historically the "tagging" is simply not visible. > (Again, just my HO (and i'm just an observer more than anything else, > really), but I don't like the idea of replacing a source format to a > new system if the new system doesn't have at least all the features of > the old one. As much as I enjoy working with docutils, it does not > seem to offer all that LaTeX and its multiple conversions has to offer > now.) As I found when I started the LaTeX-to-XML conversion tool that's currently in the Python CVS, the level of effort to get 100% support is enormous. There's a lot hiding in the use of these tools over the past many years, in spite of their shortcomings. -Fred -- Fred L. Drake, Jr. From Felix.Wiemann at gmx.net Wed May 25 21:38:08 2005 From: Felix.Wiemann at gmx.net (Felix Wiemann) Date: Wed, 25 May 2005 21:38:08 +0200 Subject: [Doc-SIG] Python docs in reST References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <17035.31628.509536.426587@montanaro.dyndns.org> <428B879A.7000700@colorstudy.com> Message-ID: <874qcr85b3.fsf@news2.ososo.de> Ian Bicking wrote: > (I've also copied David Goodger and Felix Wiemann, as they might have > more to say about this and I'm not sure if they are tracking this list) We're both subscribed. (I read this list with lower priority than the Docutils lists, though.) > * I believe ReST markup is a superset of the conventional Python > Latex. Not yet. We need to have some custom code specially designed for Python standard documentation. ReST isn't capable of expressing all things that can be expressed in Python-doc-LaTeX yet. > * ReST is a superset, so there's a bunch of things that I think are > much more easily expressed there than in the Latex. When it comes to comparing reST and LaTeX, I'd be wary of using terms like "superset" because the two languages cover different scopes. > (Note: I have no idea what the actual Latex looks like, I only say > this from familiarity with its output ;) The problem with this is that > it may be hard to mix documents generated from the two sources and > keep a consistent style. As long as you need to mix LaTeX and reST standard-doc sources, making Docutils output Python-doc-LaTeX seems like the most viable approach. > It's readily extensible on the local level by adding new directives > and inline roles. Adding new directives isn't possible at the moment. (Even though that might be a good idea maybe.) > * There's no tools that I know of to generate ReST. I think it would > have to be based on heuristics, with manual editing. Theoretically you could write a full-blown reST-writer, but hacking together a prototype might be not too difficult, if you leave out tables. > I think the warnings are pretty good, though, so I think this is > doable without having to carefully proofread all of the output. Yes. > * I personally find writing ReST pleasant, but I think Michael is > right that it's not the Latex that keeps people from contributing. Dunno. For me, as someone who has both a running LaTeX system and some LaTeX knowledge, it might indeed keep me from contributing, partly because it takes more than just typing "latex" to get the files processed, partly because I don't like LaTeX as an input format. > [...] Well, does anyone write docs on a whim? If not, reST is a great opportunity to change that. :-) For the Docutils project, writing docs is quite easy indeed, because they're all in reST. -- For private mail please ensure that the header contains 'Felix Wiemann'. http://www.ososo.de/ From Felix.Wiemann at gmx.net Wed May 25 21:42:32 2005 From: Felix.Wiemann at gmx.net (Felix Wiemann) Date: Wed, 25 May 2005 21:42:32 +0200 Subject: [Doc-SIG] Python docs in reST? References: <17032.3517.362406.820105@montanaro.dyndns.org> Message-ID: <87vf576qjb.fsf@news2.ososo.de> Skip Montanaro wrote: > [...] That led me to thinking about converting the current docs to > reST. [...] Pointers to past threads cheerfully accepted. The topic was partly covered in the recent thread "two subjects: docutils for python docs, and integrating documentation in source code": > Thoughts? Currently not possible (it will require some code), but +1 on the idea. -- For private mail please ensure that the header contains 'Felix Wiemann'. http://www.ososo.de/ From skip at pobox.com Wed May 25 22:42:42 2005 From: skip at pobox.com (Skip Montanaro) Date: Wed, 25 May 2005 15:42:42 -0500 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <874qcr85b3.fsf@news2.ososo.de> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <17035.31628.509536.426587@montanaro.dyndns.org> <428B879A.7000700@colorstudy.com> <874qcr85b3.fsf@news2.ososo.de> Message-ID: <17044.58178.397257.332103@montanaro.dyndns.org> Felix> As long as you need to mix LaTeX and reST standard-doc sources, Felix> making Docutils output Python-doc-LaTeX seems like the most Felix> viable approach. (I started this thread way back when...) One of my original motivations was to make it easier to experiment with different ways of presenting the generated HTML. I have a suspicion that tweaking the Docutils HTML generator would be easier than tweaking latex2html. For example, I'm aware of a couple wiki wrappers available to provide annotation capability for static pages. Wikalong is a general tool that's unfortunately Firefox-specific. Andrew Kuchling wrote an experimental wrapper that's specific to Python's docs. It seems to me it would be better to simply generate HTML output that integrates the annotation capability with the static display. Skip From Felix.Wiemann at gmx.net Wed May 25 23:08:25 2005 From: Felix.Wiemann at gmx.net (Felix Wiemann) Date: Wed, 25 May 2005 23:08:25 +0200 Subject: [Doc-SIG] Python docs in reST References: <428B6860.9020900@colorstudy.com> <87k6ls2kl7.fsf@news2.ososo.de> <200505241632.24134.fdrake@acm.org> Message-ID: <87r7fv6mk6.fsf@news2.ososo.de> Fred L. Drake, Jr. wrote: > For most purposes, the standard include: directive seems good enough > for me, so long as header-levels are all considered relative to > context I'm not sure what you mean. Basically, the "include" *literally* inserts the contents of the included file into the input stream. > I've increasingly come to think that it makes sense to load all the > documents as a single collection (one really big document), for the > purposes of dealing with linking. Yes. If you want to load not only chapters but the *whole* Python documentation into a document, you'd end up with unacceptable parse times. To fix this, you'd need to parse all input files (without resolving references etc.) and store the results (namely Docutils node trees) in XML files. Later, when creating the big document, the XML files can be read and assembled, which should probably not take very much time. Disadvantages: * Need to use "make" or implement make-like functionality to regenerate XML files only when necessary. * Need to be able to serialize Docutils node trees as XML. There is currently an XML writer, but the original node tree isn't reconstructable from the XML files. So we'd need to implement proper serialization support first (not too difficult, though). > Splitting the output into separate pieces is a separate issue in many > ways, Yes. > but needs to be done before other transformations. What other transformations? -- For private mail please ensure that the header contains 'Felix Wiemann'. http://www.ososo.de/ From Felix.Wiemann at gmx.net Wed May 25 23:12:46 2005 From: Felix.Wiemann at gmx.net (Felix Wiemann) Date: Wed, 25 May 2005 23:12:46 +0200 Subject: [Doc-SIG] Python docs in reST? References: <05May18.144608pdt.58617@synergy1.parc.xerox.com> <87oeb77jpq.fsf@wilson.rwth-aachen.de> Message-ID: <87k6ln6mcx.fsf@news2.ososo.de> Torsten Bronger wrote: > Hall?chen! Hall?li! :) > I use field lists at the moment. That's okay, but it's not real > logical markup since the reST interpreter has no chance to recognise > it as a function definition. Besides, its pdf and HTML results are > poor. The PDF output is being worked on. What's bothering you about the HTML output? -- For private mail please ensure that the header contains 'Felix Wiemann'. http://www.ososo.de/ From ianb at colorstudy.com Wed May 25 23:28:58 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Wed, 25 May 2005 16:28:58 -0500 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <87r7fv6mk6.fsf@news2.ososo.de> References: <428B6860.9020900@colorstudy.com> <87k6ls2kl7.fsf@news2.ososo.de> <200505241632.24134.fdrake@acm.org> <87r7fv6mk6.fsf@news2.ososo.de> Message-ID: <4294EE1A.4090207@colorstudy.com> Felix Wiemann wrote: > If you want to load not only chapters but the *whole* Python > documentation into a document, you'd end up with unacceptable parse > times. To fix this, you'd need to parse all input files (without > resolving references etc.) and store the results (namely Docutils node > trees) in XML files. Later, when creating the big document, the XML > files can be read and assembled, which should probably not take very > much time. Hey, now we're getting back to exactly what I suggested! ;) Really, XML or rendered HTML, it's not a huge difference. So, back to that expedient solution I talked about before, you can fairly easily get docutils to not try to resolve any links at all, outputing some marker that indicates this. Something like this: class DelayedLinkResolver(nodes.SparseNodeVisitor): def visit_reference(self, node): if node.resolved or not node.hasattr('refname'): return node.resolved = 1 class DelayedLink(Transform): default_priority = 0 def apply(self): visitor = DelayedLinkResolver(self.document) self.document.walk(visitor) Then later you look for ..., where refname contains the simplified link name (e.g., `This Link`_ becomes This Link in the HTML). There's a chance this will break with indirect links, like: See foo_ .. _foo: bar_ But I'm sure that's resolvable. And sure, there may be a better more abstract way, but this is Easy Right Now, and all the subtle and hard problems (IMHO) are in how you express and resolve links, and this gets you to that problem more quickly. Or maybe this seems ideal to me because I've never cared about any format but HTML (just to admit my biases ;). -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From Felix.Wiemann at gmx.net Wed May 25 23:57:18 2005 From: Felix.Wiemann at gmx.net (Felix Wiemann) Date: Wed, 25 May 2005 23:57:18 +0200 Subject: [Doc-SIG] Python docs in reST References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <87k6ls2kl7.fsf@news2.ososo.de> <42902A51.90703@colorstudy.com> <17040.45129.807736.306028@montanaro.dyndns.org> <8393fff0505221139292ecd28@mail.gmail.com> Message-ID: <87fywb6kap.fsf@news2.ososo.de> Martin Blais wrote: > 1. why exactly do we want to change the documentation system? Ideas: * We want to have **well-readable** documentation source. * We want to get **flexibility** in that we can modify the parsed document using Python code. In particular I have Docutils' document tree in mind -- use rst2pseudoxml.py to see what it looks like. You can easily apply transformations on these trees, using your favorite programming language ;-). This is not (easily?) possible with LaTeX. * We want an **easy-to-use** documentation system. Any documentation should be renderable by typing "sometool.py input.txt output.html". Getting multi-document output should be similarly easy. -- For private mail please ensure that the header contains 'Felix Wiemann'. http://www.ososo.de/ From Felix.Wiemann at gmx.net Thu May 26 00:14:41 2005 From: Felix.Wiemann at gmx.net (Felix Wiemann) Date: Thu, 26 May 2005 00:14:41 +0200 Subject: [Doc-SIG] Python docs in reST References: <428B6860.9020900@colorstudy.com> <87k6ls2kl7.fsf@news2.ososo.de> <200505241632.24134.fdrake@acm.org> <87r7fv6mk6.fsf@news2.ososo.de> <4294EE1A.4090207@colorstudy.com> Message-ID: <8764x76jhq.fsf@news2.ososo.de> Ian Bicking wrote: > Felix Wiemann wrote: > >> [Big document from many input files:] you'd need to parse all input >> files (without resolving references etc.) and store the results >> (namely Docutils node trees) in XML files. > > Hey, now we're getting back to exactly what I suggested! ;) Really, XML > or rendered HTML, it's not a huge difference. Oh, it is. :-) Given a proper serialization algorithm, you could cleanly deserialize an XML file into the original node tree, which is not possible with HTML. > So, back to that expedient solution I talked about before, you can > fairly easily get docutils to not try to resolve any links at all, > outputing some marker that indicates this. > > [...] Then later you look for ..., where refname > contains the simplified link name (e.g., `This Link`_ becomes refname="this-link">This Link in the HTML). There's a chance this > will break with indirect links [...] But I'm sure that's resolvable. I fear you'd end up re-implementing Docutils' reference-resolving system. I had to work with it once before, and trust me, it's quite a complicated beast. I wouldn't say a hack which does what you described would be a bad thing though. -- For private mail please ensure that the header contains 'Felix Wiemann'. http://www.ososo.de/ From fdrake at acm.org Thu May 26 01:29:46 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Wed, 25 May 2005 19:29:46 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <87fywb6kap.fsf@news2.ososo.de> References: <8393fff0505221139292ecd28@mail.gmail.com> <87fywb6kap.fsf@news2.ososo.de> Message-ID: <200505251929.46721.fdrake@acm.org> On Wednesday 25 May 2005 17:57, Felix Wiemann wrote: > * We want to have **well-readable** documentation source. We have that now. Not ReST, but its very readable. > * We want to get **flexibility** in that we can modify the parsed > document using Python code. Is this a goal, or is this a means to an end? This sounds like the later to me. > In particular I have Docutils' document tree in mind -- use > rst2pseudoxml.py to see what it looks like. You can easily apply > transformations on these trees, using your favorite programming > language ;-). This is not (easily?) possible with LaTeX. Not easily, but mostly because there are things the markup syntax doesn't encode directly. The handling of paragraphs also doesn't help. > * We want an **easy-to-use** documentation system. Any documentation > should be renderable by typing "sometool.py input.txt output.html". > Getting multi-document output should be similarly easy. Again, I think we have this now. "mkhowto input.tex" is pretty straightforward. -Fred -- Fred L. Drake, Jr. From bronger at physik.rwth-aachen.de Thu May 26 01:53:20 2005 From: bronger at physik.rwth-aachen.de (Torsten Bronger) Date: Thu, 26 May 2005 01:53:20 +0200 Subject: [Doc-SIG] Python docs in reST? References: <05May18.144608pdt.58617@synergy1.parc.xerox.com> <87oeb77jpq.fsf@wilson.rwth-aachen.de> <87k6ln6mcx.fsf@news2.ososo.de> Message-ID: <87fywayia7.fsf@wilson.rwth-aachen.de> Hall?chen! Felix Wiemann writes: > Torsten Bronger wrote: > >> I use field lists at the moment. That's okay, but it's not real >> logical markup since the reST interpreter has no chance to >> recognise it as a function definition. Besides, its pdf and HTML >> results are poor. > > The PDF output is being worked on. What's bothering you about the > HTML output? I dislike this table-like style. Currently, my source may contain this: unlock ...... Relinquish a lock for the specified resource. :Call: unlock(vi) :Parameters: `vi` : ViSession Unique logical identifier to a session. :Return values: None. I want "Call" and "Parameters" to go away, and I don't want to feel forced to start a new section for every function. But first and foremost, I want to have the impression that I tell the reST interpreter everything I can. Here, for example, I know that "unlock" is the function's name, "vi" is a parameter object, "ViSession" is a type. However, I can't pass this knowledge to reST. http://pp3.sf.net/manual/Other-layout-parameters.html shows how it could look like: Just a signature line with proper emphasis on the functions's name and all identifiers in italics. Big skip before it, and indentation of the whole explanation. Tsch?, Torsten. -- Torsten Bronger, aquisgrana, europa vetus From fdrake at acm.org Thu May 26 01:55:47 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Wed, 25 May 2005 19:55:47 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <87r7fv6mk6.fsf@news2.ososo.de> References: <200505241632.24134.fdrake@acm.org> <87r7fv6mk6.fsf@news2.ososo.de> Message-ID: <200505251955.47328.fdrake@acm.org> On Wednesday 25 May 2005 17:08, Felix Wiemann wrote: > I'm not sure what you mean. Basically, the "include" *literally* > inserts the contents of the included file into the input stream. I just mean that this should be a node insertion; header identification should be completely independent of what underline characters are used in the context document. This may already be the case; I've simply not checked that. > If you want to load not only chapters but the *whole* Python > documentation into a document, you'd end up with unacceptable parse The Library Reference is currently parsed all at once; that's the bulk of the documentation by far. The time to generate HTML for that is already long. If a ReST version of the document could be processed in approximately the same length of time, that's good enough. A better system would be welcome, and likely not too hard. > times. To fix this, you'd need to parse all input files (without > resolving references etc.) and store the results (namely Docutils node > trees) in XML files. Later, when creating the big document, the XML Maybe XML, maybe not. Depends on how open the system should be internally. I'd certainly like it to be open, and keeping it open to extensibility is not a bad thing in my book. > files can be read and assembled, which should probably not take very > much time. Disadvantages: > > * Need to use "make" or implement make-like functionality to regenerate > XML files only when necessary. If you're trying to avoid processing effort, you need some way to determine what can't be avoided. If we have a system that "cooks" each input document into a node tree and references to included documents, it shouldn't be hard to knit things together. This only matters if the actual parse is slow; anything else will be relatively quick. Loading structure takes time too, regardless of whether the structure is encoded in XML or as a Python pickle. If there's a significant parse time (and I think there is), we need to work on the performance of the parser. (Has anyone specifically looked at this yet? If so, it would be interesting to hear of any specific issues that were found and still need attention.) > > but needs to be done before other transformations. > > What other transformations? Things like generating page-local Tables of Contents come after, similar to things like generating the final form of the links. -Fred -- Fred L. Drake, Jr. From mwh at python.net Thu May 26 10:26:40 2005 From: mwh at python.net (Michael Hudson) Date: Thu, 26 May 2005 09:26:40 +0100 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <874qcr85b3.fsf@news2.ososo.de> (Felix Wiemann's message of "Wed, 25 May 2005 21:38:08 +0200") References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <17035.31628.509536.426587@montanaro.dyndns.org> <428B879A.7000700@colorstudy.com> <874qcr85b3.fsf@news2.ososo.de> Message-ID: <2m8y22z933.fsf@starship.python.net> Felix Wiemann writes: > Dunno. For me, as someone who has both a running LaTeX system and some > LaTeX knowledge, it might indeed keep me from contributing, partly > because it takes more than just typing "latex" to get the files > processed, partly because I don't like LaTeX as an input format. Arrgh, this is a non-argument. "mkhowto mydoc.tex" is well under the usage difficulty bar. Cheers, mwh -- For their next act, they'll no doubt be buying a firewall running under NT, which makes about as much sense as building a prison out of meringue. -- -:Tanuki:- -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html From mike at pcblokes.com Thu May 26 10:05:32 2005 From: mike at pcblokes.com (Michael Foord) Date: Thu, 26 May 2005 09:05:32 +0100 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <87fywb6kap.fsf@news2.ososo.de> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <87k6ls2kl7.fsf@news2.ososo.de> <42902A51.90703@colorstudy.com> <17040.45129.807736.306028@montanaro.dyndns.org> <8393fff0505221139292ecd28@mail.gmail.com> <87fywb6kap.fsf@news2.ososo.de> Message-ID: <4295834C.9010707@pcblokes.com> Felix Wiemann wrote: >Martin Blais wrote: > > > >>1. why exactly do we want to change the documentation system? >> >> > >Ideas: > >* We want to have **well-readable** documentation source. > >* We want to get **flexibility** in that we can modify the parsed > document using Python code. > > In particular I have Docutils' document tree in mind -- use > rst2pseudoxml.py to see what it looks like. You can easily apply > transformations on these trees, using your favorite programming > language ;-). This is not (easily?) possible with LaTeX. > >* We want an **easy-to-use** documentation system. Any documentation > should be renderable by typing "sometool.py input.txt output.html". > Getting multi-document output should be similarly easy. > > > As a side issue - it would be nice for developers of other modules/projects to be able to *easily* generate documentation that is consistent with the (IMO) nice looking Python documentation. I confess to *not* having looked into Latex markup [#]_ - so I haven't a clue how difficult it is to use, but I love reST. If Latex is *that* easy to use why bother creating reST ;-) In order to achieve this (becoming a useful documentation system for Python projects) - docutils needs to be capable of handling anything that is in the Python docs. So it is certainly in the interests of the docutils project to address these issues... whether that means it should be adopted as *the* method of documenting Python is another matter. Hopefully natural selection will start to work... Surely implementing a Python source reader (that works by introspection or whatever) that extracts docstrings and inserts it *into* a reST document (still a two pass process) would help as a short term measure. Hmmm..... I might even implement something like that myself. Best Regards, Fuzzy http://www.voidspace.org.uk/python .. [#] I use HTML and reST... I honestly can't be bothered with another one... sorry... From martin.blais at gmail.com Thu May 26 18:10:12 2005 From: martin.blais at gmail.com (Martin Blais) Date: Thu, 26 May 2005 12:10:12 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <4295834C.9010707@pcblokes.com> References: <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <87k6ls2kl7.fsf@news2.ososo.de> <42902A51.90703@colorstudy.com> <17040.45129.807736.306028@montanaro.dyndns.org> <8393fff0505221139292ecd28@mail.gmail.com> <87fywb6kap.fsf@news2.ososo.de> <4295834C.9010707@pcblokes.com> Message-ID: <8393fff050526091067e0e00f@mail.gmail.com> On 5/26/05, Michael Foord wrote: > Felix Wiemann wrote: > > >Martin Blais wrote: > > > > > > > >>1. why exactly do we want to change the documentation system? > >> > >> > > > >Ideas: > > > >* We want to have **well-readable** documentation source. hmmm... the current latex markup is as simple as it can be. it's got a word, and some form of delimiters, e.g. \function{foo}. i don't feel that `foo`:function: is all that much better. the sectioning uses markup (rest doesn't). apart from that, there aren't many other constructs that are difficult to read, are there? > >* We want to get **flexibility** in that we can modify the parsed > > document using Python code. > > > > In particular I have Docutils' document tree in mind -- use > > rst2pseudoxml.py to see what it looks like. You can easily apply > > transformations on these trees, using your favorite programming > > language ;-). This is not (easily?) possible with LaTeX. yes, that is a good argument. but... rest allows you to view the input files directly, as output (ascii output = input). this assumes that the markup is kept to its *bare minimum*. this is an *explicit* reason for creating rest in the first place. the more we are talking about adding markup on rest, the stranger this whole discussion is becoming. for example, if you're trying to plug too much semantics on top of rest, and you add lots of ugly markup, then you're kind-of breaking what rest is all about (and to a certain extent this has already been happening with some of the directives to docutils). with rest, the source has to more or less "look like" the output. that's the whole point of rest. this is not a requirement for latex sources. i see a contradiction: docutils is a toolchain with a parser that extracts semantic (paragraphs, sections, etc.) from source documents that have no markup (or very little). it has been explicitly discussed NOT to add too much visible markup on the docutils list, and AFAIK it is an explicit goal of rest to keep the input syntax as unobstrusive as possible. and then, having too little markup does not allow us to do all the things that we need to do for a proper documentation system. we need to mark functions, classes, bits of code, variables, argument, optional arguments, etc. to generate high quality docs with proper indexes and good-looking output, you *really* need those things. there is no compromise about that, we need to be able to tell the doc system what is what, and that requires some form of [ugly] markup. therefore, there is the possibility that rest could never be used to generate the kind of document that we need to generate, due to self-imposed limitations. if you add lots of ugly markup to rest, you're creating something else, it's not rest anymore. bluntly stated, the goal of rest is NOT to have markup, and we need LOTS of markup. reconcile that. maybe we have to recognize that rest has its limits, and rest recognizes the limits of its own domain, there are some places it doesn't want to go. it is fine for certain types of documents, but not for others. IMHO the worst thing that could happen is that one day I open a text file supposedly writtne in rest, and that i can't understand it because i have to learn what all that new markup is all about. my point is: attempting to make rest jump through hoops to make it do all the stuff that latex does is likely to break the beauty of rest. that would suck. > >* We want an **easy-to-use** documentation system. Any documentation > > should be renderable by typing "sometool.py input.txt output.html". > > Getting multi-document output should be similarly easy. the more i hear this, the more i see great "value" in splitting the docs from the python source. it's 95% of the way there already with mkhowto. > As a side issue - it would be nice for developers of other > modules/projects to be able to *easily* generate documentation that is > consistent with the (IMO) nice looking Python documentation. > > I confess to *not* having looked into Latex markup [#]_ - so I haven't > a clue how difficult it is to use, but I love reST. If Latex is *that* > easy to use why bother creating reST ;-) (it really isn't all that much harder than rest.) > In order to achieve this (becoming a useful documentation system for > Python projects) - docutils needs to be capable of handling anything > that is in the Python docs. So it is certainly in the interests of the > docutils project to address these issues... whether that means it should i disagree... personally, i wish that docutils remains as simple as possible (that is the raison d'etre of it) if it means it has limitations, then fine by me. i really don't see why docutils "needs" to be capable of handling all that is in the python docs. docutils needs to do what it does best: extract document structure from files with no markup (or very little). AFAIK it was never meant as a complete replacement for a document preparation system. it was created from the recognition by many different people at the same time that one could extract structure from simple text files. > be adopted as *the* method of documenting Python is another matter. > Hopefully natural selection will start to work... > > Surely implementing a Python source reader (that works by introspection > or whatever) that extracts docstrings and inserts it *into* a reST > document (still a two pass process) would help as a short term measure. > Hmmm..... I might even implement something like that myself. while you're at it, can you implement output in the latex style as well? i'm getting the feeling that this would help as a long term measure. to summarize all these discussions, there have been 3 separate threads: 1. splitting the docs system into a separate package (make it easier to reuse); 2. combining information from the source code with information in the document sources; 3. converting sources to another system (some people think latex is too difficult); cheers, From ianb at colorstudy.com Thu May 26 18:57:05 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Thu, 26 May 2005 11:57:05 -0500 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <4295834C.9010707@pcblokes.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <87k6ls2kl7.fsf@news2.ososo.de> <42902A51.90703@colorstudy.com> <17040.45129.807736.306028@montanaro.dyndns.org> <8393fff0505221139292ecd28@mail.gmail.com> <87fywb6kap.fsf@news2.ososo.de> <4295834C.9010707@pcblokes.com> Message-ID: <4295FFE1.3080102@colorstudy.com> (note that none of this message relates to the Python reference documentation...) Michael Foord wrote: > As a side issue - it would be nice for developers of other > modules/projects to be able to *easily* generate documentation that is > consistent with the (IMO) nice looking Python documentation. > > I confess to *not* having looked into Latex markup [#]_ - so I haven't > a clue how difficult it is to use, but I love reST. If Latex is *that* > easy to use why bother creating reST ;-) > > In order to achieve this (becoming a useful documentation system for > Python projects) - docutils needs to be capable of handling anything > that is in the Python docs. So it is certainly in the interests of the > docutils project to address these issues... whether that means it should > be adopted as *the* method of documenting Python is another matter. > Hopefully natural selection will start to work... > > Surely implementing a Python source reader (that works by introspection > or whatever) that extracts docstrings and inserts it *into* a reST > document (still a two pass process) would help as a short term measure. > Hmmm..... I might even implement something like that myself. Incidentally, I just recently wrote a tool to do this... well, right now it creates a single document from extracted source, but it's very young, and it should be able to create fragments as well, for inclusion in other documents. The development model, as I have time to put into it, will be primarily about producing the desired output, without any particular committment to an underlying architecture (I've already written two other systems which I've thrown away, so I'm trying not to focus too much on implementation). It's intended to extract documentation from source that is written specifically with this tool in mind, and only documents things that are explicitly specified (using __all__ and magic attributes now, probably more annotations in the future). It allows things like documenting an interface from another module, without including that module itself in the documentation anywhere. The code: http://svn.pythonpaste.org/Paste/trunk/paste/docsupport/ What it currently produces: http://pythonpaste.org/docs/reference.html Part of the importance to Paste is that the objects to be documented aren't generally directly referenced; they are put into a stack of WSGI middleware implicitly, are return values from functions, etc. Even now the module organization of the document isn't helpful, it should be organized based on functionality. Indexing is also high up on my list for it, including custom indexes (for Paste that includes domain-specific things like WSGI keys added, configuration keys used, middleware and servers available, etc). -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From fdrake at acm.org Thu May 26 22:34:18 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Thu, 26 May 2005 16:34:18 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <8393fff050526091067e0e00f@mail.gmail.com> References: <4295834C.9010707@pcblokes.com> <8393fff050526091067e0e00f@mail.gmail.com> Message-ID: <200505261634.18730.fdrake@acm.org> On Thursday 26 May 2005 12:10, Martin Blais wrote: > the sectioning > uses markup (rest doesn't). apart from that, there aren't many other > constructs that are difficult to read, are there? Last I checked, ReST needed markup for sectioning as well, but had the advantage that section level was implied by the concrete markup, not by naming. This makes more sense in light of re-usable content components that may appear at different section levels in different documents. (Yes, I do consider heading underlines to be markup, no matter how reasonable!) > self-imposed limitations. if you add lots of ugly markup to rest, > you're creating something else, it's not rest anymore. bluntly > stated, the goal of rest is NOT to have markup, and we need LOTS of > markup. reconcile that. This is another reason I'm not convinced we need to change markup systems. > the more i hear this, the more i see great "value" in splitting the > docs from the python source. it's 95% of the way there already with > mkhowto. There are a couple of aspects to this: 1. Splitting the documentation tools from the Python sources. This is something that is specifically desirable now. There are a number of other projects that use the LaTeX markup from Python; these are mostly related to Python in some way. (These also create a reason to continue to maintain the LaTeX markup.) > i disagree... personally, i wish that docutils remains as simple as > possible (that is the raison d'etre of it) if it means it has > limitations, then fine by me. i really don't see why docutils > "needs" to be capable of handling all that is in the python docs. It certainly doesn't need to support it all out of the box, especially the constructs specific to documenting Python (or other OO programming languages). It would be nice to have the option of using it for documenting at least Python packages of small to moderate size, with support from an add-on library of text roles and directives. > docutils needs to do what it does best: extract document structure > from files with no markup (or very little). AFAIK it was never meant > as a complete replacement for a document preparation system. it was > created from the recognition by many different people at the same time > that one could extract structure from simple text files. Having a structure that supports the Python documentation is not a trivial exercise, and I'm not sold that it needs to be done in ReST. I think there's more value in being explicit than in being minimal, so it's questionable that the goal of converting the "standard" Python documentation to ReST makes sense. > > Surely implementing a Python source reader (that works by introspection > > or whatever) that extracts docstrings and inserts it *into* a reST > > document (still a two pass process) would help as a short term measure. It shouldn't be too hard to implement a specialized URL scheme that converts an object reference to documentation text for some particular system. -Fred -- Fred L. Drake, Jr. From skip at pobox.com Thu May 26 23:10:37 2005 From: skip at pobox.com (Skip Montanaro) Date: Thu, 26 May 2005 16:10:37 -0500 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <200505261634.18730.fdrake@acm.org> References: <4295834C.9010707@pcblokes.com> <8393fff050526091067e0e00f@mail.gmail.com> <200505261634.18730.fdrake@acm.org> Message-ID: <17046.15181.614156.808968@montanaro.dyndns.org> Fred> This is another reason I'm not convinced we need to change markup Fred> systems. I'm not convinced we actually need to change markup systems either. My suggestion that perhaps migrating to reST was motivated by: * my dread (real or imagined) that trying to coax different output from latex2html would be a big PITA * a perhaps invalid assumption that reST-as-input would generate more documentation contributions based upon the occasional comment I've seen over the years that (paraphrasing), "I can't contribute - I don't know how to use LaTeX." I'm personally comfortable with LaTeX. Fred has yet to complain about my markup errors, so if we can find other ways to address the above problems/perceptions, I'd actually be happy to keep LaTeX. Skip From fdrake at acm.org Thu May 26 23:45:50 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Thu, 26 May 2005 17:45:50 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <17046.15181.614156.808968@montanaro.dyndns.org> References: <200505261634.18730.fdrake@acm.org> <17046.15181.614156.808968@montanaro.dyndns.org> Message-ID: <200505261745.51144.fdrake@acm.org> On Thursday 26 May 2005 17:10, Skip Montanaro wrote: > I'm not convinced we actually need to change markup systems either. My > suggestion that perhaps migrating to reST was motivated by: > > * my dread (real or imagined) that trying to coax different output > from latex2html would be a big PITA LaTeX2HTML can be a pain to extend, but it's doable, and we generally don't need to do much that isn't Python-specific. When we do, it's generally because we want to make it easier to affect the presentation in a systematic and cross-browser way (rather than rely on advanced CSS implementations). > * a perhaps invalid assumption that reST-as-input would generate more > documentation contributions based upon the occasional comment I've > seen over the years that (paraphrasing), "I can't contribute - I > don't know how to use LaTeX." That comes from people looking for excuses; most people who ask for ways to contribute don't follow up, and I'm very careful to point out that the use of LaTeX is not required. > I'm personally comfortable with LaTeX. Fred has yet to complain about my > markup errors, so if we can find other ways to address the above > problems/perceptions, I'd actually be happy to keep LaTeX. I think most of the actually limitations of LaTeX aren't that important. If we could make a few changes, it would be nice, but there isn't anything pressing that needs to be changed. There are enough things that we simply don't yet deal with. -Fred -- Fred L. Drake, Jr. From bronger at physik.rwth-aachen.de Fri May 27 00:09:47 2005 From: bronger at physik.rwth-aachen.de (Torsten Bronger) Date: Fri, 27 May 2005 00:09:47 +0200 Subject: [Doc-SIG] Python docs in reST References: <4295834C.9010707@pcblokes.com> <8393fff050526091067e0e00f@mail.gmail.com> <200505261634.18730.fdrake@acm.org> <17046.15181.614156.808968@montanaro.dyndns.org> Message-ID: <87oeaxy6z8.fsf@wilson.rwth-aachen.de> Hall?chen! Skip Montanaro writes: > [...] > > I'm not convinced we actually need to change markup systems either. My > suggestion that perhaps migrating to reST was motivated by: > > * my dread (real or imagined) that trying to coax different > output from latex2html would be a big PITA It is possible to tweak tex4ht for this purpose. It would generate some sort of XML which you could use as the starting point, for example. tex4ht+XSLT is very slow, but reST isn't a sprinter either. Unfortunately, tex4ht is not easy to setup. By the way, is there an agreement on a clearly defined subset of the LaTeX language or Python documentation? If you allow everything that TeX/LaTeX can process (even if you recommend certain ways more than others) you end up with LaTeX files that only TeX/LaTeX can process. > * a perhaps invalid assumption that reST-as-input would > generate more documentation contributions based upon the > occasional comment I've seen over the years that > (paraphrasing), "I can't contribute - I don't know how to > use LaTeX." I think that this will happen for reST, too. Besides, many people know LaTeX already, but nobody will know reST. However, I don't think that this is a fatal point: Complex structure (and computer documentation is amongst the most complex documents at all) always requires complex markup, in one way or the other. (Too much implicit magic must be learned, too.) > I'm personally comfortable with LaTeX. Fred has yet to complain > about my markup errors, so if we can find other ways to address > the above problems/perceptions, I'd actually be happy to keep > LaTeX. I know LaTeX very well, and I can say that I like it very much. However, it's difficult to guarantee translations to other formats. Concerning "there's more than one way to do it", LaTeX is much worse than Perl. Even if you keep the core documentation consistent, it would be better to have a rock-solid standard doc system in Python. Let alone the problems when trying to add a feature to your documentation system. I have thorough experiences with XML (http://tbook.sf.net), and I think that it's the best way to archive and to process documentation. Since you can't (well, don't want to) input it directly, something like reST with an XML backend -- among others -- is the way to go, in my opinion. Even if for really complex projects the reST source becomes somewhat cluttered, too, it can be an efficient replacement for LaTeX docs. It contains so many clever tricks to keep the "plain text" appearance that I'm optimistic that the rest can be added rather nicely, too. Tsch?, Torsten. -- Torsten Bronger, aquisgrana, europa vetus From fdrake at acm.org Fri May 27 01:04:28 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Thu, 26 May 2005 19:04:28 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <87oeaxy6z8.fsf@wilson.rwth-aachen.de> References: <17046.15181.614156.808968@montanaro.dyndns.org> <87oeaxy6z8.fsf@wilson.rwth-aachen.de> Message-ID: <200505261904.28713.fdrake@acm.org> On Thursday 26 May 2005 18:09, Torsten Bronger wrote: > It is possible to tweak tex4ht for this purpose. It would generate > some sort of XML which you could use as the starting point, for > example. tex4ht+XSLT is very slow, but reST isn't a sprinter > either. Unfortunately, tex4ht is not easy to setup. The current software is generally quite easy to set up on modern linux distributions, where everything is provided in standard packages except the Python-specific portions. We use different tools for different output formats. > By the way, is there an agreement on a clearly defined subset of the > LaTeX language or Python documentation? If you allow everything > that TeX/LaTeX can process (even if you recommend certain ways more > than others) you end up with LaTeX files that only TeX/LaTeX can > process. There aren't really any restrictions; what recommendations there are can be found in "Documenting Python": http://docs.python.org/doc/doc.html If there are any particular constructs you consider a problem, we should discuss the issue and determine what the right thing to do is. Since we haven't had a lot of specific complaints, there aren't any constructs we've declared unsupported. For things in the standard documentation, I've tried to ensure consistency, but that can be done by fiat simply by changing the sources in CVS. > I know LaTeX very well, and I can say that I like it very much. > However, it's difficult to guarantee translations to other formats. > Concerning "there's more than one way to do it", LaTeX is much worse > than Perl. Agreed. As described above, we very much take Python's consenting-adult approach to markup. As (whatever-we-use) gets more widely accepted, it makes sense to define the constraints more carefully. > I have thorough experiences with XML (http://tbook.sf.net), and I That site appears to be down at the moment. > think that it's the best way to archive and to process > documentation. Since you can't (well, don't want to) input it > directly, something like reST with an XML backend -- among others -- > is the way to go, in my opinion. It's not clear to me that you really want an extra front-end to hide what's happening. Writing in ReST only makes sense if ReST sufficiently reflects the data model. If it does, there's no need for an XML step (though software can use one if convenient as an implementation detail); if it doesn't (and that seems to be the case, if we don't want to introduce the extra markup in ReST), then it's not clear that we gain from it. > Even if for really complex projects the reST source becomes somewhat > cluttered, too, it can be an efficient replacement for LaTeX docs. > It contains so many clever tricks to keep the "plain text" > appearance that I'm optimistic that the rest can be added rather > nicely, too. I'm skeptical. If the actual data model is hidden, what I suspect will happen is that more documentation will be written with the presentational layer in mind and the logical markup skipped ("it's supposed to be italic, right?"); this reduces the value of the documentation and the potential for re-use is reduced. Going back to add logical markup after the fact is very tedious and likely to miss things that would not otherwise be missed. -Fred -- Fred L. Drake, Jr. From Felix.Wiemann at gmx.net Fri May 27 01:19:02 2005 From: Felix.Wiemann at gmx.net (Felix Wiemann) Date: Fri, 27 May 2005 01:19:02 +0200 Subject: [Doc-SIG] [ANN] Docutils 0.3.9 released Message-ID: <87vf55zic9.fsf@news2.ososo.de> Docutils 0.3.9 Released ======================= Docutils 0.3.9 has been released. You can download it from . The Docutils homepage is located at . Major Changes Since Docutils 0.3.7 ================================== * Added "file_insertion_enabled" and "raw_enabled" settings. See . * Added auto-enumerated lists. See . * Added "header" and "footer" directives. See . * Added "list-table" directive. See . * Added support for section subtitles. See . * Added "field_name_limit" and "option_limit" settings to HTML writer. See . * Added "cloak_email_addresses" setting to HTML writer. See . * UTF-8 BOMs are now removed from the input stream. * Many minor improvements and bug fixes. See . -- Felix Wiemann -- http://www.ososo.de/ From janssen at parc.xerox.com Fri May 27 06:55:10 2005 From: janssen at parc.xerox.com (Bill Janssen) Date: Thu, 26 May 2005 21:55:10 PDT Subject: [Doc-SIG] Python docs in reST In-Reply-To: Your message of "Thu, 26 May 2005 09:10:12 PDT." <8393fff050526091067e0e00f@mail.gmail.com> Message-ID: <05May26.215519pdt."58617"@synergy1.parc.xerox.com> Martin Blais writes: > i see a contradiction: docutils is a toolchain with a parser that > extracts semantic (paragraphs, sections, etc.) from source documents > that have no markup (or very little). it has been explicitly > discussed NOT to add too much visible markup on the docutils list, and > AFAIK it is an explicit goal of rest to keep the input syntax as > unobstrusive as possible. > > and then, having too little markup does not allow us to do all the > things that we need to do for a proper documentation system. This is the contradiction I was trying to point out last week. After using reST to document a system, I think I'd pick something else next time. I like the general "markup on a different line" style of troff right now. Though it would need a new macro package to really use, so LaTeX is probably just as good, given the right macro package. Bill From janssen at parc.xerox.com Fri May 27 06:58:30 2005 From: janssen at parc.xerox.com (Bill Janssen) Date: Thu, 26 May 2005 21:58:30 PDT Subject: [Doc-SIG] Python docs in reST In-Reply-To: Your message of "Thu, 26 May 2005 14:45:50 PDT." <200505261745.51144.fdrake@acm.org> Message-ID: <05May26.215834pdt."58617"@synergy1.parc.xerox.com> > LaTeX2HTML can be a pain to extend, but it's doable, and we generally don't > need to do much that isn't Python-specific. Another possibility would be to convert the PDF output from pdftex to HTML, by extending some parser like that in xpdf. Bill From bronger at physik.rwth-aachen.de Fri May 27 08:03:36 2005 From: bronger at physik.rwth-aachen.de (Torsten Bronger) Date: Fri, 27 May 2005 08:03:36 +0200 Subject: [Doc-SIG] Python docs in reST References: <17046.15181.614156.808968@montanaro.dyndns.org> <87oeaxy6z8.fsf@wilson.rwth-aachen.de> <200505261904.28713.fdrake@acm.org> Message-ID: <87br6xxl1j.fsf@wilson.rwth-aachen.de> Hall?chen! "Fred L. Drake, Jr." writes: > On Thursday 26 May 2005 18:09, Torsten Bronger wrote: > > [...] > >> I have thorough experiences with XML (http://tbook.sf.net), and I > > That site appears to be down at the moment. Well, I misspelt it: tbookdtd.sf.net. At the moment it's only XML, so I'm looking for a human-friendly input format. I've evaluated many Wiki formats, but reST is the most promising so far. >> think that it's the best way to archive and to process >> documentation. Since you can't (well, don't want to) input it >> directly, something like reST with an XML backend -- among others >> -- is the way to go, in my opinion. > > It's not clear to me that you really want an extra front-end to > hide what's happening. Writing in ReST only makes sense if ReST > sufficiently reflects the data model. In the current situation I don't see a reliable way to get to XML. Even if you don't use XML in the typical workflow, its possibility alone assures flexibility and reusability. It's just so drop-dead simple to do all this with XML. One month ago I completed a texi2latex converter in XSLT, which covers more spec features than the original processor(!) with only 120kB of source code (and XSLT is verbouse). For our students newspaper, I use wiki --> XHTML --> LaTeX with great ease. Our authors come from all faculties and are mostly Word users, but with this system they submit ready-to-use articles. (Wiki --> XHTML is done with Python, of course. ;) Theoretically, you *can* achieve this also with an appropriately constrained LaTeX, therefore my question about a well-defined subset. LaTeX becomes really nasty not until you use more than ASCII, which is rare in computer documentation. However, for this way there must be a real "Python doc" LaTeX with an explicit specification. > [...] > >> Even if for really complex projects the reST source becomes >> somewhat cluttered, too, it can be an efficient replacement for >> LaTeX docs. It contains so many clever tricks to keep the "plain >> text" appearance that I'm optimistic that the rest can be added >> rather nicely, too. > > I'm skeptical. If the actual data model is hidden, what I suspect > will happen is that more documentation will be written with the > presentational layer in mind and the logical markup skipped ("it's > supposed to be italic, right?"); This definitely is a problem with the current reST, however, I think it's solvable. There is a finite number of typical text patterns in computer docs, and the reST standard must give you the canonical way to write them. At the moment, the format is a little bit wishy-washy, and the specification even more so. For some things there doesn't seem to be a data model yet. Maybe reST should be designed not so much as something with an XML backend, but as a frontend to an XML format one has agreed upon in the first place. You're right agree that a documentation format is only useful if it delivers logical markup, and this would be a step towards it. I admit that a certain amount of implicit behaviour will always be necessary (aka magic). However, I find this desirable, because often you have to make things clear to LaTeX that the doc processor could and should figure out itself. This may be a matter of taste, though. Tsch?, Torsten. -- Torsten Bronger, aquisgrana, europa vetus From martin.blais at gmail.com Fri May 27 14:30:43 2005 From: martin.blais at gmail.com (Martin Blais) Date: Fri, 27 May 2005 08:30:43 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <87br6xxl1j.fsf@wilson.rwth-aachen.de> References: <17046.15181.614156.808968@montanaro.dyndns.org> <87oeaxy6z8.fsf@wilson.rwth-aachen.de> <200505261904.28713.fdrake@acm.org> <87br6xxl1j.fsf@wilson.rwth-aachen.de> Message-ID: <8393fff0505270530413e5b5e@mail.gmail.com> On 5/27/05, Torsten Bronger wrote: > > That site appears to be down at the moment. > > Well, I misspelt it: tbookdtd.sf.net. At the moment it's only XML, > so I'm looking for a human-friendly input format. I've evaluated > many Wiki formats, but reST is the most promising so far. about tbook: interesting idea, but with XML as input, a severe limitation is that you can't extend and create macros for your specific document. that is a really powerful part of LaTeX. do you allow people to somehow provide files to do create new element types? on a different note: something that would have great value (and tremendous hacker brownie points) is for someone to tap deep into LaTeX and to provide its internal representation in Python or XML to do its output... there is pdflatex, (ps)latex, wondering if someone could produce xmllatex? (like, write a latex-to-xml translator) From bronger at physik.rwth-aachen.de Fri May 27 14:51:08 2005 From: bronger at physik.rwth-aachen.de (Torsten Bronger) Date: Fri, 27 May 2005 14:51:08 +0200 Subject: [Doc-SIG] Python docs in reST References: <17046.15181.614156.808968@montanaro.dyndns.org> <87oeaxy6z8.fsf@wilson.rwth-aachen.de> <200505261904.28713.fdrake@acm.org> <87br6xxl1j.fsf@wilson.rwth-aachen.de> <8393fff0505270530413e5b5e@mail.gmail.com> Message-ID: <87hdgox26b.fsf@wilson.rwth-aachen.de> Hall?chen! Martin Blais writes: > [...] > > about tbook: interesting idea, but with XML as input, a severe > limitation is that you can't extend and create macros for your > specific document. that is a really powerful part of LaTeX. do > you allow people to somehow provide files to do create new element > types? Yes, this is done via "customisation layers" in XSLT. If you know XSLT, it's pretty easy an elegant. (Not my idea, but DocBook's.) > on a different note: something that would have great value (and > tremendous hacker brownie points) is for someone to tap deep into > LaTeX and to provide its internal representation in Python or XML > to do its output... there is pdflatex, (ps)latex, wondering if > someone could produce xmllatex? (like, write a latex-to-xml > translator) This is done by TeX4ht in a quite clever way: It deploys lots of hooks in the dvi file and processes it, which means that it needn't muck about the LaTeX syntax with all its quirks. There is no "internal representation" I'm afraid. Therefore, transforming LaTeX to high-structure formats is -- according to a common saying in a TeX newsgroup -- like making potatoes out of chips. Tsch?, Torsten. -- Torsten Bronger, aquisgrana, europa vetus From mike at pcblokes.com Fri May 27 15:32:31 2005 From: mike at pcblokes.com (Michael Foord) Date: Fri, 27 May 2005 14:32:31 +0100 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <200505261634.18730.fdrake@acm.org> References: <4295834C.9010707@pcblokes.com> <8393fff050526091067e0e00f@mail.gmail.com> <200505261634.18730.fdrake@acm.org> Message-ID: <4297216F.4080800@pcblokes.com> Fred L. Drake, Jr. wrote: >[snip..] > > > > Surely implementing a Python source reader (that works by introspection > > > or whatever) that extracts docstrings and inserts it *into* a reST > > > document (still a two pass process) would help as a short term measure. > >It shouldn't be too hard to implement a specialized URL scheme that converts >an object reference to documentation text for some particular system. > > > > I don't fully follow you Fred - do you mean a URL scheme that *docutils* follows to retrieve the source text, or a URL scheme that another tool recognises and replaces with the text (but still making the original readable) ? (or something else altogether ?) I could knock up the second of these - inserting reST output into a document with information about the object referenced. If I can get the style right I could even investigate LaTex output (for insertion into LaTex documents). I would probably still do this with introspection - although I note the frustration of Mark Williamson trying to use Epydoc to generate docs for Twisted. It can't work for him *because* it uses introspection...... Ian's offering produces nice looking output in the page he references - but the implementation seems tied to his paste framework. Best Regards, Fuzzy http://www.voidspace.org.uk/python > -Fred > > > From Felix.Wiemann at gmx.net Fri May 27 15:48:42 2005 From: Felix.Wiemann at gmx.net (Felix Wiemann) Date: Fri, 27 May 2005 15:48:42 +0200 Subject: [Doc-SIG] Python docs in reST? References: <05May18.144608pdt.58617@synergy1.parc.xerox.com> <87oeb77jpq.fsf@wilson.rwth-aachen.de> <87k6ln6mcx.fsf@news2.ososo.de> <87fywayia7.fsf@wilson.rwth-aachen.de> Message-ID: <87r7fssrt1.fsf@news2.ososo.de> Torsten Bronger wrote: > [Field lists:] I dislike this table-like style. Currently, my source > may contain this: > > unlock > ...... > > Relinquish a lock for the specified resource. > > :Call: unlock(vi) > :Parameters: > `vi` : ViSession > Unique logical identifier to a session. > :Return values: > None. > > I want "Call" and "Parameters" to go away, and I don't want to feel > forced to start a new section for every function. These two problems can be solved with roles and definition lists: .. role:: call unlock Relinquish a lock for the specified resource. :call:`unlock(vi)` `vi` : ViSession Unique logical identifier to a session. Return values: None. If you like, you can add a "parameter" role and write :: :parameter:`vi` : ViSession but that looks rather clumsy, because you'd have to mark up every single parameter. If you don't like the deep nesting and the duplication of the function name, you could place the :call: definition at the top level: .. role:: call .. role:: parameter :call:`unlock(vi)` Relinquish a lock for the specified resource. :parameter:`vi` : ViSession Unique logical identifier to a session. Return values: None. Looks quite nice in the HTML output, IMO. > But first and foremost, I want to have the impression that I tell the > reST interpreter everything I can. Here, for example, I know that > "unlock" is the function's name, "vi" is a parameter object, > "ViSession" is a type. However, I can't pass this knowledge to reST. You can. Just define a role for every thing you want to mark up, like: .. role:: function_name .. role:: type etc. And then write e.g. :type:`ViSession`. This looks over-formalized (for my taste), but that's not really reST's fault -- you have that problem in every markup language, I'd suppose. -- For private mail please ensure that the header contains 'Felix Wiemann'. http://www.ososo.de/ From Felix.Wiemann at gmx.net Fri May 27 16:32:55 2005 From: Felix.Wiemann at gmx.net (Felix Wiemann) Date: Fri, 27 May 2005 16:32:55 +0200 Subject: [Doc-SIG] Python docs in reST References: <200505241632.24134.fdrake@acm.org> <87r7fv6mk6.fsf@news2.ososo.de> <200505251955.47328.fdrake@acm.org> Message-ID: <87mzqgsprc.fsf@news2.ososo.de> Fred L. Drake, Jr. wrote: > Felix Wiemann wrote: > >> Basically, the "include" *literally* inserts the contents of the >> included file into the input stream. > > I just mean that this should be a node insertion; header identification should > be completely independent of what underline characters are used in the > context document. Then you need a new "subdocument" directive. The "include" directive can't do that. >> times. To fix this, you'd need to parse all input files (without >> resolving references etc.) and store the results (namely Docutils node >> trees) in XML files. Later, when creating the big document, the XML > > Maybe XML, maybe not. Depends on how open the system should be > internally. If you want to store Docutils' node tree, XML is the most natural choice. That wouldn't change the openness of the system, and neither is anybody expected to fiddle with XML files. It's just a serialization of an intermediate processing stage. > This only matters if the actual parse is slow; anything else will be > relatively quick. Loading structure takes time too, regardless of > whether the structure is encoded in XML or as a Python pickle. Yes, but it's probably still much master than the reST parser, which processes about 10 KB/sec at 1.4 GHz. > If there's a significant parse time (and I think there is), we need to > work on the performance of the parser. > > (Has anyone specifically looked at this yet? If so, it would be interesting > to hear of any specific issues that were found and still need attention.) >From my profiling results, much time is spent in the statemachine and in the regular expressions. Dunno how much this can be optimized, though. >>> [output-splitting] needs to be done before other transformations. >> >> What other transformations? > > Things like generating page-local Tables of Contents come after, similar to > things like generating the final form of the links. Thanks; we'll have to keep this in mind. -- For private mail please ensure that the header contains 'Felix Wiemann'. http://www.ososo.de/ From Felix.Wiemann at gmx.net Fri May 27 17:44:21 2005 From: Felix.Wiemann at gmx.net (Felix Wiemann) Date: Fri, 27 May 2005 17:44:21 +0200 Subject: [Doc-SIG] Python docs in reST References: <4295834C.9010707@pcblokes.com> <8393fff050526091067e0e00f@mail.gmail.com> <200505261634.18730.fdrake@acm.org> Message-ID: <87fyw8smga.fsf@news2.ososo.de> Fred L. Drake, Jr. wrote: > There are a number of other projects that use the LaTeX markup from > Python; these are mostly related to Python in some way. (These also > create a reason to continue to maintain the LaTeX markup.) The ability to output LaTeX code compatible to the current Python-docs markup is a must-have for a Python-docs-in-reST implementation. So the existing LaTeX markup would still persist, even if Python's standard documentation was converted to reST. > [Docutils] certainly doesn't need to support it all out of the box, > especially the constructs specific to documenting Python (or other OO > programming languages). Yes; an implementation of "reST for Python docs" would probably be implemented as a plugin. I doubt something like that will ever appear in the Docutils core (so nobody needs to worry that standard-reST gets cluttered because of that; even though the objection that much explicit markup contradicts the goals of reST still stands). > It would be nice to have the option of using it for documenting at > least Python packages of small to moderate size, with support from an > add-on library of text roles and directives. Yes, that's what I was thinking of, too. -- For private mail please ensure that the header contains 'Felix Wiemann'. http://www.ososo.de/ From ianb at colorstudy.com Fri May 27 18:10:29 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 27 May 2005 11:10:29 -0500 Subject: [Doc-SIG] paste.docsupport (was: Python docs in reST) In-Reply-To: <42972398.7080109@pcblokes.com> References: <428A6299.6050005@colorstudy.com> <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <87k6ls2kl7.fsf@news2.ososo.de> <42902A51.90703@colorstudy.com> <17040.45129.807736.306028@montanaro.dyndns.org> <8393fff0505221139292ecd28@mail.gmail.com> <87fywb6kap.fsf@news2.ososo.de> <4295834C.9010707@pcblokes.com> <4295FFE1.3080102@colorstudy.com> <42972398.7080109@pcblokes.com> Message-ID: <42974675.50801@colorstudy.com> (I thought I'd reply back on-list, since this is still about documentation) Michael Foord wrote: > I've had a *quick* look at the paste docsupport. The output looks nice - > but the code looks tied to the paste framework. > > I have a couple of questions I hope you don't mind (I'm considering > implementing something myself - but don't want to duplicate something > you'd do better :-). > > Is your docsupport *specific* to paste - or *could* it work with other > modules ? Right now there's no separation of modules for the Paste and generic code, but the objects are fairly partitioned between the two. There are some Paste-specific DocItem subclasses in docsupport.metadata; it's my expectation that any framework will require domain-specific metadata, and stuff like WSGIKey are examples of that. There's also one custom extractor (HTTPExtractor). > Does it produce standalone references for modules ? Could it be hacked > to support pulling out *specific* docstrings ? (I'd like to code a > convention in my reST docs - that allows for the referencing of specific > docstrings. That way I can produce documentation around my docstrings > rather than have my docstrings *as* documentation - if you see the > difference). Right now you run the extract.py module and pass in a package or module name. It creates a single document right now, but I'd definitely like to make it create a tree of files in the future, and probably automatically create a file that includes from that tree as well, as a kind of minimal fully-automated reference doc -- but you could include those same files into other documents as well. From there it would be really nice to have a link from the including document (with the docstring in isolation) to the larger reference document (that was organized by module), and from there to the actual source (maybe a highlighted source with lots of id tags). That would be neat. > Does it use any conventions in the docstrings to produce the output - or > does it just take the function (class, method etc) name and parameters > and docstrings - with no additional processing ? Right now there's no additional processing. If you look in docsupport.extract there's individual classes to create the formatting of different types of objects. E.g., MethodExtractor (which should probably be named FunctionExtractor) does the functions. You could register different implementations, or subclass the ones that are there. And actually though the docstring is treated like an opaque item currently, extractors look at other things. For example, you can do:: def blah(self, *args, **kw): ... return self.wrapped(*args, **kw) blah.proxy = 'wrapped' And the signature in the docs should match the signature for wrapped. wsgilib.interactive is an example of this. The attribute 'returns' is also searched for a return type -- well, actually just any documentable object (or string reference to an object) which gets inlined in the docs. Right now I'm not doing any real inspection of the docstrings; I haven't felt the need yet, and I'm not sure what distinctions I'll need to make. But at some point I'm sure it will be necessary. Most of the annotation so far has gone into more transparent structures, like attributes. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From bronger at physik.rwth-aachen.de Fri May 27 18:16:31 2005 From: bronger at physik.rwth-aachen.de (Torsten Bronger) Date: Fri, 27 May 2005 18:16:31 +0200 Subject: [Doc-SIG] Python docs in reST? References: <05May18.144608pdt.58617@synergy1.parc.xerox.com> <87oeb77jpq.fsf@wilson.rwth-aachen.de> <87k6ln6mcx.fsf@news2.ososo.de> <87fywayia7.fsf@wilson.rwth-aachen.de> <87r7fssrt1.fsf@news2.ososo.de> Message-ID: <87is14y78g.fsf@wilson.rwth-aachen.de> Hall?chen! Felix Wiemann writes: > Torsten Bronger wrote: > >> [...] >> >> :Call: unlock(vi) >> :Parameters: >> `vi` : ViSession >> Unique logical identifier to a session. >> :Return values: >> None. >> >> I want "Call" and "Parameters" to go away, and I don't want to feel >> forced to start a new section for every function. > > These two problems can be solved with roles and definition lists: > > [...] > > Looks quite nice in the HTML output, IMO. Yes indeed. >> But first and foremost, I want to have the impression that I tell >> the reST interpreter everything I can. Here, for example, I know >> that "unlock" is the function's name, "vi" is a parameter object, >> "ViSession" is a type. However, I can't pass this knowledge to >> reST. > > You can. Just define a role for every thing you want to mark up, > like: > > .. role:: function_name > .. role:: type > > etc. This is only half the way, because the parser doesn't know what it really means. Such basic concepts should be in reST's core in my opinion. > And then write e.g. :type:`ViSession`. This looks over-formalized > (for my taste), but that's not really reST's fault -- you have > that problem in every markup language, I'd suppose. It depends on how implicit you dare to make reST. If you define a template for a method or function explanation, it looks rwther simple in the source. Nevertheless, the transformer knows exactly what every token means. (As long as the parser can rightfully assume that you describe *Python* code.) Tsch?, Torsten. -- Torsten Bronger, aquisgrana, europa vetus From ianb at colorstudy.com Fri May 27 18:23:02 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 27 May 2005 11:23:02 -0500 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <4297216F.4080800@pcblokes.com> References: <4295834C.9010707@pcblokes.com> <8393fff050526091067e0e00f@mail.gmail.com> <200505261634.18730.fdrake@acm.org> <4297216F.4080800@pcblokes.com> Message-ID: <42974966.1020308@colorstudy.com> Michael Foord wrote: > I would probably still do this with introspection - although I note the > frustration of Mark Williamson trying to use Epydoc to generate docs for > Twisted. It can't work for him *because* it uses introspection...... I think it's kind of inevitable; I don't think it's possible to make a good document extraction system without object introspection, and that it's easier to simply resolve the non-importability of modules (or build docs on project machines that are fully set up, or whatever). Obviously this doesn't work well when you are trying to generate documents for someone else's project, but I think it's possible to produce very good results when the package wasn't intended for use with a given extraction tool. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org From Sylvain.Thenault at logilab.fr Fri May 27 18:34:16 2005 From: Sylvain.Thenault at logilab.fr (Sylvain =?iso-8859-1?Q?Th=E9nault?=) Date: Fri, 27 May 2005 18:34:16 +0200 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <42974966.1020308@colorstudy.com> References: <4295834C.9010707@pcblokes.com> <8393fff050526091067e0e00f@mail.gmail.com> <200505261634.18730.fdrake@acm.org> <4297216F.4080800@pcblokes.com> <42974966.1020308@colorstudy.com> Message-ID: <20050527163416.GC5805@logilab.fr> On Friday 27 May ? 11:23, Ian Bicking wrote: > Michael Foord wrote: > > I would probably still do this with introspection - although I note the > > frustration of Mark Williamson trying to use Epydoc to generate docs for > > Twisted. It can't work for him *because* it uses introspection...... > > I think it's kind of inevitable; I don't think it's possible to make a > good document extraction system without object introspection, and that > it's easier to simply resolve the non-importability of modules (or build > docs on project machines that are fully set up, or whatever). Obviously > this doesn't work well when you are trying to generate documents for > someone else's project, but I think it's possible to produce very good > results when the package wasn't intended for use with a given extraction > tool. however it's possible to make source code introspection without actually import modules and instrospecting living object, using things such as the compiler module of the std lib. -- Sylvain Th?nault LOGILAB, Paris (France). http://www.logilab.com http://www.logilab.fr http://www.logilab.org From Felix.Wiemann at gmx.net Fri May 27 18:37:27 2005 From: Felix.Wiemann at gmx.net (Felix Wiemann) Date: Fri, 27 May 2005 18:37:27 +0200 Subject: [Doc-SIG] Python docs in reST References: <8393fff0505221139292ecd28@mail.gmail.com> <87fywb6kap.fsf@news2.ososo.de> <200505251929.46721.fdrake@acm.org> Message-ID: <87acmgsjzs.fsf@news2.ososo.de> Fred L. Drake, Jr. wrote: > Felix Wiemann wrote: > >> * We want to have **well-readable** documentation source. > > We have that now. Not ReST, but its very readable. I just quickly wrote up a short LaTeX document to test that. It's not the Python doc format but just a normal LaTeX document, but I presume it won't make much of a difference. ---------------------------------------------------------------------- \documentclass{article} \begin{document} \begin{verbatim} >>> print "foo\nbar" foo bar >>> \end{verbatim} Consider the \texttt{foo$\backslash$nbar} string! \vspace{1em}\noindent\texttt{% This is the \emph{first} line.\\ This is the \emph{second} line.} \vspace{1em} I'm not gonna try to render a table with its $\backslash$hlines here. It's \emph{completely} unreadable, IMO. \end{document} ---------------------------------------------------------------------- Now have a look the same thing as reST: ---------------------------------------------------------------------- >>> print "foo\nbar" foo bar >>> Consider the ``foo\nbar`` string! .. parsed-literal:: This is the *first* line. This is the *second* line. I'm not gonna try to render a table with its \\hlines here. It's *completely* unreadable, IMO. ---------------------------------------------------------------------- I find the reST document shorter and more readable than the LaTeX document. (I will admit though that some of that readability gets lost if you explicitly mark up function names and the like.) Furthermore, and that's where my example stops, there are many block-level constructs which are extremely simple in reST but require significant markup in LaTeX: * In reST I can insert tables as simple tables, grid tables (with support from Emacs!), list-tables and CSV-tables. In LaTeX there is a "tabular" environment whose syntax I have to look up everytime I use it, and the table contents aren't nicely viewable in LaTeX (because they're interspersed with "&", "\\" and "\hline"). * There are some handy list-like constructs in reST, e.g. field lists or definition lists. I just type :Field name: Field body or Term : classifier Associated paragraph. and I have really nice output in HTML and LaTeX. In LaTeX I simply wouldn't know how to do this off the top of my head. It's possible of course, but it's just not simple and straightforward, and for somebody who doesn't have a LaTeX reference at hand it must be really difficult. >> * We want to get **flexibility** in that we can modify the parsed >> document using Python code. > > Is this a goal, or is this a means to an end? Flexibility is a goal by itself, IMO. >> * We want an **easy-to-use** documentation system. Any documentation >> should be renderable by typing "sometool.py input.txt output.html". >> Getting multi-document output should be similarly easy. > > Again, I think we have this now. "mkhowto input.tex" is pretty > straightforward. But it doesn't work out of the box and it's not easy to install. ~ $ cd source/python/python/dist/src/Doc/lib/ ~/source/python/python/dist/src/Doc/lib $ locate mkhowto /home/felix/source/python/python/dist/src/Doc/tools/mkhowto ~/source/python/python/dist/src/Doc/lib $ /home/felix/source/python/python/dist/src/Doc/tools/mkhowto libbase64.tex +++ TEXINPUTS=/home/felix/source/python/python/dist/src/Doc/lib:.:/home/felix/tex:/var/home/felix/source/python/python/dist/src/Doc/paper-letter:/var/home/felix/source/python/python/dist/src/Doc/texinputs: +++ latex libbase64 *** Session transcript and error messages are in /home/felix/source/python/python/dist/src/Doc/lib/libbase64/libbase64.how. *** Exited with status 1. [snipped error messages] Doc/README doesn't tell me what to install to make it work. Installing Docutils is very easy, OTOH. Download, unpack, click install.py. -- For private mail please ensure that the header contains 'Felix Wiemann'. http://www.ososo.de/ From Felix.Wiemann at gmx.net Fri May 27 19:03:54 2005 From: Felix.Wiemann at gmx.net (Felix Wiemann) Date: Fri, 27 May 2005 19:03:54 +0200 Subject: [Doc-SIG] Python docs in reST References: <428AEE84.8060208@pcblokes.com> <8393fff05051803515b0e2a93@mail.gmail.com> <428B6860.9020900@colorstudy.com> <87k6ls2kl7.fsf@news2.ososo.de> <42902A51.90703@colorstudy.com> <17040.45129.807736.306028@montanaro.dyndns.org> <8393fff0505221139292ecd28@mail.gmail.com> <87fywb6kap.fsf@news2.ososo.de> <4295834C.9010707@pcblokes.com> <8393fff050526091067e0e00f@mail.gmail.com> Message-ID: <8764x4sirp.fsf@news2.ososo.de> Martin Blais wrote: > Michael Foord wrote: > >> Felix Wiemann wrote: >> >>> Ideas: >>> >>> * We want to have **well-readable** documentation source. > > hmmm... the current latex markup is as simple as it can be. it's got > a word, and some form of delimiters, e.g. \function{foo}. i don't > feel that `foo`:function: is all that much better. No, indeed not. However, there are other differences I outlined in . > rest allows you to view the input files directly, as output (ascii > output = input). this assumes that the markup is kept to its *bare > minimum*. this is an *explicit* reason for creating rest in the first > place. the more we are talking about adding markup on rest, the > stranger this whole discussion is becoming. First of all, just to avoid misunderstandings, I don't think roles like :function: or :class: will be added to standard-reST. If it all, it will be an extension (possibly shipped with the Docutils distribution though) which must be explicitly activated (probably from inside the reST document, like ".. extension:: python-doc"). While the goal of reST is to avoid explicit markup, IMO it's fine to use explicit markup where considered necessary and it doesn't make reST less useful. > [...] with rest, the source has to more or less "look like" the > output. That's a goal, but it's not a requirement: You can still use reST if the output doesn't look exactly like the reST source. And even if you mark up function names, parameters and class names, still a lot of reST's WYSIWYG and simplicity is retained, and that's a big advantage over LaTeX. > IMHO the worst thing that could happen is that one day I open a text > file supposedly writtne in rest, and that i can't understand it > because i have to learn what all that new markup is all about. As I wrote above, all that explicit markup won't be part of standard-reST. -- For private mail please ensure that the header contains 'Felix Wiemann'. http://www.ososo.de/ From Felix.Wiemann at gmx.net Fri May 27 19:47:55 2005 From: Felix.Wiemann at gmx.net (Felix Wiemann) Date: Fri, 27 May 2005 19:47:55 +0200 Subject: [Doc-SIG] Python docs in reST? References: <05May18.144608pdt.58617@synergy1.parc.xerox.com> <87oeb77jpq.fsf@wilson.rwth-aachen.de> <87k6ln6mcx.fsf@news2.ososo.de> <87fywayia7.fsf@wilson.rwth-aachen.de> <87r7fssrt1.fsf@news2.ososo.de> <87is14y78g.fsf@wilson.rwth-aachen.de> Message-ID: <871x7ssgqc.fsf@news2.ososo.de> Torsten Bronger wrote: > Felix Wiemann writes: > >> Just define a role for every thing you want to mark up, like: >> >> .. role:: function_name >> .. role:: type >> >> etc. > > This is only half the way, because the parser doesn't know what it > really means. It doesn't need to, because it wouldn't treat them differently. However, since the markup is there, any tool can recognize those special tokens. You can apply different formatting in the HTML output using a CSS stylesheet, and you can extract them from Docutils' XML output just using an XML parser. > Such basic concepts should be in reST's core in my opinion. I don't think that's a good idea. It's too much explicit markup, and I just don't see the practical use of this. And after all, integrating it into reST's core would just mean making the ".. role:: function_name" commands implicit; there's no reason you cannot assemble them in a file and explicitly ".. include::" them yourself. >> And then write e.g. :type:`ViSession`. This looks over-formalized > > If you define a template for a method or function explanation, it > looks rwther simple in the source. Yeah, that's true. It's certainly something worth keeping in mind when writing a Python documentation system with Docutils. -- For private mail please ensure that the header contains 'Felix Wiemann'. http://www.ososo.de/ From martin.blais at gmail.com Fri May 27 20:00:49 2005 From: martin.blais at gmail.com (Martin Blais) Date: Fri, 27 May 2005 14:00:49 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <87acmgsjzs.fsf@news2.ososo.de> References: <8393fff0505221139292ecd28@mail.gmail.com> <87fywb6kap.fsf@news2.ososo.de> <200505251929.46721.fdrake@acm.org> <87acmgsjzs.fsf@news2.ososo.de> Message-ID: <8393fff05052711003aaff14e@mail.gmail.com> On 5/27/05, Felix Wiemann wrote: > Fred L. Drake, Jr. wrote: > > >> * We want an **easy-to-use** documentation system. Any documentation > >> should be renderable by typing "sometool.py input.txt output.html". > >> Getting multi-document output should be similarly easy. > > > > Again, I think we have this now. "mkhowto input.tex" is pretty > > straightforward. > > But it doesn't work out of the box and it's not easy to install. > > ~ $ cd source/python/python/dist/src/Doc/lib/ > > ~/source/python/python/dist/src/Doc/lib $ locate mkhowto > /home/felix/source/python/python/dist/src/Doc/tools/mkhowto > > ~/source/python/python/dist/src/Doc/lib $ /home/felix/source/python/python/dist/src/Doc/tools/mkhowto libbase64.tex > +++ TEXINPUTS=/home/felix/source/python/python/dist/src/Doc/lib:.:/home/felix/tex:/var/home/felix/source/python/python/dist/src/Doc/paper-letter:/var/home/felix/source/python/python/dist/src/Doc/texinputs: > +++ latex libbase64 > *** Session transcript and error messages are in /home/felix/source/python/python/dist/src/Doc/lib/libbase64/libbase64.how. > *** Exited with status 1. > [snipped error messages] > > Doc/README doesn't tell me what to install to make it work. indeed, but that's really just a question of packaging. all that needs to be done (and i did some of that earlier for one project but never finished it completely) is to make it independent, give mkhowto a fancy-looking name like python-build-docs, remove all the stuff specific to the python doc (they are easy to find from looking at the makefile, i.e. getting the version, some custom footers and notices, etc.), and voila. it's low-hanging fruit. (i should just shut up and do it grmmbl.) From fdrake at acm.org Fri May 27 20:08:53 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Fri, 27 May 2005 14:08:53 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <8393fff05052711003aaff14e@mail.gmail.com> References: <87acmgsjzs.fsf@news2.ososo.de> <8393fff05052711003aaff14e@mail.gmail.com> Message-ID: <200505271408.53234.fdrake@acm.org> On Friday 27 May 2005 14:00, Martin Blais wrote: > indeed, but that's really just a question of packaging. all that > needs to be done (and i did some of that earlier for one project but > never finished it completely) is to make it independent, Yes, packaging remains to be done, but that's not a hard problem. It just needs to be done. I definately plan to do this. > give mkhowto > a fancy-looking name like python-build-docs, I'm torn about the name. I *like* mkhowto. I don't like long names like python-build-docs. -Fred -- Fred L. Drake, Jr. From fdrake at acm.org Fri May 27 20:37:40 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Fri, 27 May 2005 14:37:40 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <8764x4sirp.fsf@news2.ososo.de> References: <8393fff050526091067e0e00f@mail.gmail.com> <8764x4sirp.fsf@news2.ososo.de> Message-ID: <200505271437.40357.fdrake@acm.org> On Friday 27 May 2005 13:03, Felix Wiemann wrote: > First of all, just to avoid misunderstandings, I don't think roles like > :function: or :class: will be added to standard-reST. If it all, it > will be an extension (possibly shipped with the Docutils distribution > though) which must be explicitly activated (probably from inside the > reST document, like ".. extension:: python-doc"). I'd rather such an extension not be part of the docutils package; it should be possible to keep a separate release schedule for the extension package. > While the goal of reST is to avoid explicit markup, IMO it's fine to use > explicit markup where considered necessary and it doesn't make reST less > useful. Yep. I just don't see a way to avoid lots of markup. There are probably some ways to make the default interpreted text role context-sensitive to avoid having to be really heavy with markup in places where we can determine what's the most reasonable role. (`var` in a function description with an argument named "var" should be able to determine that `var` refers to the parameter.) > And even if you mark up function names, parameters and class names, > still a lot of reST's WYSIWYG and simplicity is retained, and that's a > big advantage over LaTeX. The ReST syntax is certainly nicer for things like different kinds of lists. Being able to specialize lists is a requirement, but should not be too difficult using specialized directives. -Fred -- Fred L. Drake, Jr. From fdrake at acm.org Fri May 27 20:38:32 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Fri, 27 May 2005 14:38:32 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <4297216F.4080800@pcblokes.com> References: <200505261634.18730.fdrake@acm.org> <4297216F.4080800@pcblokes.com> Message-ID: <200505271438.32795.fdrake@acm.org> On Friday 27 May 2005 09:32, Michael Foord wrote: > I don't fully follow you Fred - do you mean a URL scheme that *docutils* > follows to retrieve the source text, or a URL scheme that another tool > recognises and replaces with the text (but still making the original > readable) ? I think that's an implementation detail; it doesn't really matter if it inserts nodes in the tree or performs a text replacement as a separate step. > I would probably still do this with introspection - although I note the > frustration of Mark Williamson trying to use Epydoc to generate docs for > Twisted. It can't work for him *because* it uses introspection...... I've used epydoc a bit, but it's hard to use a lot on large projects without a general buy-in from the whole team. Given that it hasn't received any maintenance for a long time, it's hard to commit to. The introspection is an issue in the general case, but extraction via source parsing isn't trivial either. That's why there's always going to be a place for manual documentation construction. -Fred -- Fred L. Drake, Jr. From fdrake at acm.org Fri May 27 20:49:02 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Fri, 27 May 2005 14:49:02 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <87mzqgsprc.fsf@news2.ososo.de> References: <200505251955.47328.fdrake@acm.org> <87mzqgsprc.fsf@news2.ososo.de> Message-ID: <200505271449.02439.fdrake@acm.org> On Friday 27 May 2005 10:32, Felix Wiemann wrote: > Then you need a new "subdocument" directive. The "include" directive > can't do that. If ..include doesn't do it, we'd certainly need it. I'd not want an included document to be able to change the current tree context, only add children to the current node. ..subdocument would be used to include content, and ..include would be used to load custom definitions. > If you want to store Docutils' node tree, XML is the most natural > choice. That wouldn't change the openness of the system, and neither is > anybody expected to fiddle with XML files. It's just a serialization of > an intermediate processing stage. Yes; whether XML or something else makes sense depends on how "open" that intermediate form needs to be. An alternative might be pickles, which are substantially less open, but can probably be loaded more quickly. > From my profiling results, much time is spent in the statemachine and in > the regular expressions. Dunno how much this can be optimized, though. Good to know. I've not reviewed this code, but it seems like a good time to check it out. Not sure how much time I'll have; I've only managed time for these discussions because I'm on vacation this week. Needless to say, that's rapidly coming to a close. (Though I'll still keep up as best I can.) -Fred -- Fred L. Drake, Jr. From fdrake at acm.org Fri May 27 21:10:49 2005 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Fri, 27 May 2005 15:10:49 -0400 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <87acmgsjzs.fsf@news2.ososo.de> References: <200505251929.46721.fdrake@acm.org> <87acmgsjzs.fsf@news2.ososo.de> Message-ID: <200505271510.49573.fdrake@acm.org> On Friday 27 May 2005 12:37, Felix Wiemann wrote: > I just quickly wrote up a short LaTeX document to test that. It's not > the Python doc format but just a normal LaTeX document, but I presume it > won't make much of a difference. Don't make that assumption. > ---------------------------------------------------------------------- > \documentclass{article} > \begin{document} > \begin{verbatim} > > >>> print "foo\nbar" > > foo > bar > > \end{verbatim} Good so far. > Consider the \texttt{foo$\backslash$nbar} string! Consider the \code{foo\nbar} string! > \vspace{1em}\noindent\texttt{% > This is the \emph{first} line.\\ > This is the \emph{second} line.} > \vspace{1em} I'm not sure what the use-case is here, but there shouldn't be anything like this in the Python docs. This should probably be an "alltt" environment like this: \begin{alltt} This is the \emph{first} line. This is the \emph{second} line. \end{alltt} The line-block syntax in ReST is nice, but I don't know that there's much use for it in the Python documentation. We do use alltt and verbatim environments a lot. > I'm not gonna try to render a table with its $\backslash$hlines here. > It's \emph{completely} unreadable, IMO. > \end{document} > ---------------------------------------------------------------------- Yes, tables don't do well in LaTeX. I'm not so familiar with the ReST syntax either, but had a hard time with the few I've done in ReST (lines end up *way* too long; at least that's not a constraint in LaTeX, since I can put rows on separate lines). Note that tables in Python documentation tend to be marked up very differently than in raw LaTeX. > * In reST I can insert tables as simple tables, grid tables (with > support from Emacs!), list-tables and CSV-tables. In LaTeX there is a > "tabular" environment whose syntax I have to look up everytime I use > it, and the table contents aren't nicely viewable in LaTeX (because > they're interspersed with "&", "\\" and "\hline"). As noted above, the Python documentation style wraps the raw LaTeX table constructs. There are reasons for this. :-) > * There are some handy list-like constructs in reST, e.g. field lists or > definition lists. I just type Indeed. The nice list syntaxes in ReST are a definate benefit. > In LaTeX I simply wouldn't know how to do this off the top of my head. > It's possible of course, but it's just not simple and straightforward, > and for somebody who doesn't have a LaTeX reference at hand it must be > really difficult. We don't need to do this directly in standard LaTeX markup; we introduce specialized list-based environments so they can be marked semantically, and processing can be applied as needed. -Fred -- Fred L. Drake, Jr. From ianb at colorstudy.com Fri May 27 21:56:45 2005 From: ianb at colorstudy.com (Ian Bicking) Date: Fri, 27 May 2005 14:56:45 -0500 Subject: [Doc-SIG] Python docs in reST In-Reply-To: <200505271437.40357.fdrake@acm.org> References: <8393fff050526091067e0e00f@mail.gmail.com> <8764x4sirp.fsf@news2.ososo.de> <200505271437.40357.fdrake@acm.org> Message-ID: <42977B7D.7080700@colorstudy.com> Fred L. Drake, Jr. wrote: > > While the goal of reST is to avoid explicit markup, IMO it's fine to use > > explicit markup where considered necessary and it doesn't make reST less > > useful. > > Yep. I just don't see a way to avoid lots of markup. There are probably some > ways to make the default interpreted text role context-sensitive to avoid > having to be really heavy with markup in places where we can determine what's > the most reasonable role. (`var` in a function description with an argument > named "var" should be able to determine that `var` refers to the parameter.) I've always been fuzzy on what `name` really meant in ReST, when it doesn't have any other markup. But it seems vague enough to potentially be useful. Imagine that you specifically name objects that are documented, like: .. class:: ZipFile(file, mode='r', compression=ZIP_STORED) .. function:: is_zipfile(filename) Then when you refer to `is_zipfile`, it would know that this referred to a function, because of the directive declared elsewhere. Potentially you could have lookup rules that would first search the "local namespace" (the module's document), then look for another module (e.g., `zlib.compress`). I'm not sure if it would be reasonable to change the documentation to make everything unambiguous, for instance changing this zipfile.ZipInfo doc: Class used to represent information about a member of an archive. Instances of this class are returned by the getinfo() and infolist() methods of ZipFile objects. To: Class used to represent information about a member of an archive. Instances of this class are returned by the `ZipFile.getinfo()` and `ZipFile.infolist()` instance methods. Ugh. And I don't know if there's a good way of making indirect links, like: ... returned by the `getinfo() `_ ... But, hm... maybe that's not so bad. You wouldn't know what kind of thing you were referencing until you built all the docs (first pass) and resolved the links, but I don't know if that's actually a problem. It's just dynamically typed documentation ;) It *does* seem like "semantic markup" of references is just another way of saying "static typing" -- you are giving types to names, when only objects should have types. -- Ian Bicking / ianb at colorstudy.com / http://blog.ianbicking.org