[Doc-SIG] Some random thoughts

[Peter Funk]

Hi!

Laurence Tratt:
> Things have been a little quiet on here recently once again confirming the
> doc-sigs yoyo nature; I sometimes wonder if people here are on a 3 month
> hibernation cycle <wink>.

I apologize for not answering immediately on your annoucement of Crystal.
But at least I have downloaded your code and spend one or two hours
looking into it.  I decided to wait a bit, what other people think 
about it before going to critize someone elses code without providing
something better.

First I wonder, why you decided  to use John Aycocks Spark as parser
instead of using the builtin parser provided by the standard library.
John Aycocks parser sure has its advantages and I use it myself in 
another project.  But if the task is parsing Python sourcecode, using 
the builtin parser seems to be more obvious.

The structure of your program was somewhat hard to grasp for me:
As far as I can see, the interesting functionality is contained 
(or may I say somewhat hidden?) in the following two modules:
   crystal/Defaults/Python_To_HTML3.py
and
   crystal/Doc_Parsers/StructuredText/Facade.py
This was not easy to figure out.  The overuse of 12 packages for a
program with only 19 modules containing real code seems to be too
much structure for a 0.1 version.  The 8 char TAB-indentation, the
long lines and the waste of vertical space made your code rather hard
to read for me.

As far as I understood your code, the "outputter" has the task
to transform an intermediate structure tree and produce for example
HTML using the definitions provided by a dictionary like that in 
Python_To_HTML3.py, right?  The definitions are not much simpler
as if they would have been coded directly as Python class.
So why do you define your own special purpose language for such 
definitions, when html3_outputter__styles could have been a class
containing methods for each structure element?  May be I should
have spend more time on studying your code (which I didn't had).

> Anyway, I'd like to share a few thoughts with you based on my recent
> experiences implementing my Crystal system: take them for what they're
> worth. These are quite long, and maybe irrelevant; the recent massive debate
> on hyperlinks etc came at a point where coding was more important to me
> than reading so maybe some of these have been covered and maybe I'm just
> being dumb...
> 
> For those of you who were asleep / bored / had server problems etc, Crystal
> is a javadoc / pythondoc type system which you can download from:
> 
>   http://eh.org/~laurie/comp/python/crystal/
> 
> There is also some sample output on the page to give an idea of what output
> it produces on given inputs. Both of these are a little out of date (and
> please bear in mind this project is in the *very* early days - it's not user
> friendly although probably easier to get running than pythondoc!), but they
> give the general idea... I badgered my university to let me produce Crystal
> as my undergraduate degree project, so that may put a few things into
> perspective in the following...
> 
> OK, the rest from here on is definitely all IMHO.
> 
> 
>  StructuredText
> 
> One of the few things I did pull out of the recent debate was that it has
> been decreed that inline documentation should be at least based on
> StructuredText. That made me do a little U-turn as I had been working on
> something a little different; fortunately Crystal works on the idea of
> plugins so I just made a StructuredText compatible plugin[0]. Initially I
> wanted to use Jim Fultons module, but this turned out to be a bit of a
> problem for the following reasons:
> 
>   * It seems to be geared up for subclasses returning strings (I needed a
>       recursive data structure, not a string representation)
>     In fact, realistically, the implementation is set up with only HTML in
>       mind
>   * There's no real documentation for the implementation
>   * The implementation is *very* hard to understand if you haven't watched
>       it evolve

Well.  I was able to do my own small modifications on StructuredText.py
but I agree to some degree with you.  Jim Fultons approach is too heavily
based in string processing.  What I had in mind was to get rid of the
vertical space wasting empty line between UL or OL list portions.
[...]
>   * Are lists allowed to be recursive?
>      * Should this work?
>        * Is this out of this world?
[...]
>   * Ordered lists are ill defined: can they go in any order, or should it
>       be like the <OL Start = > HTML?
> 
>   * It is unclear whether the definition in a definition list should be
>       allowed to take styles. eg is:
> 
>     """
>        'code' -- description
>        *emph* -- description
>     """
[...]
>   * The example code protocol is crufty & non-overridable. What should:
> 
>     """
>     ...so for example:
> 
>        * element 1
>        * element 2
>     """
> 
>     do? C++ programs might get caught out with the actions of '::' in
>       StructuredText <wink>

Personally I would prefer the idea of Tim Peters doctest for examples.
He suggests to use the Interpreter prompt as markup for example code:

	>>> a = FooBar("Baz")
	>>> a.something_completely_different()
	>>> a.spam()

>   * Forcing anything between ' ' into <Code> seems particularly clumsy; * has
>       a good history of being an emphasis effect and ** is a cunning
>       extension to that, but ' ' seems unnatural

Agreed.  This has been discussed here before.
[...]
>   * From a purely Python perspective, having _ _ as the underline protocol
>       tends to cause __init__ type method names to come out somewhat
>       unexpectedly. But that's not StructuredTexts fault <wink>

As has been said before, underlining is bad markup anyway.  It is not
used in any reasonable typeset output.

>   * Should styles nest? So is *this **going** to work* ?

I would say *not*.  
[...]
> One thing I don't think any of us are sure of is what we should do with all
> the doc strings we've already got. Thankfully my experience is that most doc
> comments are very nearly in a StructuredText compatible format anyway.

section headlines
-----------------
I think it would buy much, if a few additional features will be added to the 
structured text idea:  lists are fine, but another important doc structure 
element are section and subsection headers.  Often they are marked up as a line 
of text containing the headline followed by line of '-' with the same
length.  Both lines may be indented.  The standard library modules 
'cgi', 'pipes' and 'pprint' contain examples of this kind of markup.

[...]
> As you can see none of these is exactly serious. It is my opinion that it is
> *not* necessary to simply output all current doc comments in a monospaced
> pre-formatted font: aesthetically this is awful, and practically I think
> I've demonstrated with Crystal that it's not really necessary.

Agreed.

Just my 2 cents, Peter
-- 
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)