, so the non-test pieces can be indented with that, leaving the
    more common `this is what would really happen, try it and see'
    flavour of embedded code (which Tim's tool will duly verify ;^) to
    be written the way the pythoneer wanted to write it.
(c) A bunch of lines sharing initial indent and mostly starting >>> form
    a logical sub-division just as long as the audience know to
    recognise it as such - and the audience here consists of pythoneers,
    so we will recognise it.

Various folk discussed language.  My ha'p'n'rth on that would go for a
variable in the module namespace, nominally

__language__ = 'en:UK'	# expect English spellings, like colour, sulphur

and I'd vote for the *default* to be Dutch (to encourage US anglophones
to get used to admitting that they speak 'en:US' or whatever it's called)
though I realise I might have to live with 'en:US'.

Why, you might ask, do I want it in the module namespace ?

So that the contents of the doc string are *all* in the same language:
it'd just be perverse to have an anglophone keyword (Language) as the
one keyword which we don't translate, in doc-strings; and the magic
names in a module's namespace, like the reserved words of the language
(*outside* the doc string) are already condemned to monolinguism, so we
might as well leverage their sacrifice to enable the purity of the doc
strings.

Either that or do something entertaining which involves looking for a
match to:

: 

and I'll be immensely impressed if you can make that work.

Of course, *within* the selected language, I'd be more than happy to
watch (if anyone can be bothered to implement) something along the lines
of (with __language__ set to an English variant)

"""blah(burble) -> wibble -- rumbles

... in English ...

Translation:
    Language: French
    ... en Francais ...

    Traduction:			# (perverse but legitimate ...)
        Langue: Allemande
        ... im Deutsch ...	# (no, really, I'm just guessing)

Translation:
    Language: Norse
    ... på Norsk ...

etc.
"""

in which Translation, Language and the language selected are given in
the host __language__, but the rest of each translation block is in the
guest language (if you see what I mean).  But, as the Norse case
illustrates, how will docstrings cope with encoding languages which need
more than ASCII provides ?  I've defaulted to borrowing HTML's character
entities for this, but I'll bet a Norse author would get swiftly fed up
with doing that ...

However, this is yet more gratuitous over-complete specification ...
We have, collectively, said enough that I'd trust any of the assembled
folk (and, for that matter, any lurkers we may have) to take David's
revised grammar (due some time soon ?) and, whatever they implement,
I'm sure I'll be much happier with it than with what we have now.

	Eddy.
--
Celui qui parle trois langues s'appelle un trilangue.
Celui qui parle deux langues s'appelle un bilangue.
Mais celui qui parle seulement un langue s'appelle un anglophone.
				-- Quebecois joke.

From Manuel Gutierrez Algaba   Wed Dec  1 22:00:06 1999
From: Manuel Gutierrez Algaba  (Manuel Gutierrez Algaba)
Date: Wed, 1 Dec 1999 22:00:06 +0000 (GMT)
Subject: [Doc-SIG] Docstring syntax discussion
In-Reply-To: <14405.28293.197778.84868@weyr.cnri.reston.va.us>
Message-ID: 

On Wed, 1 Dec 1999, Fred L. Drake, Jr. wrote:
> 
>   I'd vote for this version only; each docstring can either processed
> or not depending on the language identifier.  A language identifier in 
> the module docstring (that means the *first* one only!) can be taken
> to mean "this language is the default for this module."
> 
>  > unspecified/unknown (which is politically correct) or english (which is 
>  > probably a lot more practical, since I don't think we can teach these 
>  > americans to add the Language tag all the time:-).
> 
>   Hear, hear!  We're just a bunch of closed-door isolationists with an 
> attitude!  Bad Europeans, bad!  ;-)

Certainly , ;) We at least should turn into Europeans ( as a block)
and start speaking Spanish, the most spoken language in the EU.
I'm sure that French and German people agree with me :-P

Seriously, I'm not very sure if docstring in Polish are a good idea,
docs is supposed to be useful for everybody, not a barrier for 
anybody except Ukrainians or Slovaks or for the people the author
has written that docstrings. 

It's rather common to find German doc in GPL programms, and this
is not a very good idea, I think. English is the de-facto lingua
franca. Japanese are funny too, remember ruby? 

BTW: Don't be so happy, I mean, the David Ascher gang, yea, yeah,
the bullet-indentation crowd (MAL, DA and the others) :). I'm 
preparing a nice counterattack in the form of eval docstrings.
You'll hear from me soon. The war-axe is still hot and bloody.

Regards/Saludos
Manolo
www.ctv.es/USERS/irmina    /TeEncontreX.html   /texpython.htm
 /SantisimaInquisicion/index.html 

  If little green men land in your back yard, hide any little green women you've got in the house. -- Mike Harding, "The Armchair Anarchist's Almanac"

From bhoel@server.python.net  Wed Dec  1 20:58:00 1999
From: bhoel@server.python.net (Berthold Höllmann)
Date: 01 Dec 1999 21:58:00 +0100
Subject: [Doc-SIG] Docstring syntax discussion
In-Reply-To: Jack Jansen's message of "Wed, 01 Dec 1999 11:46:14 +0100"
References: <19991201104615.1260F370CF2@snelboot.oratrix.nl>
Message-ID: 

Jack Jansen  writes:

> > Hello all,
> > 
> > I think the current docstring syntax discussion fails to address a
> > very important issue. Not all users of Python are native english
> > speakers and writers (I'm of course one of these). I think localizing
> > python modules and programs is an important issue. Localizing does not
...
> """
> Alternative:
> 	Language: en
> 	Description:
> 		This function does nothing.
> Alternative:
> 	Language: nl
> 	Description:
> 		Deze functie doet niets.
> """
> 
> It is probably open to discussion whether the default for Language should be 
> unspecified/unknown (which is politically correct) or english (which is 
> probably a lot more practical, since I don't think we can teach these 
> americans to add the Language tag all the time:-).

My "proposal" has the advantage that even the on line documentation
remains usable for the user. Also is allows everyone to use her/his
own syntax for the documentation strings. I would like a syntax that
allows things like complicated formulas in my documentation. And I now
that I get problems when I should edit at leas two different versions
of my Documentation. I would prefer to keep it in one place, providing
on line help as well as nice printed books. My example was of course a
quick hack, cause I only frequently found time to think about this.
Also I have no idea how to add this functionality to functions instead
of classes.

Cheers

Berthold
-- 
bhoel@starship.python.net / http://starship.python.net/crew/bhoel/
        It is unlawful to use this email address for unsolicited ads
        (USC Title 47 Sec.227). I will assess a US$500 charge for
        reviewing and deleting each unsolicited ad.

From da@ski.org  Wed Dec  1 21:06:28 1999
From: da@ski.org (David Ascher)
Date: Wed, 1 Dec 1999 13:06:28 -0800 (Pacific Standard Time)
Subject: [Doc-SIG] Docstring syntax discussion
In-Reply-To: 
Message-ID: 

Language: English-US-CA-SF

On Wed, 1 Dec 1999, Manuel Gutierrez Algaba wrote:

> BTW: Don't be so happy, I mean, the David Ascher gang, yea, yeah,
> the bullet-indentation crowd (MAL, DA and the others) :). I'm 
> preparing a nice counterattack in the form of eval docstrings.
> You'll hear from me soon. The war-axe is still hot and bloody.

Oh yeah?  Well I got my switchblade, my double-barreled shotgun loaded
with rock salt at the ready, and some big old chains.  

Rich Salz once called me an 'old timer', 
  Edward Welbourne claimed I knew something about parsing,
    and now Manuel says I'm a 'gang leader'.

      Good thing on the Internet nobody knows you're a dog...

--david

From skip@mojam.com (Skip Montanaro)  Wed Dec  1 21:53:11 1999
From: skip@mojam.com (Skip Montanaro) (Skip Montanaro)
Date: Wed, 1 Dec 1999 15:53:11 -0600 (CST)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: 
References: <004901bf3b60$30f49de0$f25728a1@UNITEDSPACEALLIANCE.COM>

Message-ID: <14405.39111.334408.713057@dolphin.mojam.com>

    Edward> Skip said:
    >> len(o:sequence) -> IntType
    Edward> no, yuck, don't do it.

    Edward> Pack that information into the argument sections by all means;
    Edward> but the way for that one-liner to `name' the arguments should be
    Edward> about getting across the `what does this argument mean'
    Edward> information.  Being told

    Edward>     transcribe(s1:stream, s2:stream)

    Edward> doesn't tell me the thing I really want to know, where

    Edward>     transcribe(source, target)

Whatever... ;-)

The point I guess I tried to make but didn't was that the

    sig -> type

line types the return value but not the parameters.  If the arguments are to
be typed someplace else, that's cool.  Where will the return type be
"declared"?  Just in the -> sig?

Skip Montanaro | http://www.mojam.com/
skip@mojam.com | http://www.musi-cal.com/
847-971-7098   | Python: Programming the way Guido indented...

From tim_one@email.msn.com  Wed Dec  1 22:40:26 1999
From: tim_one@email.msn.com (Tim Peters)
Date: Wed, 1 Dec 1999 17:40:26 -0500
Subject: [Doc-SIG] docstring grammar
In-Reply-To: <001d01bf3be5$ee40ffd0$f0c809c0@lslp7o.lsl.co.uk>
Message-ID: <000001bf3c4d$10baf720$232d153f@tim>

[David Ascher]
> The only question I suppose is whether one should require a
> keyword (Test: or other) to keep the top-level syntax trivial, or
> special-case the recognition of >>>-beginning paragraphs.
>
> I'm leaning for the former, as it can evolve to the latter if there
> is sufficient call for it from the user base, and I think it does
> keep the code simpler.  But I'm willing to be swayed.

[Tony J Ibbs (Tibs)]
> No - keep the keyword. My reasoning is (a) I like it [emotional
> reaction, which is the real reason (parse that as "it feels more
> elegant")],

Note that I'm not asking to get rid of the keyword ("Test: or other" -- btw,
the very fact that David can't think of a compelling name is the very reason
">>>" is so highly desirable:  the latter is the only choice that isn't
fabricated out of thin air -- ">>>" is *natural*).  Use a keyword if you
like -- doctest doesn't care, so long as it finds ">>>" sooner or later.

> and (b) I still have the feeling that on occasion I might want non-
> test Python script in there,

It's certainly odd that people who don't use doctest are suddenly worried
about how to stop it from testing their code  -- doctest doesn't
run tests unless you run doctest.  If you do run doctest and have Python
script you don't want tested, simply refrain from starting it with ">>>"!
For example, doctest won't touch

    Example:
        m = MyClass(4, "red")
        assert len(m.color()) != m.int()

I'm not advocating that *all* code examples start with ">>>", just that
">>>" be accepted as one of the ways of introducing an example.  I have (at
least) hundreds of code examples already in that format, and they already
look nice and work great (people recognize them instantly for what they are,
and create their own with ease).

> and (c) it *is* a 'logical' subdivision of the text in exactly the
> same way as the other major divisions, and so deserves its own place.

Ah -- I don't view it as a major division at all.  So far as a doc parser is
concerned, at the "major" level a code block should be a single token (it
has no internal structure of interest).  I'd say it's much less complication
than the baroque proposed rules for recognizing bulleted lists, but is of
the same nature:  "if a line begins with such-and-such a sequence of
characters, interpret it as meaning so-and-so".

Looking at it from that view, the requirement that I write my doctest
examples as:

Test:
   >>> x + 1
   3

instead of as

>>> x + 1
3

is like requiring that everyone write:

Unordered-List:
    List-Item:
        First point.
    List-Item:
        Second point.

instead of as e.g.

+ First point.
+ Second point.

Since I have 100x more doctest examples in my modules than bulleted lists of
any flavor, the idea that the latter should be made especially easy but the
former made artificially clumsy does tend to grate .

>(to reply to Robin Friedrich later on - although I also don't understand
> his point about more tags making it harder to parse things (unless he
> means "for humans to parse")).

As well as for humans to write and to remember.

> Am I allowed to disagree with Tim Peters

Certainly!

>> You'll end up recognizing that with a regexp, like
>>
>>    r"^\s*Example:\s*"

> No! No! Whilst I realise that any General Purpose, Released With
> Python tool will probably have to use re 'cos that's all it has,
> *I* (for one) would never end up recognising anything much with
> a regexp. Follow the One True Way - convert to mxTextTools (gosh,
> I feel better now).

I didn't mean to proselytize on that issue one way or the other.
Recognizing ">>>" is near-trivial with mxTextTools too, or even with
string.find -- I'm trying to introduce  some sanity against the notion
that ">>>" is some kind of *burden* for a programmed parser to recognize.
It's not:  it's a fixed string that's extremely unlikely to appear by
accident, and by that measure is less a headache than list-item prefixes.

> ...
> because I have a sneaky feeling Tim's doc-code-tester *wants* to test
> all code given as examples to make sure they all work (or "fail in the
> right way"). Hmm.

doctest tests all and only stuff it finds in ">>>" blocks, and I've never
seen a ">>>" block in a docstring *unless* it was put there specifically for
doctest to find.  People writing "plain old" (not-to-be tested) examples
simply don't paste interactive sessions into their docstrings, so there's no
">>>", so doctest leaves their examples alone.  Instead they mix prose with
inline code fragments that fail to work as advertised 3 hours after the docs
are written <0.8 wink>.

Changing what doctest does isn't an option here:  in practice, it's proved
to be an essentially perfect solution to the problems it tried to address,
and part of "perfection" was making it dirt simple enough that even
sub-average programmers can and do use it successfully within minutes of
downloading the pkg.  I'm not mucking with the hard-won qualities that made
this possible!  doctest will continue to work fine no matter what we do
about doc markup; the only question I have here is whether Doc-SIG markup
will play nice with existing and future doctest-using modules.

the-difference-is-about-one-line-of-code<0.5-wink>-ly y'rs  - tim

From da@ski.org  Wed Dec  1 22:47:10 1999
From: da@ski.org (David Ascher)
Date: Wed, 1 Dec 1999 14:47:10 -0800 (Pacific Standard Time)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: <000001bf3c4d$10baf720$232d153f@tim>
Message-ID: 

On Wed, 1 Dec 1999, Tim Peters wrote:

On the issue of >>> vs. 'Example:\n\t>>>' -- I am willing to allow
doctests' current style as well as the 'verbose' keyword-tagged & indented
one.  I don't think the redundancy is a problem as far as users are
concerned, and if it complicates the code at all, I can just email Tim
saying it's not possible, and he'll provide the patch in 5 minutes. =)

--david

From skip@mojam.com (Skip Montanaro)  Thu Dec  2 00:06:01 1999
From: skip@mojam.com (Skip Montanaro) (Skip Montanaro)
Date: Wed, 1 Dec 1999 18:06:01 -0600 (CST)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: 
References: <000001bf3c4d$10baf720$232d153f@tim>

Message-ID: <14405.47081.929945.572825@dolphin.mojam.com>

    David> I don't think the redundancy is a problem as far as users are
    David> concerned, and if it complicates the code at all, I can just
    David> email Tim saying it's not possible, and he'll provide the patch
    David> in 5 minutes. =)

David,

You forgot about the time machine.  You should have written:

    I don't think the redundancy is a problem as far as users are erned, and
    if it complicates the code at all, I can just email Tim saying it's not
    possible, and he provided the patch 5 minutes ago.

Skip Montanaro | http://www.mojam.com/
skip@mojam.com | http://www.musi-cal.com/
847-971-7098   | Python: Programming the way Guido indented...

From da@ski.org  Thu Dec  2 00:08:31 1999
From: da@ski.org (David Ascher)
Date: Wed, 1 Dec 1999 16:08:31 -0800 (Pacific Standard Time)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: <14405.47081.929945.572825@dolphin.mojam.com>
Message-ID: 

On Wed, 1 Dec 1999, Skip Montanaro wrote:

>     David> I don't think the redundancy is a problem as far as users are
>     David> concerned, and if it complicates the code at all, I can just
>     David> email Tim saying it's not possible, and he'll provide the patch
>     David> in 5 minutes. =)
> 
> David,
> 
> You forgot about the time machine.  You should have written:
> 
>     I don't think the redundancy is a problem as far as users are erned, and
>     if it complicates the code at all, I can just email Tim saying it's not
>     possible, and he provided the patch 5 minutes ago.

Well, it's all a matter of what timezones.  Tim 'happens' three hours
later than I do, so his time machine sometimes misses 0 by a few minutes.
=)

'Tim happens.  Get used to it'

--da

From tony@lsl.co.uk  Thu Dec  2 09:45:27 1999
From: tony@lsl.co.uk (Tony J Ibbs (Tibs))
Date: Thu, 2 Dec 1999 09:45:27 -0000
Subject: [Doc-SIG] docstring grammar
In-Reply-To: 
Message-ID: <002b01bf3ca9$f68c1b60$f0c809c0@lslp7o.lsl.co.uk>

Tim Peters wrote:
> It's certainly odd that people who don't use doctest are suddenly worried
> about how to stop it from testing their code .

OK, I admit it, I've been entirely convinced that ">>>" is indeed enough of
a signifier, and that I should stop worrying now. Tim's explication of why
it wouldn't be a problem does explain to me why it won't be (damn, its
really irksome when someone makes having your mind changed so enjoyable -
maybe I just need a new mind...).

So do we need a "Test" keyword at all now? Only perhaps if one wants to have
a section which *explains* what is going on that has a separate heading, and
maybe *that* just means we want a "Title" keyword or somesuch, and if we're
getting to that level of detail right now I think I should stop.

Eddy's point that we'll probably have something that translates to
.. is also relevant, although I think that's actually now for
different reasons.

Are we there yet?

David Ascher wrote:
> 'Tim happens.  Get used to it'

Thank you! Thank you! I was wanting a Python signature to use as well as the
HPV ones!

Tibs
--
Tony J Ibbs (Tibs)      http://www.tibsnjoan.demon.co.uk/
'Tim happens. Get used to it'. (David Ascher, on the Doc-SIG)
My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.)

From uche.ogbuji@fourthought.com  Thu Dec  2 15:57:24 1999
From: uche.ogbuji@fourthought.com (uche.ogbuji@fourthought.com)
Date: Thu, 02 Dec 1999 08:57:24 -0700
Subject: [Doc-SIG] docstring grammar
In-Reply-To: Your message of "Wed, 01 Dec 1999 19:55:47 GMT."

Message-ID: <199912021557.IAA05118@localhost.localdomain>

> OK, start with the last: Tibs, you observed a while back that the human
> brain holds up to 7 (or is it 12 ?) things at the same time.  That's the
> `for humans to parse' constraint.  We want to keep to a minimum the set
> of keywords a programmer needs to be familiar with to be able to get
> pay-back from using the document format.

It sounds as if you are going for the Miller 7 +/- 2 rule, which holds that 
most people can hold between 5 and 9 units of related information at a time, 
particularly in short-term memory.

It is always a good idea in API and protocol design to keep main elements 
within the 7 +/- 2, but there is no reason not to have a few less used, 
optional and advisedly more arcane elements for the more experienced users.

So far it seems we're close to that point in "canonical doc-string", so I 
agree that as soon as the scapegoat (too bad it appears to be you, David, but 
that's what you get for introducing such a sterling proposal) comes up with a 
second proposal that takes all the recent discussion into account, we should 
call it alpha and start hacking.

> Various folk discussed language.  My ha'p'n'rth on that would go for a
> variable in the module namespace, nominally
> 
> __language__ = 'en:UK'	# expect English spellings, like colour, sulphur
> 
> and I'd vote for the *default* to be Dutch (to encourage US anglophones
> to get used to admitting that they speak 'en:US' or whatever it's called)
> though I realise I might have to live with 'en:US'.
> 
> Why, you might ask, do I want it in the module namespace ?
> 
> So that the contents of the doc string are *all* in the same language:
> it'd just be perverse to have an anglophone keyword (Language) as the
> one keyword which we don't translate, in doc-strings; and the magic
> names in a module's namespace, like the reserved words of the language
> (*outside* the doc string) are already condemned to monolinguism, so we
> might as well leverage their sacrifice to enable the purity of the doc
> strings.

I rather like this idea.  We are building a similar mechanism into 4Suite to 
support i18n messages, and it would be nice to unify the doc-string and code 
l10n mechanism.  I would rename it

__locale__ = 'en:UK'

To go in line with more common usage.  It would alctually then be nice for 
Python (1.6 feature?) to read the LC_ALL environment variable in UNIX, and the 
 equivalent on other platforms, and set a default for the __locale__.

I also like the idea that all keywords would be in the local language.

> Either that or do something entertaining which involves looking for a
> match to:
> 
> : 

Umm.  Or much rather not.

> Celui qui parle trois langues s'appelle un trilangue.
> Celui qui parle deux langues s'appelle un bilangue.
> Mais celui qui parle seulement un langue s'appelle un anglophone.
> 				-- Quebecois joke.

This is an absolute jem (or should I say 'bijou'?)  Strange how often the 
heartiest humor comes from the most cynical attitudes.

-- 
Uche Ogbuji
FourThought LLC, IT Consultants
uche.ogbuji@fourthought.com	(970)481-0805
Software engineering, project management, Intranets and Extranets
http://FourThought.com		http://OpenTechnology.org

From Edward Welbourne   Thu Dec  2 17:56:01 1999
From: Edward Welbourne  (Edward Welbourne)
Date: Thu, 2 Dec 1999 17:56:01 +0000
Subject: [Doc-SIG] docstring grammar
In-Reply-To: <199912021557.IAA05118@localhost.localdomain>
References: 
 <199912021557.IAA05118@localhost.localdomain>
Message-ID: 

> ... you are going for the Miller 7 +/- 2 rule ...
sounds familiar.  Anyone unfamiliar with this might benefit from the URL
Tibs just sent me in connection with your message:

        http://www.well.com/user/smalin/miller.html

> We are building a similar mechanism into 4Suite to support i18n messages,
ooo

> __locale__ = 'en:UK'
Sounds good.

> ... LC_ALL environment variable ... default for the __locale__.

No.  This is a user-specified value; many users may be using the same
source file, at many sites; the source file isn't changing as you change
user; __locale__ is telling pythonic tools which set of keywords is
being used in the source file's doc strings; this also isn't changing
from one user to another.

However, using LC_ALL (or equivalent) to configure which Language
section the pythonic tools extract to show to the user, now that'd be
cool.  And if the right Language is absent ... try talking to babelfish
(although I have to confess this is better for humour than sense).

>> : 

> Umm.  Or much rather not.

Sorry, it may not have been clear that that suggestion was of form `of
course, if anyone is volunteering to do something absurdly hard, far be
it from me to miss out on an opportunity to laugh myself breathless'.
Even if fate allows that there really is a way to do this (such that no
two languages will get confused by one another's keywords), it's a
fairly safe bet that about half of the keywords meaning `language' would
be words in other languages, with respectably many of them
offensive/lewd/absurd/funny in at least several other languages.

I must confess, I am looking forward to the growth of document-transfer
via babelfish and its kin.  Laughter is said to be good for one ...

	Eddy.

From fdrake@acm.org  Thu Dec  2 19:21:45 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Dec 1999 14:21:45 -0500 (EST)
Subject: [Doc-SIG] On David Ascher's Rant
In-Reply-To: <3841AC4B.3A29C921@lemburg.com>
References: 
 <3841AC4B.3A29C921@lemburg.com>
Message-ID: <14406.50889.525715.866036@weyr.cnri.reston.va.us>

M.-A. Lemburg writes:
 > I guess doc strings are just as personal to the programmer
 > as indention or naming styles: you won't get everybody to agree
 > on one way to do it.

  We don't have to get everyone to agree, or even to write the
standard format docstrings.  We need to determine one reasonable
format and get Guido to agree sanction it.  The rest will fall into
place over time.

 > Besides, I don't think this is really needed: as long as the
 > programmer can provide routines to parse his code everything
 > should be fine. This could e.g. be implemented by subclassing
 > a reader implementation which then passes the parsed tokens
 > to other code processing them for some other use.

  Nice idea.  I won't waste *any* time supporting it.
  We only *need* a very small number of things for the "next
generation" documentation to succeed (in terms of formats):

    1. One format for in-source module-reference documentation.
    2. One format for non-source module-reference documentation.
    3. One format for non-module-reference documents.

  Add to this a reasonable set of very easy to use tools that do a
good job of formatting documentation for hypertextual perusal, and
we've got a minimal system.  Specific output formats (HTML, GNU info,
HTML Help) are easily created once the first one is done.  Typeset
versions for printing are relatively easy, and nobody prints anything
anyway.  ;)
  Clearly I've abstracted away a lot of details, and there's more
design work than whacking at scripts for a couple of hours will avoid, 
but I really think that's the essence of what we need to do here.

  -Fred

--
Fred L. Drake, Jr.	     
Corporation for National Research Initiatives

From guido@CNRI.Reston.VA.US  Thu Dec  2 19:23:41 1999
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Thu, 02 Dec 1999 14:23:41 -0500
Subject: [Doc-SIG] On David Ascher's Rant
In-Reply-To: Your message of "Thu, 02 Dec 1999 14:21:45 EST."
 <14406.50889.525715.866036@weyr.cnri.reston.va.us>
References:  <3841AC4B.3A29C921@lemburg.com>
 <14406.50889.525715.866036@weyr.cnri.reston.va.us>
Message-ID: <199912021923.OAA14930@eric.cnri.reston.va.us>

> We need to determine one reasonable format and get Guido to agree
> sanction it.

At this point I'd sanction about anything that you and Dave Ascher can
agree upon.  I've been blissfully deleting the doc-sig messages
lately, knowing that the design is in good hands.  This is why I like
SIGs -- tip to the hat for the Doc-SIG!

--Guido van Rossum (home page: http://www.python.org/~guido/)

From fdrake@acm.org  Thu Dec  2 19:28:57 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Dec 1999 14:28:57 -0500 (EST)
Subject: [Doc-SIG] Re: What is important
In-Reply-To: 
References: 

Message-ID: <14406.51321.625602.696029@weyr.cnri.reston.va.us>

Peter Funk writes:
 > I agree with David.  I've joined the list only recently (some

  I do too, some days.  ;-)

 > current documentation for Python and the module library is very good.
 > Unfortunately some very important parts are still missing: Something
 > like Fredrik Lundhs "An Introduction to Tkinter" ---also it is still

  Agreed.  A number of those here know I sometimes send notes asking
for specific sections as well.  Are you volunteering a section for one 
of the manuals?  ;)  I'd love to be able to add new sections/chapters, 
and Andrew Kuchling would welcome new "How To" documents as well
(http://www.python.org/doc/howto/).

  -Fred

--
Fred L. Drake, Jr.	     
Corporation for National Research Initiatives

From fdrake@acm.org  Thu Dec  2 19:44:49 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Dec 1999 14:44:49 -0500 (EST)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: <38426C4A.ACC76AB5@lemburg.com>
References: 
 <38426C4A.ACC76AB5@lemburg.com>
Message-ID: <14406.52273.417510.947441@weyr.cnri.reston.va.us>

M.-A. Lemburg writes:
 > This raises the question of whether to parse or evaluate the
 > loaded module. Evaluation has the benefit of providing "automatic"

  Guido agrees with me on this one, sorry: it must be extractable from=20=

the parse tree.  Evaluation of module code is a *huge* no-no; we
should be able to run documentation tools on unknown (=3D=3D untrusted)=

code.

 > context, i.e. the symbols defined in the global namespace
 > are exactly the ones relevant for class definitions, etc. It
 > probably makes contruction of interdepence graphs a lot easier
 > to write. On the downside you have unwanted side effects due to
 > loading different modules.

  No, these will always be hard in Python.  Order of imports can be
significant, and the set of imports can change over the life of a
module (imports can be delayed to reduce startup time, or a number of
alternatives may be supported).

 > =B7 Mentioning the function/method signature is ok, but sometimes
 >   not needed since e.g. the byte code has enough information to
 >   deduce the signature from it. This is not true for builtin
 >   function which is probably the reason for all builtin doc
 >   strings to include the signature.

  That's right.  There's little need for signature repitition for
Python code.  There will be times it is appropriate, but that's the
oddball case.

 > =B7 I would extend the reference scheme to a lookup in the module
 >   globals in case the local one (in the Reference section) fails.
 >   You could then write e.g. "For details see the [string] module."
 >   and the doc tool would then generate some hyperlink to the
 >   string module provided the string module is loaded into the
 >   global namespace.

  Definately; the search sequence should mirror that of the runtime,
and the details need not be repeated.  That's what we have the
language reference for.

 > =B7 Standard symbols like __version__ could be included and used
 >   by the doc tool per default without the user specifying
 >   any special "Version:: %(__version__)s" % globals() tags.

  Definately.

  -Fred

--
Fred L. Drake, Jr.=09     
Corporation for National Research Initiatives

From fdrake@acm.org  Thu Dec  2 19:50:11 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Dec 1999 14:50:11 -0500 (EST)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: 
References: 
 <38426C4A.ACC76AB5@lemburg.com>

Message-ID: <14406.52595.555655.604073@weyr.cnri.reston.va.us>

Edward Welbourne writes:
 > We have, it occurs to me, another important namespace: unimported
 > modules.  Thus the string module doesn't import re, I assume, but may
 > wish to refer to it (e.g. to say `this function is a cheap variant of
 > the eponymous one in re') in its doc-strings.  Fortunately, we also have
...
 > Note, however, that some bypassing of this may be achieved using the
 > [blah] notation (which is good).

  Excellent point!
  I think this can be handled very nicely by stating that the [name]
syntax use (in this order):

    1. the standard Python search sequence
    2. fully-qualified names into the standard library

  The latter can be implemented in a number of different ways
depending on the desired level of efficiency and willingness to
pre-process documentation.

  -Fred

--
Fred L. Drake, Jr.	     
Corporation for National Research Initiatives

From fdrake@acm.org  Thu Dec  2 19:53:50 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Dec 1999 14:53:50 -0500 (EST)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: 
References: <38426C4A.ACC76AB5@lemburg.com>

Message-ID: <14406.52814.241790.308268@weyr.cnri.reston.va.us>

David Ascher writes:
 > Right.  It's not true for builtins, extension module functions, and I'm
 > not sure how easy it is for JPython code.  I have no problem with somehow
 > making it easy to omit those in cases where the information can be
 > obtained through the bytecode.

  The same information can be obtained from the parse tree; there's no 
need to generate or examine bytecode, unless you want to extend this
to work on .pyc files for which you don't have sources!

 > Fine.  I think that falls somewhat outside of the 'docstring' proposal,
 > but I agree with it.

  We should think of in-source documentation; docstrings are simply an 
important location for data-entry.  ;-)

  -Fred

--
Fred L. Drake, Jr.	     
Corporation for National Research Initiatives

From fdrake@acm.org  Thu Dec  2 20:10:44 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Dec 1999 15:10:44 -0500 (EST)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: <006b01bf3ae3$25b6fe00$0501a8c0@bobcat>
References: 
 <006b01bf3ae3$25b6fe00$0501a8c0@bobcat>
Message-ID: <14406.53828.589607.28770@weyr.cnri.reston.va.us>

Mark Hammond writes:
 > * IMO, importing the module to extract this information is fine.  For
 > the 1% of cases where it is not and the author of the module needs to

  No, it's not.  Never trust someone else's code until you've read the 
documentation, and don't trust the documentation if they wrote it.
Well, maybe *that's* going a little too far... but import is not
acceptable.  Using the parse tree also allows the order to be
well-defined, while introspection doesn't allow that at all.

 > chance of it one day existing :-)  Indeed, do it the simple way, and
 > the first person who needs the parse-only option can help code it :-)

  I maintain that the parse tree is the simple route to getting a
reliable tool, and I am working on coding one.  Neat, huh?

 > As a final note:  The tool should be written with distinct "generate"
 > and "collate" phases, simply to resolve the cross-references.  It is
 > unreasonable to expect that all cross-references will be capable of
 > being resolved in a single pass.  Note sure exactly what this means
 > from an implementation POV, but it is important.

  Easy from an implementation POV, and that's exactly my approach.
(My intention is to be able to document entire packages at a time as
well, rather than individual modules.)

  -Fred

--
Fred L. Drake, Jr.	     
Corporation for National Research Initiatives

From fdrake@acm.org  Thu Dec  2 20:45:13 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Dec 1999 15:45:13 -0500 (EST)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: <005501bf3d04$af1dea00$f25728a1@UNITEDSPACEALLIANCE.COM>
References: 
 <006b01bf3ae3$25b6fe00$0501a8c0@bobcat>
 <14406.53828.589607.28770@weyr.cnri.reston.va.us>
 <005501bf3d04$af1dea00$f25728a1@UNITEDSPACEALLIANCE.COM>
Message-ID: <14406.55897.793161.850862@weyr.cnri.reston.va.us>

Robin Friedrich writes:
 > We could argue this forever. Gendoc solved this by collecting info either
 > way, based on a runtime switch. As an author running this tool obviously I
 > trust the module/package to be imported and generate the docs that way. And
 > of course C module doc strings will need this feature.

  And I maintain that you can't get enough information from a C
extension that implements new types.  Method information is not
available without instantiating the new objects, and doing that is
difficult to determine how to do (or even if it's needed if the module 
doesn't export the newly-defined type objects).

 > Again, this is an application issue not a doc string grammar issue.

  Aside from the information discoverability issue, I'm perfectly
happy for someone to write a tool (or make pythondoc easily usable off 
the shelf) that uses import.  I have no intention of stopping anyone
from producing tools here!

  -Fred

--
Fred L. Drake, Jr.	     
Corporation for National Research Initiatives

From fdrake@acm.org  Thu Dec  2 20:55:29 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Dec 1999 15:55:29 -0500 (EST)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: 
References: <14406.53828.589607.28770@weyr.cnri.reston.va.us>

Message-ID: <14406.56513.540162.51075@weyr.cnri.reston.va.us>

David Ascher writes:
 > Indeed.  We've slyly conned someone who *does* know about parsing to do
 > the tool. =)

  What?  You mean I have to go back to school for all those classes I
skipped?  Well, maybe if you cover tuition+salary, and ask nicely.
Can I take a few classes on type theory as well?
  Actually, I started the code well before your proposal, but I've
been too swamped with various non- and semi-related things to do much
the last few days.  But that's starting to look better, and I got a
little more done last night.

  -Fred

--
Fred L. Drake, Jr.	     
Corporation for National Research Initiatives

From friedrich@pythonpros.com  Thu Dec  2 20:34:12 1999
From: friedrich@pythonpros.com (Robin Friedrich)
Date: Thu, 2 Dec 1999 14:34:12 -0600
Subject: [Doc-SIG] docstring grammar
References: <006b01bf3ae3$25b6fe00$0501a8c0@bobcat> <14406.53828.589607.28770@weyr.cnri.reston.va.us>
Message-ID: <005501bf3d04$af1dea00$f25728a1@UNITEDSPACEALLIANCE.COM>

From: Fred L. Drake, Jr. 
> Mark Hammond writes:
>  > * IMO, importing the module to extract this information is fine.  For
>  > the 1% of cases where it is not and the author of the module needs to
>
>   No, it's not.  Never trust someone else's code until you've read the
> documentation, and don't trust the documentation if they wrote it.
> Well, maybe *that's* going a little too far... but import is not
> acceptable.  Using the parse tree also allows the order to be
> well-defined, while introspection doesn't allow that at all.

We could argue this forever. Gendoc solved this by collecting info either
way, based on a runtime switch. As an author running this tool obviously I
trust the module/package to be imported and generate the docs that way. And
of course C module doc strings will need this feature.
Again, this is an application issue not a doc string grammar issue.
>
>  > chance of it one day existing :-)  Indeed, do it the simple way, and
>  > the first person who needs the parse-only option can help code it :-)
>
>   I maintain that the parse tree is the simple route to getting a
> reliable tool, and I am working on coding one.  Neat, huh?

I hope everyone slinging code looks at pythondoc first.

-Robin

From guido@CNRI.Reston.VA.US  Thu Dec  2 21:05:25 1999
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Thu, 02 Dec 1999 16:05:25 -0500
Subject: [Doc-SIG] docstring grammar
In-Reply-To: Your message of "Thu, 02 Dec 1999 14:34:12 CST."
 <005501bf3d04$af1dea00$f25728a1@UNITEDSPACEALLIANCE.COM>
References: <006b01bf3ae3$25b6fe00$0501a8c0@bobcat> <14406.53828.589607.28770@weyr.cnri.reston.va.us>
 <005501bf3d04$af1dea00$f25728a1@UNITEDSPACEALLIANCE.COM>
Message-ID: <199912022105.QAA15158@eric.cnri.reston.va.us>

[Robin Friedrich, on runtime vs. compile-time docstring extraction]
> We could argue this forever. Gendoc solved this by collecting info either
> way, based on a runtime switch. As an author running this tool obviously I
> trust the module/package to be imported and generate the docs that way. And
> of course C module doc strings will need this feature.
> Again, this is an application issue not a doc string grammar issue.

One issue: if I'm sloppy in my writing, I could easily have escape
sequences like \n in the doc string that are expanded by the
importing.  E.g. the comments for asynchat contain sentences like

# The handle_read() method looks at the input stream for the current
# 'terminator' (usually '\r\n' for single-line responses, '\r\n.\r\n'
# for multi-line output), calling self.found_terminator() on its
# receipt.

If we translate this into a doc string, either the doc string has to
be a triple-quoted *raw* string, or we'll have to double all the
backslashes, lest they be indistinguishable from real newlines:

"""
The handle_read() method looks at the input stream for the current
'terminator' (usually '\\r\\n' for single-line responses, '\\r\\n.\\r\\n'
for multi-line output), calling self.found_terminator() on its
receipt.
"""

Without doubling, this would become

"""
The handle_read() method looks at the input stream for the current
'terminator' (usually '\r
' for single-line responses, '\r
.\r
'
for multi-line output), calling self.found_terminator() on its
receipt.
"""

Just an annoyance, but something that the tool needs to consider.

(Now going back to trust-Fred-and-David mode :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From fdrake@acm.org  Thu Dec  2 21:29:52 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Dec 1999 16:29:52 -0500 (EST)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: <199912022105.QAA15158@eric.cnri.reston.va.us>
References: 
 <006b01bf3ae3$25b6fe00$0501a8c0@bobcat>
 <14406.53828.589607.28770@weyr.cnri.reston.va.us>
 <005501bf3d04$af1dea00$f25728a1@UNITEDSPACEALLIANCE.COM>
 <199912022105.QAA15158@eric.cnri.reston.va.us>
Message-ID: <14406.58576.943965.293063@weyr.cnri.reston.va.us>

Guido van Rossum writes:
 > One issue: if I'm sloppy in my writing, I could easily have escape
 > sequences like \n in the doc string that are expanded by the
 > importing.  E.g. the comments for asynchat contain sentences like
...
 > Just an annoyance, but something that the tool needs to consider.

  I don't think the concern is likely to go away by using a
parse-based approach.  Get your docstrings write or you've got a real
newline!
  Anyway it gets done, I don't think there's any call for tons of
magic interpretation for the escapes you put in your docstring.  We
don't want to introduce the magic-python-docstring-extraction
encoding.  The only reasonable way I know to get the actual string
from the parse tree is to yank the string representation of the
token.STRING node and eval() it.  Otherwise I have to re-write the
escaped-string to Python-string conversion in Python for both raw and
cooked strings.  If I have to consider screwed up strings on top of
that, we're back to the HTML-as-practiced problem.  I've done that
once, and don't plan to do it again!

 > (Now going back to trust-Fred-and-David mode :-)

  That's right.  Go back to sleep, go back to sleeep.....

  -Fred

--
Fred L. Drake, Jr.	     
Corporation for National Research Initiatives

From jack@oratrix.nl  Thu Dec  2 21:53:51 1999
From: jack@oratrix.nl (Jack Jansen)
Date: Thu, 02 Dec 1999 22:53:51 +0100
Subject: [Doc-SIG] docstring grammar
In-Reply-To: Message by "Fred L. Drake, Jr."  ,
 Thu, 2 Dec 1999 14:44:49 -0500 (EST) , <14406.52273.417510.947441@weyr.cnri.reston.va.us>
Message-ID: <19991202215356.8A2D2D52C5@oratrix.oratrix.nl>

Recently, "Fred L. Drake, Jr."  said:

>   Guido agrees with me on this one, sorry: it must be extractable from=20=
> 
> the parse tree.  Evaluation of module code is a *huge* no-no; we
> should be able to run documentation tools on unknown (=3D=3D untrusted)=
> 
> code.

If this is indeed agreed upon by everyone (even if only for the reason 
that Our Benevolent Dictator Says So) it makes the discussion quite a
bit simpler, as various of the things people proposed over the last
few days failed the no-execution test.
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 

From fdrake@acm.org  Thu Dec  2 22:09:38 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Dec 1999 17:09:38 -0500 (EST)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: <19991202215356.8A2D2D52C5@oratrix.oratrix.nl>
References: 
 <14406.52273.417510.947441@weyr.cnri.reston.va.us>
 <19991202215356.8A2D2D52C5@oratrix.oratrix.nl>
Message-ID: <14406.60962.991888.994231@weyr.cnri.reston.va.us>

Jack Jansen writes:
 > If this is indeed agreed upon by everyone (even if only for the reason 
 > that Our Benevolent Dictator Says So) it makes the discussion quite a
 > bit simpler, as various of the things people proposed over the last
 > few days failed the no-execution test.

  Discussion of what goes into the docstrings is still relevant.
There's also things like someone (MAL?) showed:

	__version__ = "$Revision: 1.10$"[11:-1]

that become much harder to detect; I would encourage people to use

	__version__ = "$Revision: 1.10$"

and expect the tools to recognized common things like RCS/SCCS cruft,
and decode appropriately.  These could be done in a module like
docutils.misc.  ;-)

  -Fred

--
Fred L. Drake, Jr.	     
Corporation for National Research Initiatives

From fdrake@acm.org  Thu Dec  2 22:58:14 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 2 Dec 1999 17:58:14 -0500 (EST)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: 
References: <000b01bf3b07$a465ad40$c92d153f@tim>

Message-ID: <14406.63878.954494.808286@weyr.cnri.reston.va.us>

David Ascher writes:
 > The only question I suppose is whether one should require a keyword (Test:
 > or other) to keep the top-level syntax trivial, or special-case the
 > recognition of >>>-beginning paragraphs.
 > 
 > I'm leaning for the former, as it can evolve to the latter if there is
 > sufficient call for it from the user base, and I think it does keep the
 > code simpler.  But I'm willing to be swayed.

  Sway.  ;-)
  I think anything that starts ">>>" or "..." should automagically be
a verbatim-thingy.  This is easy enough to implement and avoids excess 
cruft in docstrings.

  -Fred

--
Fred L. Drake, Jr.	     
Corporation for National Research Initiatives

From irmina@ctv.es  Fri Dec  3 00:24:41 1999
From: irmina@ctv.es (Manuel Gutierrez Algaba)
Date: Fri, 3 Dec 1999 00:24:41 +0000 (GMT)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: <19991202215356.8A2D2D52C5@oratrix.oratrix.nl>
Message-ID: 

On Thu, 2 Dec 1999, Jack Jansen wrote:
> Recently, "Fred L. Drake, Jr."  said:
> 
> >   Guido agrees with me on this one, sorry: it must be extractable from=20=
> > 
> > the parse tree.  Evaluation of module code is a *huge* no-no; we
> > should be able to run documentation tools on unknown (=3D=3D untrusted)=
> > 
> > code.
> 
> If this is indeed agreed upon by everyone (even if only for the reason 
> that Our Benevolent Dictator Says So) it makes the discussion quite a
> bit simpler, as various of the things people proposed over the last
> few days failed the no-execution test.

There's a safe execution test.
Using eval(docstring, safe_functions, safe_functions)

safe_functions are simply methods of a class, and classes that
do not anything at all but gathering info. Hold a minute(or a day)
and you'll have a cohesive response no-parse-tree-like.

Regards/Saludos
Manolo
www.ctv.es/USERS/irmina    /TeEncontreX.html   /texpython.htm
 /SantisimaInquisicion/index.html 

  You will always find something in the last place you look.

From tim_one@email.msn.com  Fri Dec  3 06:14:32 1999
From: tim_one@email.msn.com (Tim Peters)
Date: Fri, 3 Dec 1999 01:14:32 -0500
Subject: [Doc-SIG] docstring grammar
In-Reply-To: <14406.63878.954494.808286@weyr.cnri.reston.va.us>
Message-ID: <000901bf3d55$aa048280$3a2d153f@tim>

[Fred L. Drake, Jr.]
>   I think anything that starts ">>>" or "..." should automagically be
> a verbatim-thingy.  This is easy enough to implement and avoids excess
> cruft in docstrings.

Don't jinx it, Fred!  David already caved in on this one -- and seemed very
happy to get his child back .

Note that I was pushing for less and more than that:  did not ask for "..."
to be special (for all I know, it's someone's ellipsis attempt), but did ask
for leading ">>>" to mean that the entire *paragraph* starting there be
treated verbatim(*).  Where "a paragraph" means "all lines up to but not
including the next all-whitespace line, or end of docstring, whichever comes
first".

That's meant to cover pasted-in interactive shell sessions (which my
doctest.py makes heavy use of).  I don't think it's enough to supply
... functionality, if for no other reason than that an empty line
ends it.  But then I'm not sure I've seen
anything else proposed that can span embedded whitespace lines either.

computers-suck-ly y'rs  - tim

(*) "Verbatim" meaning, of course, not verbatim, but with leading whitespace
equal to the leading whitespace of the initial >>> line stripped.

From mal@lemburg.com  Fri Dec  3 10:25:03 1999
From: mal@lemburg.com (M.-A. Lemburg)
Date: Fri, 03 Dec 1999 11:25:03 +0100
Subject: [Doc-SIG] docstring grammar
References: 
 <38426C4A.ACC76AB5@lemburg.com> <14406.52273.417510.947441@weyr.cnri.reston.va.us>
Message-ID: <38479A7F.3852A7B0@lemburg.com>

"Fred L. Drake, Jr." wrote:
> 
> M.-A. Lemburg writes:
>  > This raises the question of whether to parse or evaluate the
>  > loaded module. Evaluation has the benefit of providing "automatic"
> 
>   Guido agrees with me on this one, sorry: it must be extractable from
> the parse tree.  Evaluation of module code is a *huge* no-no; we
> should be able to run documentation tools on unknown (== untrusted)
> code.

That's why gendoc has a switch to be able to either parse
the module or import it. Note that imports are the only way
to extract information from C extensions.

>  > context, i.e. the symbols defined in the global namespace
>  > are exactly the ones relevant for class definitions, etc. It
>  > probably makes contruction of interdepence graphs a lot easier
>  > to write. On the downside you have unwanted side effects due to
>  > loading different modules.
> 
>   No, these will always be hard in Python.  Order of imports can be
> significant, and the set of imports can change over the life of a
> module (imports can be delayed to reduce startup time, or a number of
> alternatives may be supported).

I guess that's the price you have to pay for *automatic*
documentation extraction. 

Perhaps there is a way to only extract class/function/method
__doc__ strings from pyc-modules without actually running them,
since those are really our only targets.

[looks at some module code objects...]

It wasn't obvious from the code objects I just looked at,
but there could be way... after all the information must hidden
somewhere between those bytes codes ;-)

-- 
Marc-Andre Lemburg
______________________________________________________________________
Y2000:                                                    28 days left
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/

From skip@mojam.com (Skip Montanaro)  Fri Dec  3 12:40:28 1999
From: skip@mojam.com (Skip Montanaro) (Skip Montanaro)
Date: Fri, 3 Dec 1999 06:40:28 -0600 (CST)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: <14406.63878.954494.808286@weyr.cnri.reston.va.us>
References: <000b01bf3b07$a465ad40$c92d153f@tim>

 <14406.63878.954494.808286@weyr.cnri.reston.va.us>
Message-ID: <14407.47676.107419.77342@dolphin.mojam.com>

    Fred> I think anything that starts ">>>" or "..." should automagically
    Fred> be a verbatim-thingy.  This is easy enough to implement and avoids
    Fred> excess cruft in docstrings.

I'd amend that to 

    Anything that starts ">>>" or "..." should automagically begin a
    verbatim-thingy that extends to the first whitespace-only or blank line.

Skip Montanaro | http://www.mojam.com/
skip@mojam.com | http://www.musi-cal.com/
847-971-7098   | Python: Programming the way Guido indented...

From guido@CNRI.Reston.VA.US  Fri Dec  3 13:52:02 1999
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Fri, 03 Dec 1999 08:52:02 -0500
Subject: [Doc-SIG] docstring grammar
In-Reply-To: Your message of "Fri, 03 Dec 1999 11:25:03 +0100."
 <38479A7F.3852A7B0@lemburg.com>
References:  <38426C4A.ACC76AB5@lemburg.com> <14406.52273.417510.947441@weyr.cnri.reston.va.us>
 <38479A7F.3852A7B0@lemburg.com>
Message-ID: <199912031352.IAA16391@eric.cnri.reston.va.us>

> That's why gendoc has a switch to be able to either parse
> the module or import it. Note that imports are the only way
> to extract information from C extensions.

Hm...  C extensions are also the most dangerous (in some cases) to
import, and further more this restricts you to generating
documentiation for modules that actually work on your current
platform.  Not a good idea.

> Perhaps there is a way to only extract class/function/method
> __doc__ strings from pyc-modules without actually running them,
> since those are really our only targets.
> 
> [looks at some module code objects...]
> 
> It wasn't obvious from the code objects I just looked at,
> but there could be way... after all the information must hidden
> somewhere between those bytes codes ;-)

Quite easily:

    & python
    Python 1.5.2+ (#929, Aug  4 1999, 13:59:33)  [GCC 2.8.1] on sunos5
    Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
    [startup.py ...]
    [startup.py done]
    >>> import string
    >>> fn = string.__file__
    >>> fn
    '/usr/local/lib/python1.5/string.pyc'
    >>> import marshal
    >>> f = open(fn, "rb")
    >>> f.seek(8)
    >>> c = marshal.load(f)
    >>> f.close()
    >>> c

    >>> dir(c)
    ['co_argcount', 'co_code', 'co_consts', 'co_filename', 'co_firstlineno', 'co_flags', 'co_lnotab', 'co_name', 'co_names', 'co_nlocals', 'co_stacksize', 'co_varnames']
    >>> print c.co_consts[0]
    Common string manipulations.

    Public module variables:

    whitespace -- a string containing all characters considered whitespace
    lowercase -- a string containing all characters considered lowercase letters
    uppercase -- a string containing all characters considered uppercase letters
    letters -- a string containing all characters considered letters
    digits -- a string containing all characters considered decimal digits
    hexdigits -- a string containing all characters considered hexadecimal digits
    octdigits -- a string containing all characters considered octal digits

    >>> codes = filter(lambda x: type(x).__name__ == "code", c.co_consts)
    >>> for x in codes: print x.co_consts[0]; print "-"*20

    lower(s) -> string

	    Return a copy of the string s converted to lowercase.

    --------------------
    (etc.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From paul@prescod.net  Fri Dec  3 14:31:54 1999
From: paul@prescod.net (Paul Prescod)
Date: Fri, 03 Dec 1999 08:31:54 -0600
Subject: [Doc-SIG] docstring grammar
References:  <38426C4A.ACC76AB5@lemburg.com> <14406.52273.417510.947441@weyr.cnri.reston.va.us>
 <38479A7F.3852A7B0@lemburg.com> <199912031352.IAA16391@eric.cnri.reston.va.us>
Message-ID: <3847D45A.9E3F9128@prescod.net>

Guido van Rossum wrote:
> 
> Hm...  C extensions are also the most dangerous (in some cases) to
> import, and further more this restricts you to generating
> documentiation for modules that actually work on your current
> platform.  Not a good idea.

What I'm hearing is that C extensions should just NOT be documented
inline. Perhaps the interperter should look for their docstrings in .pdc
files...to be defined another day!!!
-- 
 Paul Prescod  - ISOGEN Consulting Engineer speaking for himself
"I always wanted to be somebody, but I should have been more
specific." --Lily Tomlin

From guido@CNRI.Reston.VA.US  Fri Dec  3 14:42:08 1999
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Fri, 03 Dec 1999 09:42:08 -0500
Subject: [Doc-SIG] docstring grammar
In-Reply-To: Your message of "Fri, 03 Dec 1999 08:31:54 CST."
 <3847D45A.9E3F9128@prescod.net>
References:  <38426C4A.ACC76AB5@lemburg.com> <14406.52273.417510.947441@weyr.cnri.reston.va.us> <38479A7F.3852A7B0@lemburg.com> <199912031352.IAA16391@eric.cnri.reston.va.us>
 <3847D45A.9E3F9128@prescod.net>
Message-ID: <199912031442.JAA16553@eric.cnri.reston.va.us>

> Guido van Rossum wrote:
> > 
> > Hm...  C extensions are also the most dangerous (in some cases) to
> > import, and further more this restricts you to generating
> > documentiation for modules that actually work on your current
> > platform.  Not a good idea.

Paul Prescod replied:

> What I'm hearing is that C extensions should just NOT be documented
> inline. Perhaps the interperter should look for their docstrings in .pdc
> files...to be defined another day!!!

C extensions should still have doc strings, but these only serve a
function for interactive use, not to generate the separate
documentation.

The separate documentation of C modules could be extracted in a
different way from the source -- but that's a separate problem.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From fdrake@acm.org  Fri Dec  3 16:11:46 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Fri, 3 Dec 1999 11:11:46 -0500 (EST)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: <14407.47676.107419.77342@dolphin.mojam.com>
References: <000b01bf3b07$a465ad40$c92d153f@tim>

 <14406.63878.954494.808286@weyr.cnri.reston.va.us>
 <14407.47676.107419.77342@dolphin.mojam.com>
Message-ID: <14407.60354.613639.517921@weyr.cnri.reston.va.us>

Skip Montanaro writes:
 > I'd amend that to 
 > 
 >     Anything that starts ">>>" or "..." should automagically begin a
 >     verbatim-thingy that extends to the first whitespace-only or blank line.

  Definately; I was thinking about working with the docstrings
pre-chunked into paragraph-like sections based on blank lines.  I'm
*not* proposing new syntax to combine these things!  (At least not
today.  ;)

  -Fred

--
Fred L. Drake, Jr.	     
Corporation for National Research Initiatives

From fdrake@acm.org  Fri Dec  3 19:43:00 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Fri, 3 Dec 1999 14:43:00 -0500 (EST)
Subject: [Doc-SIG] XML Conversion Update
In-Reply-To: <19990829211627.04674CF320@oratrix.oratrix.nl>
References: 
 <3.0.6.32.19990828120056.00972440@gpo.iol.ie>
 <19990829211627.04674CF320@oratrix.oratrix.nl>
Message-ID: <14408.7492.909271.679977@weyr.cnri.reston.va.us>

[Cleaning out my mailbox a bit...]

Jack Jansen writes:
 > While I agree with Sean (and others) that small DTDs are a lot better
 > suited to documenting Python modules there's various
 > standard-formatting things that you'd like to borrow from existing
 > DTDs (emphasis, references to other manuals/sections, footnotes, etc).
 > 
 > Is there a way that that could be done, without dragging in the whole
 > of the (apparently huge and hairy, from the reports here) docbook DTD?

  I looked at the idea of just using the names from DocBook when the
semantics are the same, and decided that was a red herring.  DocBook
is *very* verbose, and using XML is already heavy enough in the markup 
department.  Something simple, like:

    \emph{My} plan was to use \var{bogosity} with a value of \code{1}.

becomes (in XML):

    My plan was to use
    bogosity with a value of 1.

or (in SGML):

    My plan was to use bogosity with a value
    of 1.

  There are some (less important) issues of mapping \code, since it's
used for things which I'm not at sure map to the same thing, and
similar (but less importantly) for \samp.
  Any way around it, *ML markup is very heavy for the occaisional
contributor using vi or emacs (or any other editor that isn't highly
specialized).  Even asking the emacs user to install PSGML is probably 
too much, esp. since that's too much to learn if you don't edit SGML a 
*lot*.
  Which of course gives rise to discussions like the current
discussion of inline markup...

  -Fred

--
Fred L. Drake, Jr.	     
Corporation for National Research Initiatives

From fdrake@acm.org  Fri Dec  3 19:58:25 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Fri, 3 Dec 1999 14:58:25 -0500 (EST)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: <3847D45A.9E3F9128@prescod.net>
References: 
 <38426C4A.ACC76AB5@lemburg.com>
 <14406.52273.417510.947441@weyr.cnri.reston.va.us>
 <38479A7F.3852A7B0@lemburg.com>
 <199912031352.IAA16391@eric.cnri.reston.va.us>
 <3847D45A.9E3F9128@prescod.net>
Message-ID: <14408.8417.577503.767638@weyr.cnri.reston.va.us>

Paul Prescod writes:
 > What I'm hearing is that C extensions should just NOT be documented
 > inline. Perhaps the interperter should look for their docstrings in .pdc
 > files...to be defined another day!!!

  Perhaps a reasonable approach would be to write the documentation as 
a Python source file that offered right interface and some sort of
flag that the classes are really extension types?  This would make it
reasonably easy to work with and explain, and no new markup language
has to be introduced simply to document an extension module.
  It also wouldn't hurt that no additional tools would be needed!  ;-)

  -Fred

--
Fred L. Drake, Jr.	     
Corporation for National Research Initiatives

From fdrake@acm.org  Fri Dec  3 20:01:05 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Fri, 3 Dec 1999 15:01:05 -0500 (EST)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: <38479A7F.3852A7B0@lemburg.com>
References: 
 <38426C4A.ACC76AB5@lemburg.com>
 <14406.52273.417510.947441@weyr.cnri.reston.va.us>
 <38479A7F.3852A7B0@lemburg.com>
Message-ID: <14408.8577.580840.973602@weyr.cnri.reston.va.us>

M.-A. Lemburg writes:
 > Perhaps there is a way to only extract class/function/method
 > __doc__ strings from pyc-modules without actually running them,
 > since those are really our only targets.

  I look forward to your software.  ;-)
  But see also my response to Paul's message about documenting
extension modules.

  -Fred

--
Fred L. Drake, Jr.	     
Corporation for National Research Initiatives

From fdrake@acm.org  Fri Dec  3 21:29:21 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Fri, 3 Dec 1999 16:29:21 -0500 (EST)
Subject: [Doc-SIG] XML Conversion Update
In-Reply-To: <37C70607.607DBF75@prescod.net>
References: <14277.44597.761409.555365@weyr.cnri.reston.va.us>
 <37C6C187.AC3265F@prescod.net>
 <14278.54862.399167.712458@weyr.cnri.reston.va.us>
 <37C70607.607DBF75@prescod.net>
Message-ID: <14408.13873.267250.949429@weyr.cnri.reston.va.us>

Paul Prescod writes:
 > I favor a well-defined subset of SGML using basically the XML features
 > plus end-tag minimization. That can massively cut down on the annoyance
 > factor.
 > 
 > If we ship a simple PyML to DBXML script with Python nobody will
 > complain that we are doing something "non-standard".

  I'd be happy to call it DocBook-with-minimazation and have it really 
be SGML.  Skip the translation to XML.

 > I think that trying to stick carefully to DocBook is probably too much
 > work. We should design a Python-ic variant -- just like we do with APIs.
 > We can use a transformation to "get to" the standard version (just as we
 > use classes/functions for abstraction in APIs)

  At this point, I'm not convinced that DocBook is terribly valuable
for this.  One of the goals remains to make authoring easy, and
DocBook simple has too many long names for things.  That's really
unfortunate, given the rise in the standing of DocBook in the open
source community; I generally consider that a good thing.

  -Fred

--
Fred L. Drake, Jr.	     
Corporation for National Research Initiatives

From jack@oratrix.nl  Fri Dec  3 22:44:49 1999
From: jack@oratrix.nl (Jack Jansen)
Date: Fri, 03 Dec 1999 23:44:49 +0100
Subject: [Doc-SIG] docstring grammar
In-Reply-To: Message by "Fred L. Drake, Jr."  ,
 Fri, 3 Dec 1999 14:58:25 -0500 (EST) , <14408.8417.577503.767638@weyr.cnri.reston.va.us>
Message-ID: <19991203224454.5AB4CD52C5@oratrix.oratrix.nl>

Now I'm getting confused. Confused about what the intention of
docstrings is, that is: as I see them they're quick-reference
documentation to be used by IDE's and class browsers and such to help
you while you're busy hacking. But more and more terms like extraction 
of docstrings and such pops up, so I'm wondering whether people are
envisioning a tangle/weave/web-like system where the full documentation is
included in the source...

And, yes, I know I happily joined in in the latter, with the
multilingual discussion and more, but I think I would definitely
prefer the former scheme, possibly even combined with
the-latter-in-reverse, where the Real Documentation was structured in
such a way that there could be tools that you would feed the
documentation and the source, and that would add docstrings to the source.
--
Jack Jansen             | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack    | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm 

From da@ski.org  Sat Dec  4 00:37:54 1999
From: da@ski.org (David Ascher)
Date: Fri, 3 Dec 1999 16:37:54 -0800 (Pacific Standard Time)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: <000901bf3d55$aa048280$3a2d153f@tim>
Message-ID: 

On Fri, 3 Dec 1999, Tim Peters wrote:

> I don't think it's enough to supply ... functionality, if
> for no other reason than that an empty line ends it.  But then I'm not
> sure I've seen anything else proposed that can span embedded
> whitespace lines either.

I'm not sure what the first sentence means, but re: the second, I just
want to point out that any block which starts w/ a keyword and doesn't
have a block-on-same-line-as-keyword can span lots of embedded whitespace.

Verbatim:

  this is paragraph one

  this is paragraph two

  this is paragraph three

and the body of Verbatim is:

"""
  this is paragraph one

  this is paragraph two

  this is paragraph three
"""

i.e., six lines (counting the first empty line)

From guido@CNRI.Reston.VA.US  Sat Dec  4 00:53:26 1999
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Fri, 03 Dec 1999 19:53:26 -0500
Subject: [Doc-SIG] docstring grammar
In-Reply-To: Your message of "Fri, 03 Dec 1999 14:58:25 EST."
 <14408.8417.577503.767638@weyr.cnri.reston.va.us>
References:  <38426C4A.ACC76AB5@lemburg.com> <14406.52273.417510.947441@weyr.cnri.reston.va.us> <38479A7F.3852A7B0@lemburg.com> <199912031352.IAA16391@eric.cnri.reston.va.us> <3847D45A.9E3F9128@prescod.net>
 <14408.8417.577503.767638@weyr.cnri.reston.va.us>
Message-ID: <199912040053.TAA17428@eric.cnri.reston.va.us>

> Paul Prescod writes:
>  > What I'm hearing is that C extensions should just NOT be documented
>  > inline. Perhaps the interperter should look for their docstrings in .pdc
>  > files...to be defined another day!!!

Fred Drake:
>   Perhaps a reasonable approach would be to write the documentation as 
> a Python source file that offered right interface and some sort of
> flag that the classes are really extension types?  This would make it
> reasonably easy to work with and explain, and no new markup language
> has to be introduced simply to document an extension module.

I experimented with this for the threading module.  Java also does
this for native methods (which always have a stub declaring their
types in a Java class file).

On the downside, it decouples the doc from the source, which was the
primary motivation for docstring extraction, and perhaps writing it
directly in latex/SGML/whatever is easier than writing a dummy Python
module -- it certainly gives more control.  Plus, it's more likely
that specialized editors exist.

>   It also wouldn't hurt that no additional tools would be needed!  ;-)

But probably existing tools would have to be extended to know about
this arrangement, since it's not completely transparent.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From da@ski.org  Sat Dec  4 06:18:54 1999
From: da@ski.org (David Ascher)
Date: Fri, 3 Dec 1999 22:18:54 -0800 (Pacific Standard Time)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: <199912040053.TAA17428@eric.cnri.reston.va.us>
Message-ID: 

On Fri, 3 Dec 1999, Guido van Rossum wrote:

> > Paul Prescod writes:
> >  > What I'm hearing is that C extensions should just NOT be documented
> >  > inline. Perhaps the interperter should look for their docstrings in .pdc
> >  > files...to be defined another day!!!
> 
> Fred Drake:
> >   Perhaps a reasonable approach would be to write the documentation as 
> > a Python source file that offered right interface and some sort of
> > flag that the classes are really extension types?  This would make it
> > reasonably easy to work with and explain, and no new markup language
> > has to be introduced simply to document an extension module.
> 
> I experimented with this for the threading module.  Java also does
> this for native methods (which always have a stub declaring their
> types in a Java class file).

But I think it's crucial that IDEs for example can find out what the
signature for functions defined in extension modules is. So, I'm going to
propose:

  look for module_doc.py
  if found: use it
  else: 
    if user_allows_imports:
       import module and use introspection (e.g. __doc__)
    else:
       tough.

Some users trust the code, some don't.  This way what the tool provides is
up to the user.

  [ ] Enable JavaScript 
  [ ] Enable Cookies
  [ ] Enable Java
  [ ] Enable Python Import

Which brings up the question of whether we want to associate RSA
certificates with modules.  JUST KIDDING!

--da

From mal@lemburg.com  Fri Dec  3 23:57:28 1999
From: mal@lemburg.com (M.-A. Lemburg)
Date: Sat, 04 Dec 1999 00:57:28 +0100
Subject: [Doc-SIG] docstring grammar
References: 
 <38426C4A.ACC76AB5@lemburg.com>
 <14406.52273.417510.947441@weyr.cnri.reston.va.us>
 <38479A7F.3852A7B0@lemburg.com> <14408.8577.580840.973602@weyr.cnri.reston.va.us>
Message-ID: <384858E8.30ED930@lemburg.com>

"Fred L. Drake, Jr." wrote:
> 
> M.-A. Lemburg writes:
>  > Perhaps there is a way to only extract class/function/method
>  > __doc__ strings from pyc-modules without actually running them,
>  > since those are really our only targets.
> 
>   I look forward to your software.  ;-)

Ahh, yes... well... perhaps someday ;-) right now I'm busy with
other things which go one level deeper, namely the Unicode
intergration.

>   But see also my response to Paul's message about documenting
> extension modules.

About the add-on Python module with included doc strings... well,
I don't think that's easily possible since function objects
are immutable AFAIK (and these contain the doc strings). Also
separating the docs and the implementation too far is not
very convenient, e.g. I use C macros to help me with this:

Py_C_Function( mxDateTime_now,
	       "now()\n\n"
	       "Returns a DateTime-object reflecting the current local time."
	       )
{
...
}

I have to add that I maintain my package docs as a completely
separate entities: experience has it that leaving things undocumented
for a while is better if you work on new things in already pretty
wide spread tools such as mxDateTime. Also, the details you
want to include in the "real" docs sometimes don't make sense
within the code or simply don't fit anywhere.

-- 
Marc-Andre Lemburg
______________________________________________________________________
Y2000:                                                    28 days left
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/

From paul@prescod.net  Tue Dec 14 15:44:30 1999
From: paul@prescod.net (Paul Prescod)
Date: Tue, 14 Dec 1999 07:44:30 -0800
Subject: [Doc-SIG] Sorry!
Message-ID: <385665DE.9963174B@prescod.net>

...for spamming you guys all night. I want to make sure that everybody's
concerns get addressed. I'll slow down tonight.

One issue that Greg raised was the difficulty of checking builtin types
(in addition to the hairy parameterized types stuff). I've been thinking
about this and I think that the doc-sig and the types-sig have the same
problem. How do we sniff out the parameters and docstrings for methods
without running dangerous binary code.

I think that Java (and many other languages) has the right plan with
"shadow libraries." The CORBA guys already use IDL as a static library
syntax. I think that we should support both IDL and a strongly-typed
Pythonic syntax. It might work something like this:

def Int foo(a, b): 
	"Foo, defined in module"
	pass

def String foo(c, d, *args ): pass	
	"Foo, defined in module"
	pass

import _foo
locals().update( _foo.__dict__ )

Maybe we would have some kind of a keyword instead of the locals() hack.

-- 
 Paul Prescod  - ISOGEN Consulting Engineer speaking for himself
Three things to be wary of: A new kid in his prime
A man who knows the answers, and code that runs first time
http://www.geezjan.org/humor/computers/threes.html

From da@ski.org  Thu Dec 16 20:59:51 1999
From: da@ski.org (David Ascher)
Date: Thu, 16 Dec 1999 12:59:51 -0800
Subject: [Doc-SIG] update on docstring grammar discussion pending
Message-ID: <0a2901bf4808$7f3e70c0$c355cfc0@ski.org>

I've been super busy in the last couple of weeks, so haven't had a time to
polish up the docstring proposal.  I hope to do so in the next couple of
days, after I review all of the discussion on the sig (and catch up on the
types-sig to see if there's relevant stuff there).

Thank you for your patience,

--david ascher

From fdrake@acm.org  Thu Dec 16 22:14:37 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Thu, 16 Dec 1999 17:14:37 -0500 (EST)
Subject: [Doc-SIG] IPC8 Developers Day
Message-ID: <14425.25677.300452.77111@weyr.cnri.reston.va.us>

  For anyone who hasn't noticed, there will be a Doc-SIG session on
Developer's Dat at IPC8.  The session will be from 10:30 to Noon on
Thursday, 27 January.
  Now, you really *HAVE* to attend IPC8 to avoid missing the
documentation event of the millenium!  Unless of course, you subscribe 
to the media's definition of "millenium", in which case there was
nothing to miss.  ;)
  I hope to have the discussion paper and updated Doc-SIG web pages up 
next week.

  -Fred

--
Fred L. Drake, Jr.	     
Corporation for National Research Initiatives

From friedrich@pythonpros.com  Thu Dec 16 22:52:34 1999
From: friedrich@pythonpros.com (Robin Friedrich)
Date: Thu, 16 Dec 1999 16:52:34 -0600
Subject: [Doc-SIG] IPC8 Developers Day
References: <14425.25677.300452.77111@weyr.cnri.reston.va.us>
Message-ID: <00f601bf4818$5029e0c0$f25728a1@UNITEDSPACEALLIANCE.COM>

>   For anyone who hasn't noticed, there will be a Doc-SIG session on
> Developer's Dat at IPC8.  The session will be from 10:30 to Noon on
> Thursday, 27 January.
>   Now, you really *HAVE* to attend IPC8 to avoid missing the
> documentation event of the millenium!  Unless of course, you subscribe
> to the media's definition of "millenium", in which case there was
> nothing to miss.  ;)

Millennium is spelled with two n's. Regardless I'll be there to heckle Fred.
:-)
Should be great fun!!!

From skip@mojam.com (Skip Montanaro)  Fri Dec 17 17:26:46 1999
From: skip@mojam.com (Skip Montanaro) (Skip Montanaro)
Date: Fri, 17 Dec 1999 11:26:46 -0600 (CST)
Subject: [Doc-SIG] doc-sig/types-sig clash?
Message-ID: <14426.29270.837380.905106@dolphin.mojam.com>

One of the proposals regarding typing seems to be inserting stuff into doc
strings.  It is perhaps worth noting for those who don't subscribe to the
doc-sig that that bunch of ne'er-do-wells (I mean esteemed colleagues) also
has their eyes on the doc string (imagine that!).

Just raising a small flag to make sure people don't assume the doc string is
their private sandbox.  Apologies if this has already been addressed.  I'm
still wading through all the recent Python activity.

Skip

From fdrake@acm.org  Mon Dec 20 20:29:24 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Mon, 20 Dec 1999 15:29:24 -0500 (EST)
Subject: [Doc-SIG] Documentation systems for other languages
Message-ID: <14430.37284.507421.983098@weyr.cnri.reston.va.us>

  I've started a list of documentation systems used for other
languages:

	http://www.python.org/sigs/doc-sig/otherlangs.html

  The list isn't complete, and that's not really my intention.  If
there are other systems that are interesting, I'll be glad to add them 
to the list.  So far, all of these are for source-embedded docs, but
I'd love to find examples that aren't.
  If someone can send me a meaningful link to POD information, and an
example, I'll add it.  (It's on the list, just not hyperlinked.)
  This is *not* intended to be a list of general-purpose systems, so
things like DocBook and LinuxDoc are not listed.

  -Fred

--
Fred L. Drake, Jr.	     
Corporation for National Research Initiatives

From akuchlin@mems-exchange.org  Mon Dec 20 23:11:10 1999
From: akuchlin@mems-exchange.org (Andrew M. Kuchling)
Date: Mon, 20 Dec 1999 18:11:10 -0500 (EST)
Subject: [Doc-SIG] Documentation systems for other languages
In-Reply-To: <14430.37284.507421.983098@weyr.cnri.reston.va.us>
References: <14430.37284.507421.983098@weyr.cnri.reston.va.us>
Message-ID: <14430.46990.733137.457435@amarok.cnri.reston.va.us>

Fred L. Drake, Jr. writes:
>  If someone can send me a meaningful link to POD information, and an
>example, I'll add it.  (It's on the list, just not hyperlinked.)

For POD, what about http://www.perl.com/pub/doc/manual/html/pod/perlpod.html ?

I suppose literate programming systems like CWEB are too far afield,
since they usually need a general-purpose system like TeX.

-- 
A.M. Kuchling			http://starship.python.net/crew/amk/
    "If you had stayed with us, we could have given you life until death."
    "Don't I get that anyway?"
    -- Stheno and Lyta Hall, in SANDMAN #61: "The Kindly Ones:5"

From jdnier@execpc.com  Tue Dec 21 13:56:57 1999
From: jdnier@execpc.com (David Niergarth)
Date: Tue, 21 Dec 1999 07:56:57 -0600
Subject: [Doc-SIG] Documentation systems for other languages
References: <14430.37284.507421.983098@weyr.cnri.reston.va.us> <200004051129.MAA20636@lsls4p.lsl>
Message-ID: <004901bf4bbb$40964260$e46ccfa9@ep3>

Edward Welbourne wrote:

> Occam, specifically the *folding* idiom, as applied to docs in this
> case.  [Imagine, in a paper page on which your code is written, making
> sideways folds in the paper below each line of text; then you can hide a
> chunk of the page from view by concertina-ing up that bit of the page.]

I saw something similar at a presentation given by Peter Pierrou from
Excosoft at the recent Markup Technologies '99 conference. The prenentation
was titled Literate Programming in XML: Combining source code and
documentation. Apparently Excosoft uses this tool for all their internal
development work (including Python and 7-8 other languages). It's based on
their Documentor SGML/XML editor, which unfortunately is a commercial tool.
The paper is available at

ftp://www.excosoft.se/pub/seminars/litprog.pdf

in case anyone's interested.

This is a little off the topic of Fred's original question but maybe some
future free IDE for Python might incorporate the folding concept.

--David Niergarth

From fdrake@acm.org  Tue Dec 21 15:15:42 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 21 Dec 1999 10:15:42 -0500 (EST)
Subject: [Doc-SIG] Documentation systems for other languages
In-Reply-To: <14430.46990.733137.457435@amarok.cnri.reston.va.us>
References: <14430.37284.507421.983098@weyr.cnri.reston.va.us>
 <14430.46990.733137.457435@amarok.cnri.reston.va.us>
Message-ID: <14431.39326.804912.934762@weyr.cnri.reston.va.us>

Andrew M. Kuchling writes:
 > For POD, what about http://www.perl.com/pub/doc/manual/html/pod/perlpod.html ?

  Works for me!  I'd been using the installed man page, as I don't
know how to find things in the Perl world.  I've updated the page.

 > I suppose literate programming systems like CWEB are too far afield,
 > since they usually need a general-purpose system like TeX.

  Processing requirements are irrelevant for our (immediate) purposes,
I think.  This does indicate an entire category of tools that I didn't 
include a category for; I've added an acknowledgement of their
existance, but I'd rather not create a new list.  If anyone has a link 
to a fairly well maintained list of LitProg tools, I'll be glad to
include the link on the page.
  Thanks!

  -Fred

--
Fred L. Drake, Jr.	     
Corporation for National Research Initiatives

From fdrake@acm.org  Tue Dec 21 22:30:34 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 21 Dec 1999 17:30:34 -0500 (EST)
Subject: [Doc-SIG] Documentation systems for other languages
In-Reply-To: <004901bf4bbb$40964260$e46ccfa9@ep3>
References: <14430.37284.507421.983098@weyr.cnri.reston.va.us>
 <200004051129.MAA20636@lsls4p.lsl>
 <004901bf4bbb$40964260$e46ccfa9@ep3>
Message-ID: <14431.65418.143653.630323@weyr.cnri.reston.va.us>

David Niergarth writes:
 > This is a little off the topic of Fred's original question but maybe some
 > future free IDE for Python might incorporate the folding concept.

  Interesting snippets are always interesting!
  A number of people have proposed adding folding support to IDLE, but 
I don't know if anyone is actually working on it.  Heck, for all I
know, it might be in there already!

  -Fred

--
Fred L. Drake, Jr.	  
Corporation for National Research Initiatives

From fdrake@acm.org  Tue Dec 21 22:32:46 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 21 Dec 1999 17:32:46 -0500 (EST)
Subject: [Doc-SIG] Documentation systems for other languages
In-Reply-To: <200004051129.MAA20636@lsls4p.lsl>
References: <14430.37284.507421.983098@weyr.cnri.reston.va.us>
 <200004051129.MAA20636@lsls4p.lsl>
Message-ID: <14432.14.215063.798808@weyr.cnri.reston.va.us>

Edward Welbourne writes:
 > Occam, specifically the *folding* idiom, as applied to docs in this
 > case.  [Imagine, in a paper page on which your code is written, making

  Is there a pointer to something that describes this in the context
of Occam?  I've heard about folding as a general approach to editing
sources and structured documentation, but had not heard of it as
specific to a language.
  Or do I misunderstand?

  -Fred

--
Fred L. Drake, Jr.	  
Corporation for National Research Initiatives

From guido@CNRI.Reston.VA.US  Tue Dec 21 22:35:27 1999
From: guido@CNRI.Reston.VA.US (Guido van Rossum)
Date: Tue, 21 Dec 1999 17:35:27 -0500
Subject: [Doc-SIG] Documentation systems for other languages
In-Reply-To: Your message of "Tue, 21 Dec 1999 17:30:34 EST."
 <14431.65418.143653.630323@weyr.cnri.reston.va.us>
References: <14430.37284.507421.983098@weyr.cnri.reston.va.us> <200004051129.MAA20636@lsls4p.lsl> <004901bf4bbb$40964260$e46ccfa9@ep3>
 <14431.65418.143653.630323@weyr.cnri.reston.va.us>
Message-ID: <199912212235.RAA13593@eric.cnri.reston.va.us>

> From: "Fred L. Drake, Jr." 
> 
> David Niergarth writes:
>  > This is a little off the topic of Fred's original question but maybe some
>  > future free IDE for Python might incorporate the folding concept.
> 
>   Interesting snippets are always interesting!
>   A number of people have proposed adding folding support to IDLE, but 
> I don't know if anyone is actually working on it.  Heck, for all I
> know, it might be in there already!

No, it's not been done yet.  It might be tricky, given that the Tk
text widget we're using doesn't support such a concept -- we'd have to
delete the folded text from the buffer and trap keystrokes.

I agree though that it's a useful feature.  (Among many others.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From fdrake@acm.org  Tue Dec 21 22:39:12 1999
From: fdrake@acm.org (Fred L. Drake, Jr.)
Date: Tue, 21 Dec 1999 17:39:12 -0500 (EST)
Subject: [Doc-SIG] Documentation systems for other languages
In-Reply-To: <199912212235.RAA13593@eric.cnri.reston.va.us>
References: <14430.37284.507421.983098@weyr.cnri.reston.va.us>
 <200004051129.MAA20636@lsls4p.lsl>
 <004901bf4bbb$40964260$e46ccfa9@ep3>
 <14431.65418.143653.630323@weyr.cnri.reston.va.us>
 <199912212235.RAA13593@eric.cnri.reston.va.us>
Message-ID: <14432.400.950335.249824@weyr.cnri.reston.va.us>

Guido van Rossum writes:
 > No, it's not been done yet.  It might be tricky, given that the Tk
 > text widget we're using doesn't support such a concept -- we'd have to
 > delete the folded text from the buffer and trap keystrokes.

  Which of course complicates the colorizing code; I seem to recall
that it works directly with the text widget's buffer rather than an
internal representation.
  Ahh, the breakage!  ;)

  -Fred

--
Fred L. Drake, Jr.	  
Corporation for National Research Initiatives

From Manuel Gutierrez Algaba   Fri Dec 24 13:57:10 1999
From: Manuel Gutierrez Algaba  (Manuel Gutierrez Algaba)
Date: Fri, 24 Dec 1999 13:57:10 +0000 (GMT)
Subject: [Doc-SIG] Editorial for freshmeat
Message-ID: 

The need of documentation. Open Source phase II

As more and more programms are made and as more and more
information is available in USENET, free electronic books, 
tutorials, and so on, the bigger the problems to use/handle
that information/code
are. 

The main advantage of Open Source stuff: 
- Use and reuse of information/code (freely)
should be or could be greatly improved using wise
(marking) documentation.

Almost nearly every programm ( even the simplest one) is 
divided into several parts, each part is related to something
very specific (usually) :
- sockets
- GUI 
- formatting into html/TeX
- i18n
....

So every programm does a 'main' task by resolving many other
small common tasks. 

The developer of programms who learns by reading other's code
knows how to find those common tasks. Well, sometimes. Firstly,
because the code is not always well structured enough. Secondly
because he lacks the time , patience to browse in the middle of
so much code. 

The user sometimes has a very hard time trying to setup, identify
the components of the programm or simply learn how to use it. This
is mainly because of a 'monolithic' info. I'll argue against this
"man-page-hacker-like" attitude later. 

So we have a Open Source programm that can't be fully reused/used
by the lack of information or by a deficient structuration. 
This is very, very common and it'll become more painfully common
in the future. 

Most developers are happy enough when their code runs smoothly (or
simply runs), when they've managed to arrange the code so it 
compiles and does something. In fact, most developers struggle
non stop with buggy code, adding new features and so on, they
pay little attention to the fact that most of the C/python/perl/java 
modules solve relatively "general" problems (KDE programming,
sockets, i18n, special formatting). Of course, if they're good
enough they'll put most of it into libraries and objects, but then
 those libraries and objects will remain "hidden" behind the 
"main tasks" of the programm. I wonder:
If you spend three days solving small problems, why don't you 
spend ten minutes marking those solved small problems (GUI, i18n...)
somehow.

This way your programm would be useful twice: by itself, by the
objects and libraries it keeps. 

Is this marking difficult? How can I do this marking?

One answer is using XML, sgml or something like that. 

[ MUCH MORE TO BE SAID STILL]

Is this subject/style appropiated ? Is it worth that I finish
this article to be published ? It's unfinished because I'd like
the assurance of it's worth the effort.

I'm the author of some free programms(some in freshmeat),
 see URL above.

Regards/Saludos
Manolo
www.ctv.es/USERS/irmina    /TeEncontreX.html   /texpython.htm
 /SantisimaInquisicion/index.html 

  Man's reach must exceed his grasp, for why else the heavens?

From irmina@ctv.es  Sun Dec 26 15:37:09 1999
From: irmina@ctv.es (Manuel Gutierrez Algaba)
Date: Sun, 26 Dec 1999 15:37:09 +0000 (GMT)
Subject: [Doc-SIG] Mandoc (I give up temporarily)
Message-ID: 

Dear folks, 
I've been working in a eval-oriented doc application,
due to the lack of time I can't keep on building it, 
so I think the best I can do is release what I've done
so It helps to the discussion.

You can find it at:
http://www.ctv.es/USERS/irmina/mandoc/
In that directory the relevant files are:
mandoc.py (specially this one)
restrict.py
logger.py
examples.py

mainly.
mandoc.tex is rather "green" and the rest doesn't care.

Basically the underlying idea is to eval docstrings and supply
and ever-extensible-flexible set of keywords and contructs 
, and paying a bit of so much flexibility in rendering/formatting
time. 

Enjoy it.
It pretended to generate linux-sgml , so that we could have
.html, .info. , .tex, very easily.

Regards/Saludos
Manolo
www.ctv.es/USERS/irmina    /TeEncontreX.html   /texpython.htm
 /SantisimaInquisicion/index.html 

  If little green men land in your back yard, hide any little green women you've got in the house. -- Mike Harding, "The Armchair Anarchist's Almanac"

From da@ski.org  Thu Dec  2 20:36:31 1999
From: da@ski.org (David Ascher)
Date: Thu, 2 Dec 1999 12:36:31 -0800 (Pacific Standard Time)
Subject: [Doc-SIG] docstring grammar
In-Reply-To: <14406.53828.589607.28770@weyr.cnri.reston.va.us>
Message-ID: 

>   I maintain that the parse tree is the simple route to getting a
> reliable tool, and I am working on coding one.  Neat, huh?

Indeed.  We've slyly conned someone who *does* know about parsing to do
the tool. =)

I love this trick!

--da