From fdrake@acm.org Mon Apr 1 16:37:06 2002 From: fdrake@acm.org (Fred L. Drake) Date: Mon, 1 Apr 2002 11:37:06 -0500 (EST) Subject: [Doc-SIG] [development doc updates] Message-ID: <20020401163706.D36FF18ED64@grendel.zope.com> The development version of the documentation has been updated: http://python.sourceforge.net/maint-docs/ Documentation being prepared for Python 2.2.1. This version includes Andrew Kuchling's "What's New in Python 2.2" alongside the original documents. From fdrake@acm.org Mon Apr 1 20:33:00 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Mon, 1 Apr 2002 15:33:00 -0500 Subject: [Doc-SIG] New home for development version of docs Message-ID: <15528.50172.390695.897521@grendel.zope.com> The "in-developement" version of the documentation, previously found at python.sourceforge.net, is now available from the development area on python.org: http://www.python.org/dev/doc/ -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From fdrake@acm.org Mon Apr 1 22:06:24 2002 From: fdrake@acm.org (Fred L. Drake) Date: Mon, 1 Apr 2002 17:06:24 -0500 (EST) Subject: [Doc-SIG] [development doc updates] Message-ID: <20020401220624.423C818EAD1@grendel.zope.com> The development version of the documentation has been updated: http://www.python.org/dev/doc/devel/ Updated documentation on lexical scoping in the language reference. From fdrake@acm.org Wed Apr 3 22:54:42 2002 From: fdrake@acm.org (Fred L. Drake) Date: Wed, 3 Apr 2002 17:54:42 -0500 (EST) Subject: [Doc-SIG] [development doc updates] Message-ID: <20020403225442.0343418EAD1@grendel.zope.com> The development version of the documentation has been updated: http://www.python.org/dev/doc/devel/ Updated to include whatever Guido did about bool(). From fdrake@acm.org Thu Apr 4 04:32:19 2002 From: fdrake@acm.org (Fred L. Drake) Date: Wed, 3 Apr 2002 23:32:19 -0500 (EST) Subject: [Doc-SIG] [maintenance doc updates] Message-ID: <20020404043219.912DC28696@beowolf.fdrake.net> The maintenance version of the documentation has been updated: http://python.sourceforge.net/maint21-docs/ Preliminary documentation for Python 2.1.3. From jafo-python-dev@tummy.com Fri Apr 5 00:47:52 2002 From: jafo-python-dev@tummy.com (Sean Reifschneider) Date: Thu, 4 Apr 2002 17:47:52 -0700 Subject: [Doc-SIG] Re: [Python-Dev] [development doc updates] In-Reply-To: <20020404225927.530B318EAD1@grendel.zope.com>; from fdrake@acm.org on Thu, Apr 04, 2002 at 05:59:27PM -0500 References: <20020404225927.530B318EAD1@grendel.zope.com> Message-ID: <20020404174752.M16962@tummy.com> On Thu, Apr 04, 2002 at 05:59:27PM -0500, Fred L. Drake wrote: >The development version of the documentation has been updated: On the topic of dev documentation... What happened to the info distribution? I had info and HTML in the 2.2 RPM, but only the HTML were around for 2.2.1c2 when I tried to pick it up the other night... Is there going to be an info release for the 2.2.1 final, or should I just leave it as HTML-only? Sean -- Tragedy is when I cut my finger. Comedy is when you fall into an open sewer and die. -- Mel Brooks Sean Reifschneider, Inimitably Superfluous tummy.com - Linux Consulting since 1995. Qmail, KRUD, Firewalls, Python From fdrake@acm.org Fri Apr 5 01:07:31 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Thu, 4 Apr 2002 20:07:31 -0500 Subject: [Doc-SIG] Re: [Python-Dev] [development doc updates] In-Reply-To: <20020404174752.M16962@tummy.com> References: <20020404225927.530B318EAD1@grendel.zope.com> <20020404174752.M16962@tummy.com> Message-ID: <15532.63699.122156.945015@grendel.zope.com> [Let's take this to the Doc-SIG; all followups there, please!] Sean Reifschneider writes: > On the topic of dev documentation... What happened to the info > distribution? I had info and HTML in the 2.2 RPM, but only the HTML were > around for 2.2.1c2 when I tried to pick it up the other night... Is there > going to be an info release for the 2.2.1 final, or should I just leave it > as HTML-only? The GNU info files have been contributed, but a patch has just been submitted that I'll try to get to in time. In the past, we've run into problems with elisp library incompatibilies between emacs and xemacs. We'll see how the proposed patch holds up. Even if we aren't able to get it working, someone will probably contribute a version. This is how all recent GNU info versions have been prepared (like for the past four years or so). -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From goodger@users.sourceforge.net Fri Apr 5 04:33:31 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Thu, 04 Apr 2002 23:33:31 -0500 Subject: [Doc-SIG] Re: PEP 287: reStructuredText Standard Docstring Format In-Reply-To: References: <200204042338.XAA12847@crown.off.ekorp.com> <01c701c1dc35$5a2312f0$02010a0a@suxlap> Message-ID: Thank you to everyone who has commented on PEP 287, be they in favor, well-meaning but opposed, or ill-wishing troll ;-) [1]_. I have been otherwise busy and haven't been able to keep up with replies, but I have read and kept a record of all the posts so far (and will, for posts yet to come) for consideration and reference when revising the PEP. I may not be able to offer timely replies to all the posts; ultimately the revised PEP will serve as an umbrella reply. .. [1] Actually, the posts opposed and even the trolls are often the most useful, since they point out weaknesses and help strengthen the arguments. The number of people posting *in favor* of the PEP was roughly the same as the number opposed, but supporters tend to be silent while protesters tend to be quite vociferous and relentless. Over the weeks to come, I will be revising PEP 287 and related PEPs 256-258, merging the reStructuredText_ and DPS_ projects into the renamed Docutils_ project, and producing a "road map" describing the big picture. Further suggestions for improvement are welcome. .. _reStructuredText: http://structuredtext.sourceforge.net/ .. _DPS: http://docstring.sourceforge.net/ .. _Docutils: http://docutils.sourceforge.net/ A special thanks must go to Richard Jones for taking up arms (or keyboard) in my stead, and for taking the initiative to start work on `A ReStructuredText Primer`_. In hindsight it was much needed, and it's coming along nicely. Thanks Richard! .. _A ReStructuredText Primer: http://mechanicalcat.net/tech/quickstart.html -- David Goodger goodger@users.sourceforge.net Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From tony@lsl.co.uk Fri Apr 5 08:27:56 2002 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Fri, 5 Apr 2002 09:27:56 +0100 Subject: [Doc-SIG] Re: PEP 287: reStructuredText Standard Docstring Format In-Reply-To: Message-ID: <00cb01c1dc7b$c9a92c40$545aa8c0@lslp862.int.lsl.co.uk> [python-dev cut from reply list] David Goodger wrote: > A special thanks must go to Richard Jones for taking up arms > (or keyboard) in my stead, and for taking the initiative to > start work on `A ReStructuredText Primer`_. Ooh, ooh, neat stuff! > In hindsight it was much needed, and it's coming > along nicely. Thanks Richard! Oh, it was obvious it was needed, but it needed someone to do it! And the initial part of the document up now is good stuff. I must admit I like the term "Relaxed text" - should we consider a name change? (sowing trouble for the sake of it!) Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ Give a pedant an inch and they'll take 25.4mm (once they've established you're talking a post-1959 inch, of course) My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From fdrake@acm.org Fri Apr 5 18:30:58 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Fri, 5 Apr 2002 13:30:58 -0500 Subject: [Doc-SIG] Re: [Python-Dev] Searching Python docs In-Reply-To: <03e401c1dcce$bdb7fd50$e000a8c0@thomasnotebook> References: <3CAC2F2B.1950AD8F@lemburg.com> <018e01c1dbee$51b1c9f0$e000a8c0@thomasnotebook> <15532.48440.566653.321044@magrathea.basistech.com> <025701c1dcb3$037af110$e000a8c0@thomasnotebook> <3CADD457.4D821311@lemburg.com> <03e401c1dcce$bdb7fd50$e000a8c0@thomasnotebook> Message-ID: <15533.60770.80493.19497@grendel.zope.com> [This is cross-posted; followups to the Doc-SIG only, please!] Thomas Heller writes: > Hm, in the good old days this was the only way ;-) > because the chances where at most 50% that this stuff > was undocumented. I think a lot of us here remember when that 50% was less than 5%, and there was no C API manual. ;-) I still remember seeing the checkin message for api.tex 1.1. > But the docs have improved in the meantime, now you > could also use *them*. Which brings me to the issue of a decent index. (What doesn't, eventually?) We currently have two documents that discuss the C API, though they take (mostly) different approaches. The API reference has a tolerable index (the symbols are there, at least), but the Extending & Embedding document doesn't have enough index entries to be very useful (so little we don't even include an index), even though it contains some strongly overlapping information. I *think* it might be a good idea to merge the two documents, but I'm not certain I really like that. There is a strong need to add good index entries to the existing Extending & Embedding document at the very least, and a combined index would be very valuable. This is something that's been requested for the entire set of documents on many occaisions, and would be very handy. (Especially if we provided a better interface for it!) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From thomas.heller@ion-tof.com Fri Apr 5 18:53:56 2002 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 5 Apr 2002 20:53:56 +0200 Subject: [Doc-SIG] Re: [Python-Dev] Searching Python docs References: <3CAC2F2B.1950AD8F@lemburg.com><018e01c1dbee$51b1c9f0$e000a8c0@thomasnotebook><15532.48440.566653.321044@magrathea.basistech.com><025701c1dcb3$037af110$e000a8c0@thomasnotebook><3CADD457.4D821311@lemburg.com><03e401c1dcce$bdb7fd50$e000a8c0@thomasnotebook> <15533.60770.80493.19497@grendel.zope.com> Message-ID: <043001c1dcd3$3d1d8700$e000a8c0@thomasnotebook> From: "Fred L. Drake, Jr." > > [This is cross-posted; followups to the Doc-SIG only, please!] (Great, the next mailing list for me to subscribe :-) > > Which brings me to the issue of a decent index. (What doesn't, > eventually?) > > We currently have two documents that discuss the C API, though they > take (mostly) different approaches. The API reference has a tolerable > index (the symbols are there, at least), but the Extending & Embedding > document doesn't have enough index entries to be very useful (so > little we don't even include an index), even though it contains some > strongly overlapping information. > > I *think* it might be a good idea to merge the two documents, but I'm > not certain I really like that. There is a strong need to add good > index entries to the existing Extending & Embedding document at the > very least, and a combined index would be very valuable. This is > something that's been requested for the entire set of documents on > many occaisions, and would be very handy. (Especially if we provided > a better interface for it!) IMO an index is not needed for the tutorial style manuals, and Extending & Embedding is one of them. I'm fine with the existing indexes for the api, lib, and ref manuals. They contain the complete entries which I have to look up from time to time. My experience is as follows: I used to use the HTMLHelp version of the Python docs most of the time, mainly for context-sensitive help from the editor: Put the cursor on the 'PyObject_GetItemString' word, press F1, and look at the docs for this function. This is a use case which is very common for me, and I have not yet been able to drive a web search engine to do that for me. The problem is that I always have to locate and download (or even build) the HTMLHelp version separately. Therefore I wrote my pyhelp script which does the same from the command line, and now also as CGI script. Again, in case anyone wants a to look at a preview: http://starship.python.net/crew/theller/cgi-bin/pyhelp.cgi (Any comments/suggestions/flames greatly appreciated). Enter PyDict_ and press the search button: I find it very useful. Note that this script (which is in an early stage) finds the index entries from the api, lib and ref manuals. Thomas From guido@python.org Fri Apr 5 19:17:03 2002 From: guido@python.org (Guido van Rossum) Date: Fri, 05 Apr 2002 14:17:03 -0500 Subject: [Doc-SIG] Re: [Python-Dev] Searching Python docs In-Reply-To: Your message of "Fri, 05 Apr 2002 13:30:58 EST." <15533.60770.80493.19497@grendel.zope.com> References: <3CAC2F2B.1950AD8F@lemburg.com> <018e01c1dbee$51b1c9f0$e000a8c0@thomasnotebook> <15532.48440.566653.321044@magrathea.basistech.com> <025701c1dcb3$037af110$e000a8c0@thomasnotebook> <3CADD457.4D821311@lemburg.com> <03e401c1dcce$bdb7fd50$e000a8c0@thomasnotebook> <15533.60770.80493.19497@grendel.zope.com> Message-ID: <200204051917.g35JH3w15377@pcp742651pcs.reston01.va.comcast.net> > We currently have two documents that discuss the C API, though they > take (mostly) different approaches. The API reference has a tolerable > index (the symbols are there, at least), but the Extending & Embedding > document doesn't have enough index entries to be very useful (so > little we don't even include an index), even though it contains some > strongly overlapping information. > > I *think* it might be a good idea to merge the two documents, but I'm > not certain I really like that. There is a strong need to add good > index entries to the existing Extending & Embedding document at the > very least, and a combined index would be very valuable. This is > something that's been requested for the entire set of documents on > many occaisions, and would be very handy. (Especially if we provided > a better interface for it!) The E&E document should be turned into a tutorial, with a bunch of platform specific things about building. The API reference documentation (in particular PyArg_ArgParse and Py_BuildValue) should be moved from the E&E document to the API document. --Guido van Rossum (home page: http://www.python.org/~guido/) From goodger@users.sourceforge.net Tue Apr 9 01:20:09 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Mon, 08 Apr 2002 20:20:09 -0400 Subject: [Doc-SIG] Re: PEP 287: reStructuredText Standard Docstring Format In-Reply-To: <00cb01c1dc7b$c9a92c40$545aa8c0@lslp862.int.lsl.co.uk> Message-ID: Tony J Ibbs (Tibs) wrote: > I must admit I like the term "Relaxed text" - should we consider a > name change? We could always add "relaxed" to the list of meanings for the "re" in "reStructuredText", and pretend that we've hijacked the time machine. > (sowing trouble for the sake of it!) What else is new? :> -- David Goodger goodger@users.sourceforge.net Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From fredrik@pythonware.com Tue Apr 9 08:12:30 2002 From: fredrik@pythonware.com (Fredrik Lundh) Date: Tue, 9 Apr 2002 09:12:30 +0200 Subject: [Doc-SIG] ann: xmltoys 1.0a4, featuring pythondoc Message-ID: <00e101c1df95$eeb672a0$ced241d5@hagrid> I just released xmltoys 1.0a4, which includes an updated version of the javadoc-like pythondoc document parser: http://effbot.org/downloads/index.cgi/xmltoys-1.0a4-20020407.zip (click on download to get the code) -- about xmltoys pythondoc this tool uses introspection to find functions, classes, and methods in a module, and source code scanning to extract javadoc-style comments. the description part is written in HTML, and converted to XHTML fragments by the parser. additional information is extracted from the source code, and from javadoc tags in the comment. the result is stored in an XML infoset. the tool comes with a relatively simple HTML converter; a docbook converter is in the works. you can of course also run the result through your favourite XML processor. more information can be found here: http://effbot.org/guides/pythondoc.htm http://java.sun.com/j2se/javadoc/ http://java.sun.com/j2se/1.4/docs/tooldocs/solaris/javadoc.html sample output (showing off some rather obvious bugs): http://effbot.org/doc/ -- about the 1.0a4 release this version has been tested with itself, PIL, and a couple of other rather large libraries. unlike the first release, it can also parse the python standard library without getting stuck on None objects (but for obvious reasons, it doesn't generate any output for the standard library... it uses the new HTMLParser module, so the current version requires Python 2.2. I plan to get rid of that dependency for the next release. enjoy /F From fdrake@acm.org Tue Apr 9 17:30:32 2002 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 9 Apr 2002 12:30:32 -0400 Subject: [Doc-SIG] 2.2.1 docs Message-ID: <15539.5928.855738.194157@grendel.zope.com> The Python 2.2.1 docs have now been published on python.org, and are listed as the "current" documentation. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From vdesmedt@cso.ulb.ac.be Fri Apr 12 08:56:26 2002 From: vdesmedt@cso.ulb.ac.be (Vivian De Smedt) Date: Fri, 12 Apr 2002 09:56:26 +0200 Subject: [Doc-SIG] State of xml documentation standart project. Message-ID: <5.1.0.14.2.20020412094652.00a6adc0@cso.ulb.ac.be> Dear all, I have write some module documentation using LaTeX according to the documentation standart of Python but I didn't succeed in converting them into html pages using Latex2Html on my windows platform. Hence I'am interested in the xml version of the documentation standart and the existing tools that already exist to convert from: latex to xml xml to latex xml to html If they do not exist I will be glad to help writing xslt filter to produce both html and LaTeX. I'am sorry not to be able to find what's happening with the xml migration project and I hope not bothering you with question I should be able to answer alone. Vivian. From brian@sweetapp.com Fri Apr 12 17:56:15 2002 From: brian@sweetapp.com (Brian Quinlan) Date: Fri, 12 Apr 2002 09:56:15 -0700 Subject: [Doc-SIG] State of xml documentation standart project. In-Reply-To: <5.1.0.14.2.20020412094652.00a6adc0@cso.ulb.ac.be> Message-ID: <003b01c1e242$f6371b10$445d4540@Dell2> This is a multi-part message in MIME format. ------=_NextPart_000_003C_01C1E208.49D84310 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Vivian wrote: > I'am sorry not to be able to find what's happening with the xml migration > project and I hope not bothering you with question I should be able to > answer alone. In order to avoid learning LaTeX for my Python extensions, I wrote a simple XSLT stylesheet that converts my made-up XML grammar to html. I've attached the XML, stylesheet and resulting HTML. I too would be winning to help move us towards a standard SGML documentation solution. Cheers, Brian ------=_NextPart_000_003C_01C1E208.49D84310 Content-Type: text/xml; name="PySilverCityDocs.xml" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="PySilverCityDocs.xml"
=20 SilverCity SilverCity=20 --- Multilanguage lexical analysis package

SilverCity is a library that can provide lexical analysis for over 20 = different=20 programming languages. SilverCity is packaged as both a C++ library and = as a=20 Python extension. This documentation applies to the Python = extension.

At this point I'd like to acknoledge that this documentation is = incomplete. Writting isn't a hobby of mine. So if you need any help, just let me know at = <brian@sweetapp.com>.

Scintilla Scintilla is the open-source source editing = component upon which SilverCity is built Python tokenize module Python's built-in lexical scanner for Python source = code. =0D Module Contents find_lexer_module_by_id id The find_lexer_module_by_id function returns a LexerModule object given an integer constant. These constants are defined in the=20 ScintillaConstants module.

A ValueError is raised if the provided constant does not map to a = LexerModule.

 find_lexer_module_by_name name The find_lexer_module_by_name function returns a LexerModule object given it's name.

A ValueError is raised if no = LexerModule has the given name

 WordList keywords Create a new WordList instance.=20 This class is used by the=20 LexerModule class to determine which words should be lexed as keywords.

keywords should be a string containing keywords separated by spaces=20 e.g. "and assert break class..."

WordList objects have no methods. They simply = act as placeholders for=20 language keywords.

 PropertySet properties Create a new PropertySet instance.=20 This class is used by the=20 LexerModule class to determine various lexer options. For example, the 'styling.within.preprocessor' property determines if the C lexer should use a single or multiple lexical states when parsing C preprocessor expressions.

properties should be a dictionary of lexer options.

LexerModule objects

The LexerModule class provides a single method:

get_number_of_wordlists Returns the number of WordLists that the lexer = requires.=20 This is the number of WordLists that must be passed to the tokenize_by_style.

If the number of required WordLists cannot be determined, a = ValueError, is raised

 tokenize_by_style source keywords propertyset func Lexes the provided source code into a list of tokens. Each token = is a dictionary with the following keys:

KeyValue
styleThe lexical style of the token = e.g. 11
textThe text of the token e.g. = 'import'
start_indexThe index in = source where the token begins e.g. 0
end_indexThe index in = source where the token ends e.g. 5
start_columnThe column position = (0-based) where the token begins e.g. 0
end_columnThe column position = (0-based) where the token ends e.g. 5
start_lineThe line position = (0-based) where the token begins e.g. 0
end_lineThe line position (0-based) = where the token ends e.g. 0
=20

=20

source is a string containing the source code. keywords is a list of WordList = instances. The number of WordLists that should be passed = depends on the particular=20 LexerModule. =20 propertyset is a PropertySet instance. The relevant properties are dependant on the particular = LexerModule.

If the optional func argument is used, it must be = a callable object. It will be called, using keyword arguments, for each token found in the = source. Since additional keys may be added in the future, it is recommended that = additional keys be collected e.g. import SilverCity from SilverCity import ScintillaConstants =20 def func(style, text, start_column, start_line, = **other_args):=20 if style =3D=3D ScintillaConstants.SCE_P_WORD and text = =3D=3D 'import': print 'Found an import statement at (%d, %d)' % = (start_line + 1, start_column + 1) =20 =20 keywords =3D = SilverCity.WordList(SilverCity.Keywords.python_keywords) properties =3D SilverCity.PropertySet() lexer =3D = SilverCity.find_lexer_module_by_id(ScintillaConstants.SCLEX_PYTHON) =20 lexer.tokenize_by_style(source_code, keywords, properties, = func)

WordList objects

WordList objects have no methods. They simply act as = placeholders for=20 language keywords.

 PropertySet objects

PropertySet objects have no methods. They act as = dictionaries were the names of the properties are the keys. All keys must be strings, values = will be converted to strings upon assignment i.e. retrieved values will always be strings. There is = no mechanism to delete assigned keys.

Different properties apply to different languages. The following = table is a complete list of properties, the language that they apply to, and their = meanings:

PropertyLanguageValues
asp.default.languageHTML Sets the default language for ASP scripts:
0 =3D> None
1 =3D> JavaScript
2 =3D> VBScript
3 =3D> Python
4 =3D> PHP
5 =3D> XML-based
styling.within.preprocessorC++ Determines if all preprocessor instruments should be lexed = identically or if subexpressions should be given different lexical states:
0 =3D> Same
1 =3D> Different
tab.timmy.whinge.levelPython The property value is a bitfield that causes different types of = incorrect=20 whitespace characters to cause there lexical states to be incremeted by = 64:
0 =3D> no checking
1 =3D> check for correct indenting
2 =3D> check for literal tabs
4 =3D> check for literal spaces used as tabs
8 =3D> check for mixed tabs and spaces

Example PropertySet usage: import SilverCity =20 propset =3D SilverCity.PropertySet({'styling.within.preprocessor' : = 0}) propset['styling.within.preprocessor'] =3D 1 # changed my mind

Stuff that should be documented better

The ScintillaConstants module contains a list of = lexer identifiers (used by find_lexer_module_by_id) and lexical = states for each LexerModule. You should take a look at this module to = find the states that are useful for your programming language.

The Keywords module contains lists of keywords that = can be used to create WordList objects.

There are also some modules that package = tokenize_by_style into a class that offers a visitor pattern (think SAX). You don't have = to worry about these modules if you don't want to. But, if you do, they are all written in = Python so you can probably muddle through.

Note that some lexer that are supported by Scintilla, are not = supported by=20 SilverCity. This is because I am lazy. Any = contributions are welcome (and should be pretty easy to make).
------=_NextPart_000_003C_01C1E208.49D84310 Content-Type: text/xml; name="doc-template.xsl" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="doc-template.xsl" =09

    SilverCity XXX - Subsections

    See Also:
    class
    ( [ 1">, ] )

    ------=_NextPart_000_003C_01C1E208.49D84310 Content-Type: text/html; name="docs.html" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="docs.html" SilverCity

    SilverCity --- Multilanguage lexical analysis package

    SilverCity is a library that can provide lexical analysis for over 20 = different programming languages. SilverCity is packaged as both a C++ = library=20 and as a Python extension. This documentation applies to the Python=20 extension.

    At this point I'd like to acknoledge that this documentation is = incomplete.=20 Writting isn't a hobby of mine. So if you need any help, just let me = know at=20 <brian@sweetapp.com>.

    Subsections=20
    • Module=20 Contents=20
      • LexerModule=20 objects=20
        • WordList=20 objects=20
          • PropertySet=20 objects=20
            • Stuff=20 that should be documented better=20

              See Also:

              Scintilla=20
              Scintilla is the open-source source editing component upon which=20 SilverCity is built
              Pytho= n=20 tokenize module=20
              Python's built-in lexical scanner for Python source code. =

              Module Contents

              find_lexer_module_by_id=20 (id)=20
              The find_lexer_module_by_id function returns a=20 LexerModule object given an integer constant. These constants = are=20 defined in the ScintillaConstants module.=20

              A ValueError is raised if the provided constant does not = map to a=20 LexerModule.

              find_lexer_module_by_name=20 (name)=20
              The find_lexer_module_by_name function returns a=20 LexerModule object given it's name.=20

              A ValueError is raised if no LexerModule has the = given=20 name

              class WordList=20 ([keywords])=20
              Create a new WordList instance. This class is used by the = LexerModule class to determine which words should be lexed as = keywords.=20

              keywords should be a string containing keywords = separated by=20 spaces e.g. "and assert break class..."

              WordList objects have no methods. They simply act as = placeholders=20 for language keywords.

              class PropertySet=20 ([properties])=20
              Create a new PropertySet instance. This class is used by = the=20 LexerModule class to determine various lexer options. For = example,=20 the 'styling.within.preprocessor' property determines if the C lexer = should=20 use a single or multiple lexical states when parsing C preprocessor=20 expressions.=20

              properties should be a dictionary of lexer options.=20

              LexerModule objects

              The LexerModule class provides a single method:

              get_number_of_wordlists ()=20
              Returns the number of WordLists that the lexer requires. = This is=20 the number of WordLists that must be passed to the = tokenize_by_style.=20

              If the number of required WordLists cannot be determined, a = ValueError, is=20 raised

              tokenize_by_style=20 (source, keywords, = propertyset[,=20 func])=20
              Lexes the provided source code into a list of tokens. Each token = is a=20 dictionary with the following keys:=20

              Key Value
              style The lexical style of the token = e.g.=20 11
              text The text of the token e.g. = 'import'
              start_index The index in source = where the=20 token begins e.g. 0
              end_index The index in source = where the=20 token ends e.g. 5
              start_column The column position (0-based) = where the=20 token begins e.g. 0
              end_column The column position (0-based) = where the=20 token ends e.g. 5
              start_line The line position (0-based) = where the=20 token begins e.g. 0
              end_line The line position (0-based) = where the=20 token ends e.g. 0

              source is a string containing the source code.=20 keywords is a list of WordList instances. The = number of=20 WordLists that should be passed depends on the particular=20 LexerModule. propertyset is a PropertySet = instance. The relevant properties are dependant on the particular=20 LexerModule.

              If the optional func argument is used, it must be a = callable=20 object. It will be called, using keyword arguments, for each token = found in=20 the source. Since additional keys may be added in the future, it is=20 recommended that additional keys be collected e.g.=20 

                          import SilverCity
            from SilverCity import ScintillaConstants
           =20
            def func(style, text, start_column, start_line, =
**other_args):=20
                if style =3D=3D ScintillaConstants.SCE_P_WORD and text =
=3D=3D 'import':
                    print 'Found an import statement at (%d, %d)' % =
(start_line + 1, start_column + 1)
           =20
           =20
            keywords =3D =
SilverCity.WordList(SilverCity.Keywords.python_keywords)
            properties =3D SilverCity.PropertySet()
            lexer =3D =
SilverCity.find_lexer_module_by_id(ScintillaConstants.SCLEX_PYTHON)
           =20
            lexer.tokenize_by_style(source_code, keywords, properties, =
func)

              WordList objects

              WordList objects have no methods. They simply act as = placeholders=20 for language keywords.

              PropertySet objects

              PropertySet objects have no methods. They act as = dictionaries were=20 the names of the properties are the keys. All keys must be strings, = values will=20 be converted to strings upon assignment i.e. retrieved values will = always be=20 strings. There is no mechanism to delete assigned keys.

              Different properties apply to different languages. The following = table is a=20 complete list of properties, the language that they apply to, and their=20 meanings:

              Property Language Values
              asp.default.language HTML Sets the default language for ASP = scripts:
              0 =3D> None
              1 =3D> JavaScript
              2 =3D> = VBScript
              3=20 =3D> Python
              4 =3D> PHP
              5 =3D> = XML-based
              styling.within.preprocessor C++ Determines if all preprocessor = instruments=20 should be lexed identically or if subexpressions should be given = different=20 lexical states:
              0 =3D> Same
              1 =3D> = Different
              tab.timmy.whinge.level Python The property value is a bitfield = that=20 causes different types of incorrect whitespace characters to cause = there=20 lexical states to be incremeted by 64:
              0 =3D> no = checking
              1 =3D>=20 check for correct indenting
              2 =3D> check for literal = tabs
              4 =3D>=20 check for literal spaces used as tabs
              8 =3D> check for mixed = tabs and=20 spaces

              Example PropertySet usage:=20 

                  import SilverCity
   =20
    propset =3D SilverCity.PropertySet({'styling.within.preprocessor' : =
0})
    propset['styling.within.preprocessor'] =3D 1 # changed my mind

              Stuff that should be documented better

              The ScintillaConstants module contains a list of lexer = identifiers=20 (used by find_lexer_module_by_id) and lexical states for each=20 LexerModule. You should take a look at this module to find the = states=20 that are useful for your programming language.

              The Keywords module contains lists of keywords that can be = used to=20 create WordList objects.

              There are also some modules that package tokenize_by_style = into a=20 class that offers a visitor pattern (think SAX). You don't have to worry = about=20 these modules if you don't want to. But, if you do, they are all written = in=20 Python so you can probably muddle through.

              Note that some lexer that are supported by Scintilla, are not = supported by=20 SilverCity. This is because I am lazy. Any contributions are = welcome=20 (and should be pretty easy to make).

              ------=_NextPart_000_003C_01C1E208.49D84310-- From goodger@users.sourceforge.net Fri Apr 19 05:01:50 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Fri, 19 Apr 2002 00:01:50 -0400 Subject: [Doc-SIG] DPS & reStructuredText 0.4 Released Message-ID: Release 0.4 of each of the Docstring Processing System and reStructuredText markup parser projects have been posted to their project sites (URLs in my .sig below). These releases are the final ones for these projects. The two projects will be merged and renamed to "Docutils"; a SourceForge project is being constructed at http://docutils.sourceforge.net/, and a 0.1 release of Docutils will follow shortly. Once the DPS and reStructuredText projects are merged and renamed, they will become inactive. So this is really a contractual obligation non-announcement; you might as well wait a day or three for Docutils 0.1. Quick links to the downloads: - http://prdownloads.sf.net/structuredtext/restructuredtext-0.4.tar.gz - http://prdownloads.sf.net/docstring/dps-0.4.tar.gz There has been a lot of progress since the last release, long ago. The parser is construct-complete, there's a standalone input file reader, and a simple HTML/CSS writer. There are a few simple front-ends in restructuredtext/tools; see its README.txt. There is still much work to be done. If you would like to be added as a developer to the new Docutils project, please let me know (I've already added some people as developers). -- David Goodger goodger@users.sourceforge.net Open-source projects: - Python Docstring Processing System: http://docstring.sourceforge.net - reStructuredText: http://structuredtext.sourceforge.net - The Go Tools Project: http://gotools.sourceforge.net From goodger@users.sourceforge.net Sat Apr 20 20:10:24 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Sat, 20 Apr 2002 15:10:24 -0400 Subject: [Doc-SIG] ANN: Docutils 0.1 Released Message-ID: The purpose of the Docutils project is to create a set of tools for processing plaintext documentation into useful formats, such as HTML, XML, and TeX. It is the subject of PEPs 256 & 258. Docutils currently supports input from standalone files; soon it will process PEPs, and eventually it will support inline documentation from Python modules and packages. Docutils uses the reStructuredText markup, the subject of PEP 287; other markups are possible. It produces simple HTML for CSS; other formats will become available in time. Quick link to the download: http://prdownloads.sf.net/docutils/docutils-0.1.tar.gz Docutils home page: http://docutils.sourceforge.net/ To subscribe to the mailing lists: - Development discussions (docutils-develop@lists.sourceforge.net): http://lists.sourceforge.net/lists/listinfo/docutils-develop - CVS checkin messages: http://lists.sourceforge.net/lists/listinfo/docutils-checkins High-level discussions take place on the Python Doc-SIG mailing list: http://mail.python.org/mailman/listinfo/doc-sig, mailto:Doc-SIG@python.org A ReStructuredText Primer (gentle introduction): http://docutils.sf.net/docs/rst/quickstart.html Quick reStructuredText (user reference): http://docutils.sf.net/docs/rst/quickref.html There has been a lot of progress lately. The reStructuredText parser is feature-complete, there's a standalone input file reader, and a simple HTML/CSS writer. The web site's HTML files are generated from reStructuredText source (the quickref being the only exception). There are a few simple front-ends in docutils/tools; see the README: http://docutils.sf.net/README.html. There is still much work to be done. Contributors are welcome! Release 0.1 (2002-04-20) ======================== This is the first release of Docutils, merged from the now inactive reStructuredText_ and `Docstring Processing System`_ projects. For the pre-Docutils history, see the `reStructuredText HISTORY`_ and `DPS HISTORY`_ files. .. _reStructuredText: http://structuredtext.sf.net/ .. _Docstring Processing System: http://docstring.sf.net/ .. _reStructuredText HISTORY: http://structuredtext.sf.net/HISTORY.html .. _DPS HISTORY: http://docstring.sf.net/HISTORY.html General changes: renamed 'dps' package to 'docutils'; renamed 'restructuredtext' subpackage to 'rst'; merged the codebases; merged the test suites (reStructuredText's test/test_states renamed to test/test_rst); and many modifications required to make it all work. -- David Goodger Open-source projects: - Python Docutils: http://docutils.sourceforge.net/ - The Go Tools Project: http://gotools.sourceforge.net/ From goodger@users.sourceforge.net Mon Apr 29 02:40:33 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Sun, 28 Apr 2002 21:40:33 -0400 Subject: [Doc-SIG] Auto-Numbered Enumerated Lists (reStructuredText) Message-ID: [The following is an excerpt from http://docutils.sf.net/spec/rst/alternatives.html (working version, not checked in yet). It was prompted by a suggestion by Fred Bremmer, which became alternative 3. I'm looking for comments, pro or con, on all aspects. Should this be implemented? (The implementation itself is trivial.) Which syntax is best? Which behavior? Any other suggestions? TIA.] The advantage of auto-numbered enumerated lists would be similar to that of auto-numbered footnotes: lists could be written and rearranged without having to manually renumber them. The disadvantages are also the same: input and output wouldn't match exactly; the markup may be ugly or confusing (depending on which alternative is chosen). 1. Use the "#" symbol. Example:: #. Item 1. #. Item 2. #. Item 3. Advantage: simple, explicit. Disadvantage: enumeration sequence cannot be specified (limited to arabic numerals); ugly. 2. As a variation on #1, first initialize the enumeration sequence? For example:: a) Item a. #) Item b. #) Item c. Advantages: simple, explicit, any enumeration sequence possible. Disadvantages: ugly; perhaps confusing with mixed concrete/abstract enumerators. 3. Alternative suggested by Fred Bremmer, from experience with MoinMoin:: 1. Item 1. 1. Item 2. 1. Item 3. Advantages: enumeration sequence is explicit (could be multiple "a." or "(I)" tokens). Disadvantages: perhaps confusing; otherwise erroneous input (e.g., a duplicate item "1.") would pass silently, either causing a problem later in the list (if no blank lines between items) or creating two lists (with blanks). Take this input for example:: 1. Item 1. 1. Unintentional duplicate of item 1. 2. Item 2. Currently the parser will produce two list, "1" and "1,2" (no warnings, because of the presence of blank lines). Using Fred's notation, the current behavior is "1,1,2 -> 1 1,2" (without blank lines between items, it would be "1,1,2 -> 1 [WARNING] 1,2"). What should the behavior be with auto-numbering? Fred has produced a patch__, whose initial behavior is as follows:: 1,1,1 -> 1,2,3 1,2,2 -> 1,2,3 3,3,3 -> 3,4,5 1,2,2,3 -> 1,2,3 [WARNING] 3 1,1,2 -> 1,2 [WARNING] 2 (After the "[WARNING]", the "3" would begin a new list.) I have mixed feelings about adding this functionality to the spec & parser. It would certainly be useful to some users (myself included; I often have to renumber lists). Perhaps it's too clever, asking the parser to guess too much. What if you *do* want three one-item lists in a row, each beginning with "1."? You'd have to use empty comments to force breaks. Also, I question whether "1,2,2 -> 1,2,3" is optimal behavior. In response, Fred came up with "a stricter and more explicit rule [which] would be to only auto-number silently if *all* the enumerators of a list were identical". In that case: 1,1,1 -> 1,2,3 1,2,2 -> 1,2 [WARNING] 2 3,3,3 -> 3,4,5 1,2,2,3 -> 1,2 [WARNING] 2,3 1,1,2 -> 1,2 [WARNING] 2 Should any start-value be allowed ("3,3,3"), or should auto-numbered lists be limited to begin with ordinal-1 ("1", "A", "a", "I", or "i")? __ http://sourceforge.net/tracker/index.php?func=detail&aid=548802 &group_id=38414&atid=422032 -- David Goodger Open-source projects: - Python Docutils: http://docutils.sourceforge.net/ - The Go Tools Project: http://gotools.sourceforge.net/ From tony@lsl.co.uk Mon Apr 29 09:55:48 2002 From: tony@lsl.co.uk (Tony J Ibbs (Tibs)) Date: Mon, 29 Apr 2002 09:55:48 +0100 Subject: [Doc-SIG] Auto-Numbered Enumerated Lists (reStructuredText) In-Reply-To: Message-ID: <012201c1ef5b$a7fbe450$545aa8c0@lslp862.int.lsl.co.uk> Another alternative for auto-numbered lists, sort-of similar to the way we handle footnotes at the moment, is to allow:: #1. First item. #3. Aha - I edited this in later. #2. Second item. This requires the user to ensure that list items are still unique - probably a good thing. I would suggest that it *not* be possible to have two adjacent lists with this mechanism - so the problems David describes with the list:: 1. Item 1. 1. Unintentional duplicate of item 1. 2. Item 2. would not be allowed - one would have to place a comment (or some other "breaking" text) between the two item 1.s to get two lists. I don't see that as a particularly onerous restriction. Presumably one would also allow:: #a. The first item #1. This *is* a new list. to mean two lists, though. As a side-effect, would we then allow people to assume that such items could be referred to? Consider the following:: We have a list. 1. The first item 2. The second item 3. The third item In item 2, we see the middle item. With autonumbering, such a textual reference is more problematical - but maybe we could allow:: We have a list. #1. The first item #3. The second item #2. The third item In item #3_, we see the middle item. (and no, I realise that syntax is likely to be wrong, but it's the *concept* I'm interested in). On the other hand, I'm actually agnostic on whether we need autonumbered lists at all - but then I'm the sort of person who will go back and adjust indentation in a long document to make it look pretty, anyway, so I'm maybe not the best person to judge! Tibs -- Tony J Ibbs (Tibs) http://www.tibsnjoan.co.uk/ Well we're safe now....thank God we're in a bowling alley. - Big Bob (J.T. Walsh) in "Pleasantville" My views! Mine! Mine! (Unless Laser-Scan ask nicely to borrow them.) From pinard@iro.umontreal.ca Mon Apr 29 18:07:43 2002 From: pinard@iro.umontreal.ca (=?iso-8859-1?q?Fran=E7ois?= Pinard) Date: 29 Apr 2002 13:07:43 -0400 Subject: [Doc-SIG] Re: [development doc updates] In-Reply-To: <15532.63699.122156.945015@grendel.zope.com> References: <20020404225927.530B318EAD1@grendel.zope.com> <20020404174752.M16962@tummy.com> <15532.63699.122156.945015@grendel.zope.com> Message-ID: [Fred L. Drake, Jr.] > The GNU info files have been contributed, but a patch has just been > submitted that I'll try to get to in time. In the past, we've run > into problems with elisp library incompatibilies between emacs and > xemacs. We'll see how the proposed patch holds up. If the Emacs Lisp incompatibilities come from the Emacs/XEmacs Texinfo formatting modules, I would guess you might be better off using the `makeinfo' tool instead, which is much more speedy, anyway. `makeinfo' (as well as the stand-alone `info' reader) are part of the Texinfo distribution release. I did not closely follow recent evolution in that area, however, so please take my comment with a grain of salt. -- François Pinard http://www.iro.umontreal.ca/~pinard From goodger@users.sourceforge.net Tue Apr 30 04:31:48 2002 From: goodger@users.sourceforge.net (David Goodger) Date: Mon, 29 Apr 2002 23:31:48 -0400 Subject: [Doc-SIG] Auto-Numbered Enumerated Lists (reStructuredText) In-Reply-To: <012201c1ef5b$a7fbe450$545aa8c0@lslp862.int.lsl.co.uk> Message-ID: Tony J Ibbs (Tibs) wrote: > Another alternative for auto-numbered lists, sort-of similar to the > way we handle footnotes at the moment, is to allow:: > > #1. First item. > #3. Aha - I edited this in later. > #2. Second item. I've added this as syntax alternative 4. > This requires the user to ensure that list items are still unique - > probably a good thing That doesn't seem like a good thing to me. We'd get the ugly syntax of the first two alternatives plus the inconvenience of having to account for item numbers, without much convenience gain (just random ordering). It might fly if we modify this proposal like so: - Simply prepend a "#" to a standard list enumerator to indicate auto-enumeration. - The numbers (or letters) of the enumerators themselves are not significant, except: - as a sequence indicator (arabic, roman, alphabetic), - and perhaps as start value (first list item) > I would suggest that it *not* be possible to have two adjacent lists > with this mechanism It'll happen though. A system message (warning or error) would be generated; I assume that's what you mean. > so the problems David describes with the list:: > > 1. Item 1. > > 1. Unintentional duplicate of item 1. > > 2. Item 2. > > would not be allowed Actually, there are no "problems" with this in the current spec/parser. It results in two lists, "1" and "1,2". Were the blank lines to be omitted between items, a warning would be generated between the two lists. Under Fred's proposed semantics, the list would be parsed as "1,2" and "2". > As a side-effect, would we then allow people to assume that such > items could be referred to? That would be tricky, and going too far I think. Either all auto-numbered lists in a document would need to have unique labels, or some sort of scoping rules would be required: would a list item reference refer to the preceding list or the following list? I'm loathe to introduce such rules. A preferable solution would be to use the existing hyperlink constructs to refer to the body of a list item; the item number couldn't be transferred, but a live link could be created. Auto-enumerated lists are only a marginal convenience feature; the syntax and semantics should be simple and clean. Too complex and it's no longer convenient enough to justify its existence. -- David Goodger Open-source projects: - Python Docutils: http://docutils.sourceforge.net/ - The Go Tools Project: http://gotools.sourceforge.net/