[Compiler-sig] New version of parsetools

[John Ehresman]

Hi,

I've put a new version of our parsetools package on our ftp server at
ftp://archaeopteryx.com/pub/parsetools/parsetools-0.9.2.tgz
The package contains a tokenizer for either Python 1.5.2 or Python 2.0
source, an interface to the builtin 'pgen' parser that bypasses Python's
tokenizer, and a parse tree to AST transformer which works with parse
trees from the 1.5.2 & 2.0 parsers.  The major changes from the previous
version is support for Python 2.0, the ability to compile a grammar into
the pgen module so it's grammar can differ from the grammar in the core,
and the storage of positions of individual tokens in the source.

Included is a transformer_test.py script which will parse & transform each
line of given source files to an AST.  It's also the beginnings of Python
syntax checker with line-by-line error recovery; at most one syntax error
will be found on every line.  Running it over a C or sh file results in a
error for nearly every line (except where the source happens to be valid
Python).  It's not complete, though, because indentation is ignored and
multi-line constructs, such as if ... :\n ...\n else:\n..., are not
checked for correctness.

Please let me know of any problems,

John

------------------------------------------------------------------------
Archaeopteryx Software, Inc.                        Wing IDE for Python 
www.archaeopteryx.com                               Take Flight!