From jpe@archaeopteryx.com Wed Aug 9 16:50:06 2000 From: jpe@archaeopteryx.com (John Ehresman) Date: Wed, 9 Aug 2000 11:50:06 -0400 (EDT) Subject: [Compiler-sig] Initial release of parse tools for Python source Message-ID: [I tried to send this yesterday, but it appeared to bounce; apologies to anyone who's seeing this twice] I have uploaded a tarball of the source for tools we are using to analyze Python source code to ftp://archaeopteryx.com/pub/parsetools/. This is an preliminary release; it's working well for our purposes, but parts of it are incomplete. All source in this distribution is being released under an MIT/X Consortium style license. We will be actively developing the tools and intend to release future versions under the same license. The distribution contains a set of extension modules that provides the means to tokenize Python source code, access the "pgen" parser built into the Python runtime without going through the builtin tokenizer, and transform parse trees created by the builtin parser to AST nodes that are (almost) a subset of those used by the Python2C compiler (I don't use the AssName, AssTuple, AssList nodes and intend to add a 'Delete' node). All of the core functionality is written in C because these libraries are being used to analyze Python source code as it is being entered or modified. Unfortunately, the documentation leaves much to be desired ;). There are some incomplete notes in the README file and embedded in javadoc style comments within the source itself. There is also a logical_test.py script which will tokenize any files given to it on the command line and print out the tokens used on each logical line. I intend to release updates of this every few weeks and will set up a CVS repository at sourceforge or on archaeopteryx.com if there is interest. Please let me know of any questions or problems, John ------------------------------------------------------------------------ Archaeopteryx Software, Inc. Wing IDE for Python www.archaeopteryx.com Take Flight!