[Compiler-sig] FYI: python/nondist/src/Compiler

[Mark Hammond]

> I've made my current codebase available as part of the Python CVS
> tree, under the nondist branch.  The current code implements most of a
> Python source code to PyVM compiler.  The code is definitely rough;
> "refactoring" is the buzzword that needs to be applied.

This is definately cool.  Good job.

Ideally p2c and this compiler could share some common framework.  Greg also
mentioned this.  Ive got another project that was based on P2C that Im
working fulltime on (and Greg is helping) - so that makes it 3 "compilers"
we have going.  I bet that someone from the Types-SIG will jump on this for
their next-greatest type-inferencer :-)

So, I had a think about reusing some of this framework for other compilers,
and come to the conclusion that not as much as I had hoped can be shared
(  But still enough that it makes it worthwhile.  I can see the P2C
directory stuff being shared, along with the basic tree stuff you have.

So Im proposing that we have a bit of a fiddle with the CVS tree.  Rename
the package from "p2c" to "tools" - and the compiler itself can also be a
package.

eg:
compiler\__init__.py # Contains only: from compiler import compile
compiler\compiler.py # Existing compile.py, less the first 1/4.
compiler\tools\__init__.py # empty file
compiler\tools\transformer.py, ast.py, etc
compiler\tools\visitor.py # first 1/4 of existing compile.py

Ultimately, the idea is that:
>>> import compiler
>>> compiler.compile(code)

can be used in place of the built-in compiler.  Tools like "p2c" etc can
simply use "compiler.tools", and gain access to the core AST and
tree-walking code.

If you just change the COPYRIGHT file to reflect that it was taken from the
P2C project and renamed to "compiler.tools" everyone should be happy I would
think...

Greg suggested we can reuse some of the name-lookup code, but Im not too
sure we can.  What Jeremy has is a faithful implementation of the Python
name bindings scheme, which is perfect.  But any optimizing compiler (which
includes P2C) and any type inference systems will need to go a little bit
deeper than that - eg, P2C can optimize away certain builtin functions, but
to do that it needs to make the distinction between "global cos Ive never
seen it before" vs "global and Ive seen an assignment".

Does any of that make sense?

Mark.