[pypy-dev] Questions about the C core

[Andrew McGregor]

Has anyone thought about features that Python does not have that should be 
in Minimal?  My first candidate would be (ducks) macros, since that could 
drastically simplify constructing core Python features.  I have some 
thoughts on pythonic ways to do that which I need to write up.  Basically, 
if we use the parser module rather than a C parser, it gets to be fairly 
easy.

So, what would a pythonic macro look like?

The nearest match presently is a metaclass, but that only gets the class 
definition *after* compilation, which means that although we can modify the 
code at that point (even to the extent of retreiving the source, changing 
it and recompiling), it had to compile in the first place, so we can't add 
syntax.

Therefore a Pythonic macro should be:

A class which adds it's grammar (probably fairly constrained; limited to 
new keywords, perhaps) to the parser at parse time, by setting or appending 
to a magic attribute (__grammar__ perhaps).

Then the macro provides code which is called with the parse trees resulting 
from that grammar, modifies them, and returns the parse tree that will 
actually be compiled.  If you want to evaluate some bits at compile time, 
do that explicitly in the macro (opposite way around to lisp, where you 
have complicated rules).

It may be necessary to provide a quote: keyword and quote() (or noeval()? 
should the names be different?) builtin that quote a block or an expression 
to prevent compile-time evaluation by macros.

Advantages:
It looks like a metaclass.
It can be inherited from to use the macro in new classes (hence __grammar__ 
should be appended to rather than just set).
If the kinds of grammar rules that can be added are constrained just right, 
editors won't mess up the indentation and code using macros will still look 
like Python.
If, however, we can use unconstrained grammar mods in the core, then we can 
do things like build dictionaries inline.

Hmm, must go now, I was dreaming up an example, but I've got to go sell a 
house (Hooray :-)

Comments?

Andrew