Painful?: Using the ast module for metaprogramming

[Joseph Garvin]

On Mon, Apr 6, 2009 at 1:33 AM, "Martin v. Löwis" <martin at v.loewis.de> wrote:
>> -If I have the source to a single function definition and I pass it to
>> ast.parse, I get back an ast.Module. Why not an ast.FunctionDef?
>
> Because it is easier for processing if you always get the same type of
> result. Typically, you don't know what's in the source code, so you
> need to parse, then inspect.

I see. True I'm guessing for the applications for which the module was
originally intended. In a metaprogramming context you usually know
though.


>> -An ast.Name always has an ast.Load or ast.Store "context" associated
>> with it. This is a bit odd, since based on the actual context (what
>> the parent of the ast.Name node is) it is always clear whether the
>> variable needs to be loaded or stored
>
> It simplifies the interpretation/compilation of the code, by removing
> the need to go up in the tree. Notice that just looking at the parent
> node would not be sufficient...

I see how it avoids needing to look at the parent node in general, but
if we were compiling by recursively descending through the AST, then
we would know whether Name's would be loads or stores by the time we
got to them (we would already had to have visited an encompassing
assignment or expression) -- except in the case of your a = b = c
expression, which I'm curious how Python handles. The natural answer
is for assignment to be an expression (so b = c returns the new value
of b). But Python doesn't do that, so then I'd expect we'd have some
third ast.LoadAndStore() option for b, but examining ast.parse's
behavior it looks like it chooses Store...

>> -Why can't orelse for ast.If and ast.While default to empty []?
>
> You want to store None? That would be a type error; orelse is
> specified as "stmt*". So it must be a list.

A list is actually what I want, an empty one. The problem is that
ast.While and ast.If's constructors default to the opposite,
orelse=None. Same with keywords and args for ast.Call. Admittedly,
adding orelse=[] to the constructor calls isn't terribly burdensome,
but it does make already obfuscated looking AST mangling code even
worse.

>> -Why can't I eval a functiondef to get back a function object?
>
> Because a definition is not an expression. You can only eval
> expressions.

I understand that if function definitions were expressions, because of
the whitespace syntax there wouldn't be a way to express an assignment
to such an expression. But, why would it be problematic to let them be
expressions anyway?

>> -The module provides no means to convert an AST back into source code,
>> which would be nice for debugging.
>
> See Demo/parser/unparse.py

Thanks :)

>> -It would be nice if decorators were passed a function's AST instead
>> of a function object.
>
> How could this possibly work? If you run from a pyc file, there will
> be no AST available to pass.

I hadn't thought about bytecode compilation. In addition to the other
suggestions you would have to change it to preserve an AST.

Regards,

Joe