[issue11549] Build-out an AST optimizer, moving some functionality out of the peephole optimizer

[STINNER Victor]

STINNER Victor added the comment:

At the AST level, you have a wide range of possible optimizations. See the optimizations that I implemented in fatoptimizer (FAT Python) to have an idea:
http://fatoptimizer.readthedocs.io/en/latest/optimizations.html

FAT Python adds guards checked at runtime, something not possible (not wanted) here.

But if you start with constant folding, why not implementing constant propagation as well? What about loop unrolling?

Where is the limit? If you implement the AST optimizer in C, the limit will probably be your C skills and your motivation :-) In Python, the limit is more the Python semantics which is... hum... not well defined. For example, does it break the Python semantics to replace [i for i in (1, 2, 3)] with [1, 2, 3]? 

What if you use a debugger? Do yo expect a list comprehension or a literal list?

FYI I suspended my work on FAT Python because almost no other core developer was interested. I didn't get any support, whereas I need support to push core FAT Python features like function specialization and runtime checks (PEP 510, see also PEP 511). Moreover, I failed to show any significant speedup on non-trivial functions. I abandoned before investigating function inlining, even if FAT Python already has a basic support for function inlining.

This issue is open since 2011. The question is always the same: is it worth it?

An alternative is to experiment an AST optimizer outside CPython and come back later with more data to drive the design of such optimizer. With FAT Python, I chose to add hooks in the Python compiler, but different people told me that it's possible to do that without such hook but importlib (importlib hooks).

What do you think Naoki?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11549>
_______________________________________