[Python-Dev] AST optimizer implemented in Python

[Stefan Behnel]

"Martin v. Löwis", 11.08.2012 23:27:
>>> +1  We should add some form of setastoptimizer API in 3.4.  Please start a
>>> PEP for this.  It would be nice to include the ability to properly cache
>>> the
>>> ast optimizer output so that it does not have to run every time (in pyc
>>> files or similar, etc) but can verify that it is the specific ast optimizer
>>> that ran on a give existing cached copy.
>>
>> Once .pyc is created, I do not think that we keep the AST around any
>> longer. Otherwise, I wouldn't have to write PyXfuscator.
>>
>> https://bitbucket.org/namn/pyxfuscator
>>
>> Or perhaps I am misunderstanding you.
> 
> I think you misunderstood. What gps is concerned about (IIUC) that some
> people add ast optimizers in some run of Python, but other AST optimizers
> in a different run. Then, if you use a Python byte code
> file, you should be able to find out what AST optimizers have been
> run to create the pyc file, so you know whether you have to recompile
> or not.
> 
> Of course, if that is really a desirable feature, you may want multiple
> pyc files, per combination of AST optimizers. The __pycache__ directory
> would readily support that.
> 
> This seems to get complicated quickly, so a PEP is indeed desirable.

I think that gets overly complicated due to state explosion. Just recompile
when things do not match, or simply ignore mismatches (possibly based on a
command line switch). Compilers for statically compiled languages have been
working nicely like that for ages.

Most people won't run the interpreter with different optimiser settings,
and if they do, they can just delete the __pycache__ before the next run.

Stefan