[Python-ideas] PEP 511: API for code transformers

[Brett Cannon]

On Fri, 15 Jan 2016 at 09:40 Victor Stinner <victor.stinner at gmail.com>
wrote:

> 2016-01-15 18:22 GMT+01:00 Brett Cannon <brett at python.org>:
> > I just wanted to point out to people that the key part of this PEP is the
> > change in semantics of `-O` accepting an argument.
>
> The be exact, it's a new "-o arg" option, it's different from -O and
> -OO (uppercase). Since I don't know what to do with -O and -OO, I
> simply kept them :-D
>
> > I should also point out that this does get tricky in terms of how to
> handle
> > the stdlib if you have not pre-compiled it, e.g., if the first module
> > imported by Python is the encodings module then how to make sure the AST
> > optimizers are ready to go by the time that import happens?
>
> Since importlib reads sys.implementation.optim_tag at each import, it
> works fine.
>
> For example, you start with "opt" optimizer tag. You import everything
> needed for fatoptimizer. Then calling sys.set_code_transformers() will
> set a new optimizer flag (ex: "fat-opt"). But it works since the
> required code transformers are now available.
>

I understand all of that; my point is what if you don't compile the stdlib
for your optimization? You have to import over 20 modules before user code
gets imported. My question is how do you expect the situation to be handled
where you didn't optimize the stdlib since the 'encodings' module is
imported before anything else? If you set your `-o` flag and you want to
fail imports if the .pyc isn't there, then wouldn't that mean you are going
to fail immediately when you try and import 'encodings' in Py_Initialize()?


>
> The tricky part is more when you want to deploy an application without
> the code transformer, you have to ensure that all .py files are
> compiled to .pyc. But there is no technical issues to compile them,
> it's more a practical issue.
>
> See my second email with a lot of commands, I showed how .pyc are
> created with different .pyc filenames. Or follow my commands to try my
> "fatpython" fork to play yourself with the code ;-)
>
> > And lastly, Victor proposes that all .pyc files get an optimization tag.
> > While there is nothing technically wrong with that, PEP 488 purposefully
> > didn't do that in the default case for backwards-compatibility, so that
> will
> > need to be at least mentioned in the PEP.
>
> The PEP already contains:
> https://www.python.org/dev/peps/pep-0511/#optimizer-tag
> "Remove also the special case for the optimizer level 0 with the
> default optimizer tag 'opt' to simplify the code."
>
> Code relying on the exact .pyc filename (like unit tests) already have
> to be modified to use the optimizer tag. It's just an opportunity to
> simplify the code. I don't really care of this specific change ;-)
>

Right, it's just mentioning the backwards-compatibility issue should be
there.

-Brett
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20160116/9a05e423/attachment-0001.html>