[Python-ideas] PEP 511: API for code transformers

[Sjoerd Job Postmus]

On Sat, Jan 16, 2016 at 12:06:58PM +0100, Petr Viktorin wrote:
> The "transformer" API can be used for syntax extensions as well, but the
> registration needs to be different so the effects are localized. For
> example it could be something like::
> 
>     importlib.util.import_with_transformer(
>         'mypackage.specialmodule', MyTransformer())
> 
> or a special flag in packages::
> 
>     __transformers_for_submodules__ = [MyTransformer()]
> 
> or extendeding exec (which you actually might want to add to the PEP, to
> make giving examples easier)::
> 
>     exec("print('Hello World!')", transformers=[MyTransformer()])
> 
> or making it easier to write an import hook with them, etc...

So, you'd have to supply the transformer used before importing? That
seems like a troublesome solution to me.

A better approach (to me) would require being able to document what
transformers need to be run inside the module itself. Something like

    #:Transformers modname.TransformerClassName, modname.OtherTransformerClassName

The reason why I would prefer this, is that it makes sense to document
the transformers needed in the module itself, instead of in the code
importing the module.

As you suggest (and rightly so) to localize the effects of the
registration, it makes sense to do the registration in the affected
module.

Of course there might be some cases where you want to import a module
using a transformer it does not need to know about, but I think that
would be less likely than the case where a module knows what
transformers there should be applied.

As an added bonus, it would let you apply transformers to the
entry-point:

    #!/usr/bin/env python
    #:Transformers foo.BarTransformerMyCodeCanNotRunWithout

But as you said, this support is probably outside the scope of the PEP
anyway.

Kind regards,
Sjoerd Job