[Python-ideas] Incorporating something like byteplay into the stdlib

[Yury Selivanov]

On 2016-02-12 4:13 PM, Andrew Barnert wrote:
> On Feb 12, 2016, at 12:36, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
>>> On 2016-02-11 10:58 PM, Andrew Barnert via Python-ideas wrote:
>>> tl;dr: We should turn dis.Bytecode into a builtin mutable structure similar to byteplay.Code, to make PEP 511 bytecode transformers implementable.
>> Big -1 on the idea, sorry.
>>
>> CPython's bytecode is the implementation detail of CPython.  PyPy has some opcodes that CPython doesn't have, for example.  Who knows, maybe in CPython 4.0 we won't have code objects at all :)
>>
>> Adding something to the standard library means that it will be supported for years to come.  It means that the code is safe to use.  Which, in turn, guarantees that there will be plenty of code that depends on this new functionality.  At first some of that code will be bytecode optimizers, later someone implements LINQ-like extension, and in no time we lose our freedom to work with opcodes.
>>
>> If this "new functionality" is something that depends on CPython's internals, it will only fracture the ecosystem.  PyPy, or Pyston, or IronPython developers will either have to support byteplay-like stuff (which might be impossible), or explain their users why some libraries don't work on their platform.
> This sounds like potentially a argument against adding bytecode processors in PEP 511.[^1] But if PEP 511 *does* add bytecode processors, I don't see how my proposal makes things any worse.

The main (and only?) motivation behind PEP 511 is the optimization of 
CPython.  Maybe the new APIs will only be exposed at C level.

> Having dis (and inspect, and types.CodeType, and so on) be part of the stdlib makes it easier, not harder, to change CPython without breaking code that may need to introspect it for some reason.

You don't need mutability for introspection.

> In the same way, having a mutable dis would make it easier, not harder, to change CPython without breaking bytecode processors.

PEP 492 added a bunch of new opcodes.  Serhiy is exploring an 
opportunity of adding few more LOAD_CONST_N opcodes.  How would a 
mutable byteplay-code-like object in the dis module help that?

Interacting with bytecode in Python is generally considered unsafe, and 
used mostly for the purposes of experimentation, for which a PyPI module 
is enough.

FWIW I have lots of experience working with bytecodes.  For example, I 
have code in production systems in which decorators patch functions' 
code objects: they guard 'yield' expressions in 'finally' blocks with 
some function calls.  So far that code had only caused me maintenance 
pain. It's the only code that I have to upgrade each time a new version 
of Python is released.

>    [^1]: Then again, it's just as good an argument against import hooks, and exposing the __code__ member on function objects so decorators can change it, and so on, and years with those features hasn't created a catastrophe...

Import hooks (and even AST/parse/compile) is a much more high-level 
API.  I'm not sure we can compare them to byteplay.

Yury