[Python-ideas] Before and after the colon in funciton defs.

[Steven D'Aprano]

Nick Coghlan wrote:
> On Fri, Sep 23, 2011 at 9:51 AM, Steven D'Aprano <steve at pearwood.info> wrote:
>> With decorator syntax, the scoping rules are obvious and straightforward:
>>
>> a = 10
>> @inject(b=a)
>> def foo():
>>    a = 20
>>    return b+a
> 
> Please read the previous thread from June (linked earlier in this
> thread). Decorator syntax cannot work without deep magic, because the
> compiler *doesn't know* that injected names need to be given special
> treatment.

I have read the previous thread, and I'm fully aware that this would be 
special. I'm pretty sure I even said it would be special in one of my 
posts :)

Excluding the default "do nothing" position, and the minimalist "just 
add conventions to introspection tools" proposal, I believe that @inject 
is the least magical proposal made so far. The other proposals require a 
new keyword, new syntax, or both. They require the compiler to do extra 
work. @inject requires nothing from the compiler. It all happens when 
the decorator is called. Comparing it to the voodoo needed for super() 
is completely unfair.

It also offers the most benefits:

* all the benefits of the proposed "static" declaration
   (early-binding, micro-optimisation, monkey-patching)
* the ability to patch pre-existing functions
* no call-time cost (it's not a wrapper, its a new function)
* fewer side-effects than conventional monkey-patching
* familiar syntax
* obvious semantics
* no new keywords
* no extra "line-noise" symbols

What's not to like?

Yes, inject() will be special. But I don't believe it will be magic, or 
at least not deep magic. We can get most of the way using supported 
Python functionality already: we can pull a function object apart, make 
a copy of the pieces as needed, and reassemble them into a new function. 
All that is supported by Python, none of it requires messing with 
private attributes: it's not much more magic than what functools.wraps() 
already does.

We can even take the bytecode from the code object, and because it's 
just a string, we can perform transformations on it. The only "magic" is 
the knowledge of what transformations to perform.

Take this simple example:

 >>> import dis
 >>> def func():
...     c = 1
...     return c+d
...
 >>> dis.dis(func)
   2           0 LOAD_CONST               1 (1)
               3 STORE_FAST               0 (c)

   3           6 LOAD_FAST                0 (c)
               9 LOAD_GLOBAL              0 (d)
              12 BINARY_ADD
              13 RETURN_VALUE


@inject(d=2)(func) would probably look produce something like this:

   2           6 LOAD_CONST               2 (2)
               9 STORE_FAST               1 (d)

   3           0 LOAD_CONST               1 (1)
               3 STORE_FAST               0 (c)

   4          12 LOAD_FAST                0 (c)
              15 LOAD_FAST                1 (d)
              18 BINARY_ADD
              19 RETURN_VALUE



If we had a Python assembler to match the disassembler, it would 
probably be easy.


If we are really allergic to the idea of bytecode manipulation, perhaps 
there are other ways to solve this. Just tossing ideas around, maybe 
function objects should keep a copy of their AST around, so that instead 
of bytecode hacking, you manipulate the AST and recompile the function. 
That might be less "hacky" than manipulating the bytecode, but I don't 
know that carrying around the AST for every function is a cost that is 
worth bearing. But it's an idea.



-- 
Steven