Docorator Disected
Bengt Richter
bokr at oz.net
Tue Apr 5 20:23:01 EDT 2005
On Tue, 05 Apr 2005 18:59:32 GMT, Ron_Adam <radam2 at tampabay.rr.com> wrote:
>On Tue, 05 Apr 2005 06:52:58 GMT, bokr at oz.net (Bengt Richter) wrote:
>
>>>Ok, yes, besides the globals, but I figured that part is obvious so I
>>>didn't feel I needed to mention it. The function call works the same
>>>even though they are not nested functions.
>>
>>I am afraid that is wrong. But be happy, this may be the key to what ISTM
>>is missing in your concept of python functions ;-)
>
>The expression in the form of "function(args)(args)" is the same
>pattern in two "different" cases, which was all that I was trying to
>say. Not that the exact process of the two different cases were the
>same.
>
>>So, no, it does not "work the same." In fact, it is a misconception to talk about
>>a nested fee as if it existed ready to call in the same way as foo. It doesn't
>>exist that way until the fee def is EXECUTED, producing the callable fee.
>
>Ok, I'm going to have to be more careful in how I phrase things I
>think, I tend to over-genralize a bit. I said they were "the same",
>but meant similar, a mistake in wording, but not in my understanding.
>
>But this is a good point. In my example the calling expression does
>not yet know who the next tuple of arguments will go to until foo
>returns it. That part is the same, but as you point out in a nested
>scope foo defines fee then returns it. And in the non nested example
>fee is already defined before foo is called. And they must use
>globals to communicate because they are not share the same name space.
>They differ because fee is temporary, in the nested version, only
>existing until the expression foo(arg)(arg) is evaluated. It never
>gets assigned a name in foo's parent name space. Do I have that
>correct?
I think so.
>
>
>>We can use dis to see the above clearly:
>
>Love the byte code walk through, Thanks. Is there a resource that
>goes in depth on python byte code and the compiler? I haven't been
>able to find much on it on google.
>
I don't know of anything other than the compiler and cpython sources.
The byte codes are not all the same from version to version, since
added language features may require new byte code operations, at least
for efficiency, and (as our postings here show ;-) explaining code
clearly is as much of a job as writing it. Fortunately, python code
is pretty readable, and there is a lot of interesting reading in
Python-2.4xxx\Lib\compiler\ and Python-2.4xxx\Lib\compiler\
on your disk if you download the source installation. Lots of
other goodies as well like demo code etc.
>> >>> import time
>> >>> def globalfun(): return '[%s]'%time.ctime()
>> ...
>> >>> foo()
>> <function fee at 0x02EF802C>
>> >>> foo()(111, 222)
>> (111, 222)
>> >>> foo()(333)
>> (333, '[Mon Apr 04 22:31:23 2005][Mon Apr 04 22:31:23 2005]')
>> >>> foo()(333)
>> (333, '[Mon Apr 04 22:31:37 2005][Mon Apr 04 22:31:37 2005]')
>
>I like your idea of using time stamps to trace code! :)
Well, I wanted to emphasize the time dimension in creation and existence
of things, so I thought to tag them.
[...]
>I understand functions, sometimes it's difficult to describe just what
>it is I don't understand yet, and sometimes I fool myself by jumping
>to an invalid conclusion a little too quickly. But I do this for
>enjoyment and learning, so I'm not constrained by the need to not make
>mistakes, (those are just part of learning in my oppinion), as I would
>if my job depended on it. However it's a little frustrating when my
>inability to write well, gets in the way of expressing myself
>accurately.
I feel very much the same ;-)
>
>But a few questions remain...
>
>When a @decorator statement is found, How does the compiler handle it?
>
>Let me see if I can figure this out...using dis. :)
>
>>>> from dis import dis
>>>> def deco1(d1): return d1
>
>>>> def func1(f1):
> @deco1
> def func2(f2):
> return f2
> return func2(f1)
That last line is a bit strange, which leads to the strangeness
you ask about below.
>
>>>> func1(2)
>2
>
>>>> dis(deco1)
> 1 0 LOAD_FAST 0 (d1)
> 3 RETURN_VALUE
>>>> dis(func1)
> 2 0 LOAD_GLOBAL 0 (deco1)
> 3 LOAD_CONST 1 (<code object func2 at
>00B45CA0, file "<pyshell#11>", line 2>)
> 6 MAKE_FUNCTION 0
> 9 CALL_FUNCTION 1
> 12 STORE_FAST 1 (func2)
>
> 5 15 LOAD_FAST 1 (func2)
> 18 LOAD_FAST 0 (f1)
> 21 CALL_FUNCTION 1
> 24 RETURN_VALUE
>
>I'm not sure how to interpret this... Line 5 and below is the return
>expression. The part above it is the part I'm not sure about.
Well, it's showing the result of your python code, but the return func2(f1)
is obscuring normal decorator functionality. I suggest going from the simplest
and introducing complications stepwise. You are defining a func1 that could
be used as a decorator function, and it decorates its function argument by
explicitly calling an internally defined decorator function func2, whose internal
definition involved decorating func2 using func1. We can walk through it.
It's legal usage, it's just not the simplest example or clearest design ;-)
>
>Is the first CALL_FUNCTION calling deco1 with the result of the
>defined functions reference, as it's argument? Then storing the result
>of deco1 with the name func2?
Yes, if I understand you correctly.
I'll use list notation to represent the stack, so [] is empty and
[bottom, middle, top] is a stack with references to three things.
So _after_ each line above, we get
0 -> [deco1] This is a reference to the callable function deco1
3 -> [deco1, <code object func2>] The top is what's necessary to make the func2 function
6 -> [deco1, func2] MAKE_FUNCTION pops its arg and pushes it result (func2)
9 -> [func2*] CALL_FUNCTION 1 (one arg) called deco1(func2) and stacked the decorated func2
12 -> [] func2 reference popped from stack and stored in local func2
15 -> [func2*] local func2 (I use * to indicate it's the decorated func2) pushed on stack
18 -> [func2*, f1] stack the outer f1 argument. This is for your weird func2(f1) return value expression
21 -> [f1*] CALL_FUNCTION calls the decorated func2 with f1 as argument and stacks the modified f1
24 -> [] f1* (modified f1) is popped for return value
>
>If so the precompiler/parser is replacing the @deco1 with a call to
>the deco1 function like this.
>
> deco1( (def func2(f2):return f2) )
>
>But this causes an illegal syntax error on the def statement. So you
>can't do it directly. Or is there yet another way to view this? :)
>
With simple examples? ;-)
One of the original examples of decorating was to replace the
staticmethod and classmethod function calls that had to be done
after the method defs. So the simplest view goes back to
@deco
def foo(): pass
being the equivalent (except if deco raises and exception) of
def foo(): pass
foo = deco(foo)
The plain name @deco was then allowed to become a simple xxx.yyy(zzz,...) expression
returning a callable that would serve like the decorator function a bare name normally
referred to. And then the mechanism could be cascaded. I suggest looking at the
code for the simplest cascade of normal decorators. E.g., (putting the example in
the body of a function makes dis.dis easy ;-)
>>> def deco_outer(f): return f
...
>>> def deco_inner(f): return f
...
>>> def example():
... @deco_outer
... @deco_inner
... def foo(): pass
...
>>> import dis
>>> dis.dis(example)
2 0 LOAD_GLOBAL 0 (deco_outer) -> [deco_outer]
3 LOAD_GLOBAL 1 (deco_inner) -> [deco_outer, deco_inner]
6 LOAD_CONST 1 (<code object foo at 02EE4FA0, file "<stdin>", line 2>) -> [deco_outer, deco_inner, <code obj>]
9 MAKE_FUNCTION 0 -> [deco_outer, deco_inner, foo]
12 CALL_FUNCTION 1 -> [deco_outer, foo*] foo* == deco_inner(foo)
15 CALL_FUNCTION 1 -> [foo**] foo** == deco_outer(foo*)
18 STORE_FAST 0 (foo) -> [] foo = deco_outer(deco_inner(foo))
21 LOAD_CONST 0 (None) # ignore ;-)
24 RETURN_VALUE
I didn't return foo from example, so it generated code at 21 to return a None,
but you can ignore that, since the decoration ends with the binding of the function
name (foo here) done at 18 with STORE_FAST.
Regards,
Bengt Richter
More information about the Python-list
mailing list