[Python-ideas] Proposal for new-style decorators

[Terry Reedy]

On 4/26/2011 11:05 AM, Christophe Schlick wrote:

I got interrupted in responding this, and you have now posted parts 2 
and 3, so I will snip and revise a bit.

 > a new syntax for defining decorators in Python.

There is no special syntax for defining decorators -- just normal nested 
function or class definition syntax. To put it another way, Python does 
not have decorator objects. A decorator is simply a callable (function, 
class, or class instance with __call__ method) applied to a object 
(function or class) with the @deco syntax (before the def/class 
statement) instead of the normal call syntax (after the def/class 
statement). Decorators return either the original object (usually 
modified) or another object that is usually, but not necessarily, of the 
same kind as the input.

As Stephen noted, syntax is what is defined in the Language reference. 
Code patterns are what are embodied in the stdlib (or pypi or the Python 
cookbook or other example repositories).

What you are actually proposing is a meta-decorator (class) whose 
instances can be used as decorators because the class has a __call__ 
instance method. This sort of thing is a known alternative to the nested 
function pattern.

> programmer has no other choice than to create a dummy function name

Many of us consider dummy names a bogus problem. I recommend you skip 
this argument. In any case, this, like the other 'problems' you describe 
for nested functions, has nothing in particular with their usage as 
decorators.

> (only used for one single 'return' statement), which is never a good
> coding principle, whatever the programming language.

This strikes me as a bogus rule: a single use of a local name is quite 
common, and not just in Python. I recommend leaving this also out of 
your problem list. Stick with the two real problems.

1. The double or triple nested function pattern has a lot of boilerplate 
and can be difficult to learn. Hiding boilerplate in a class makes the 
use pattern simpler and easier to learn. This is a real benefit. One of 
the three major benefits of writing a generator function versus an 
equivalent iterator class is that is hides the boilerplate required for 
the latter. Similarly, for loops hide the boilerplate required for an 
equivalent while loop.

2. Introspection (more comments below), which your class also addresses.

> #---
>    def old_style_repeat_var(n=3, trace=True):
>      """docstring for decorating function"""

Actually, this is the docstring for the decorator making function.

>      def dummy_deco_name_never_used(func):
>      """docstring never used"""
>        # @wraps(func)
>        def dummy_func_name_never_used(*args, **keys):
>          """docstring for decorated function"""
>          if trace:
>            print "apply 'old_style_repeat_var' on %r" % func.__name__
>          for loop in range(n): func(*args, **keys)
>        return dummy_func_name_never_used
>      return dummy_deco_name_never_used
> #---

> This time a two-level function nesting is required and the code needs
> two dummy names for these two nested functions.

'deco' and 'wrapper' work for me. But I agree that this is a bit 
confusing. But I think that is somewhat inherent in calling a 
decorator-maker f1 to return decorator f2 that returns wrapper f3 that 
wraps the original function f.

 > Note that the  docstring of the middle nested function
 > is even totally invisible for introspection tools.

Not true. To see the docstring of a dynamically created temporary 
function, you have to either dynamically create it or dig inside the 
function that creates it to find the constant string:

 >>> old_style_repeat_var().__doc__
'docstring never used'

 >>> old_style_repeat_var.__code__.co_consts[1].co_consts[0]
'docstring never used'

But I am not sure why you would want to see it or even have one.

> So whether you like nested functions or not,
> there is some evidence here that the current syntax is somehow
> suboptimal.

The 'problems' of nested defs has nothing to do with decorators in 
particular. Functional programmers use them all the time.

> Another drawback of OSD is that they do not gently collaborate with
> introspection and documentation tools. For instance, let's apply our
> decorator on a silly 'test' function:
>
> #---
>    @old_style_repeat_var(n=5) # 'trace' keeps its default value
>    def test(first=0, last=0):
>      """docstring for undecorated function"""
>      print "test: first=%s last=%s" % (first, last)
> #---
>
> Now, if we try 'help' on it, we get the following answer:
>
> #---
>>>> help(test)
> dummy_func_name_never_used(*args, **keys)
>      docstring for decorated function
> #---

Only because you commented out @wraps.
Again, this is not a problem of the @decorator syntax but of *all* 
wrapping callables. Functools.partial has the same 'problem'.

> '@wraps(func)' copies the name and the docstring from the undecorated
> function to the decorated one, in order to get some useful piece of
> information when using 'help'. However, the signature of the function
> still comes from the decorated function, not the genuine one.

I am not sure what you mean. If the two signatures are different, then 
one must use the signature of the wrapper when calling it, not the 
signature of the wrappee, which is perhaps what you mean by 'the genuine
one'. The problem of generic wrappers having generic signatures (*args, 
**kwds) is endemic to using generic wrappers instead of special case 
wrappers.

> reason is that signature copying is not an easy process.

True if you want to do it generically. Copying with modification, as 
functools.partial would have to do, is even worse.

> The only
> solution is to inspect the undecorated function and then use 'exec' to
> generate a wrapper with a correct signature. This is basically what is
> done in the 'decorator' package (available at PyPI) written by Michele
> Simionato. There has been a lengthy discussion in python-dev (in 2009
> I guess, but I can't find the archive right now) whether to include or
> not this package in the standard library.

The other solution is to not use a generic wrappers with generic 
signatures but to write specific wrappers with the actual signature, 
which people did, for instance, before functools and partial() were 
added to Python.

There have been proposals but no consensus on a decorator or decolib 
module for the stdlib. I second the other recommendations to make your 
proposal available on the cookbook site, etc.

-- 
Terry Jan Reedy