Using arguments in a decorator

[Jon Clements]

On Friday, 20 April 2012 16:57:06 UTC+1, Rotwang  wrote:
> Hi all, here's a problem I don't know how to solve. I'm using Python 2.7.2.
> 
> I'm doing some stuff in Python which means I have cause to call 
> functions that take a while to return. Since I often want to call such a 
> function more than once with the same arguments, I've written a 
> decorator to eliminate repeated calls by storing a dictionary whose 
> items are arguments and their results:
> 
> def memo(func):
>      def memofunc(*args, **kwargs):
>          twargs = tuple(kwargs.items())
>          if (args, twargs) in memofunc.d:
>              return copy(memofunc.d[(args, twargs)])
>          memofunc.d[(args, twargs)] = func(*args, **kwargs)
>          return copy(memofunc.d[(args, twargs)])
>      memofunc.__name__ = func.__name__
>      memofunc.d = {}
>      return memofunc
> 
> 
> If a function f is decorated by memo, whenever f is called with 
> positional arguments args and keyword arguments kwargs, the decorated 
> function defines twargs as a hashable representation of kwargs, checks 
> whether the tuple (args, twargs) is in f's dictionary d, and if so 
> returns the previously calculated value; otherwise it calculates the 
> value and adds it to the dictionary (copy() is a function that returns 
> an object that compares equal to its argument, but whose identity is 
> different - this is useful if the return value is mutable).
> 
> As far as I know, the decorated function will always return the same 
> value as the original function. The problem is that the dictionary key 
> stored depends on how the function was called, even if two calls should 
> be equivalent; hence the original function gets called more often than 
> necessary. For example, there's this:
> 
>  >>> @memo
> def f(x, y = None, *a, **k):
> 	return x, y, a, k
> 
>  >>> f(1, 2)
> (1, 2, (), {})
>  >>> f.d
> {((1, 2), ()): (1, 2, (), {})}
>  >>> f(y = 2, x = 1)
> (1, 2, (), {})
>  >>> f.d
> {((1, 2), ()): (1, 2, (), {}), ((), (('y', 2), ('x', 1))): (1, 2, (), {})}
> 
> 
> What I'd like to be able to do is something like this:
> 
> def memo(func):
>      def memofunc(*args, **kwargs):
>          #
>          # define a tuple consisting of values for all named positional
>          # arguments occurring in the definition of func, including
>          # default arguments if values are not given by the call, call
>          # it named
>          #
>          # define another tuple consisting of any positional arguments
>          # that do not correspond to named arguments in the definition
>          # of func, call it anon
>          #
>          # define a third tuple consisting of pairs of names and values
>          # for those items in kwargs whose keys are not named in the
>          # definition of func, call it key
>          #
>          if (named, anon, key) in memofunc.d:
>              return copy(memofunc.d[(named, anon, key)])
>          memofunc.d[(named, anon, key)] = func(*args, **kwargs)
>          return copy(memofunc.d[(named, anon, key)])
>      memofunc.__name__ = func.__name__
>      memofunc.d = {}
>      return memofunc
> 
> 
> But I don't know how. I know that I can see the default arguments of the 
> original function using func.__defaults__, but without knowing the 
> number and names of func's positional arguments (which I don't know how 
> to find out) this doesn't help me. Any suggestions?
> 
> 
> -- 
> Hate music? Then you'll hate this:
> 
> http://tinyurl.com/psymix

Possibly take a look at functools.lru_cache (which is Python 3.2+), and use the code from that (at it's part of the stdlib, someone must have done design and testing on it!). http://hg.python.org/cpython/file/default/Lib/functools.py

Jon