Using arguments in a decorator

[Rotwang]

On 20/04/2012 17:10, Jon Clements wrote:
> On Friday, 20 April 2012 16:57:06 UTC+1, Rotwang  wrote:
>> Hi all, here's a problem I don't know how to solve. I'm using Python 2.7.2.
>>
>> I'm doing some stuff in Python which means I have cause to call
>> functions that take a while to return. Since I often want to call such a
>> function more than once with the same arguments, I've written a
>> decorator to eliminate repeated calls by storing a dictionary whose
>> items are arguments and their results:
>>
>> def memo(func):
>>       def memofunc(*args, **kwargs):
>>           twargs = tuple(kwargs.items())
>>           if (args, twargs) in memofunc.d:
>>               return copy(memofunc.d[(args, twargs)])
>>           memofunc.d[(args, twargs)] = func(*args, **kwargs)
>>           return copy(memofunc.d[(args, twargs)])
>>       memofunc.__name__ = func.__name__
>>       memofunc.d = {}
>>       return memofunc
>>
>>
>> If a function f is decorated by memo, whenever f is called with
>> positional arguments args and keyword arguments kwargs, the decorated
>> function defines twargs as a hashable representation of kwargs, checks
>> whether the tuple (args, twargs) is in f's dictionary d, and if so
>> returns the previously calculated value; otherwise it calculates the
>> value and adds it to the dictionary (copy() is a function that returns
>> an object that compares equal to its argument, but whose identity is
>> different - this is useful if the return value is mutable).
>>
>> As far as I know, the decorated function will always return the same
>> value as the original function. The problem is that the dictionary key
>> stored depends on how the function was called, even if two calls should
>> be equivalent; hence the original function gets called more often than
>> necessary. For example, there's this:
>>
>>   >>>  @memo
>> def f(x, y = None, *a, **k):
>> 	return x, y, a, k
>>
>>   >>>  f(1, 2)
>> (1, 2, (), {})
>>   >>>  f.d
>> {((1, 2), ()): (1, 2, (), {})}
>>   >>>  f(y = 2, x = 1)
>> (1, 2, (), {})
>>   >>>  f.d
>> {((1, 2), ()): (1, 2, (), {}), ((), (('y', 2), ('x', 1))): (1, 2, (), {})}
>>
>>
>> What I'd like to be able to do is something like this:
>>
>> def memo(func):
>>       def memofunc(*args, **kwargs):
>>           #
>>           # define a tuple consisting of values for all named positional
>>           # arguments occurring in the definition of func, including
>>           # default arguments if values are not given by the call, call
>>           # it named
>>           #
>>           # define another tuple consisting of any positional arguments
>>           # that do not correspond to named arguments in the definition
>>           # of func, call it anon
>>           #
>>           # define a third tuple consisting of pairs of names and values
>>           # for those items in kwargs whose keys are not named in the
>>           # definition of func, call it key
>>           #
>>           if (named, anon, key) in memofunc.d:
>>               return copy(memofunc.d[(named, anon, key)])
>>           memofunc.d[(named, anon, key)] = func(*args, **kwargs)
>>           return copy(memofunc.d[(named, anon, key)])
>>       memofunc.__name__ = func.__name__
>>       memofunc.d = {}
>>       return memofunc
>>
>>
>> But I don't know how. I know that I can see the default arguments of the
>> original function using func.__defaults__, but without knowing the
>> number and names of func's positional arguments (which I don't know how
>> to find out) this doesn't help me. Any suggestions?
>
> Possibly take a look at functools.lru_cache (which is Python 3.2+), and use the code from that (at it's part of the stdlib, someone must have done design and testing on it!). http://hg.python.org/cpython/file/default/Lib/functools.py

Thanks for your reply, but this decorator doesn't seem to attempt to get 
around the problem I mentioned. This is pasted from IDLE:

 >>> from functools import lru_cache
 >>> @lru_cache(None)
def f(x, y = None, *a, **k):
	return x, y, a, k

 >>> f(1, 2)
(1, 2, (), {})
 >>> f.cache_info()
CacheInfo(hits=0, misses=1, maxsize=None, currsize=1)
 >>> f(y = 2, x = 1)
(1, 2, (), {})
 >>> f.cache_info()
CacheInfo(hits=0, misses=2, maxsize=None, currsize=2)


-- 
Hate music? Then you'll hate this:

http://tinyurl.com/psymix