Easier way to save result of a function?

[Alex Martelli]

James Mitchelhill <james at disorderfeed.net> wrote:

> Sorry for the clunky subject line - I have a feeling that not knowing
> the proper terms for this is part of my problem.
> 
> I'm trying to write a class that analyses some data. I only want it to
> do as much work as necessary, so it saves method results to a
> dictionary, like so:
> 
> 
> class MyClass:
>     def __init__(self):
>         self.data = {}
> 
>     def doSomething(self):
>         return self.data.setdefault("someresult", self._doSomething())
> 
>     def _doSomething(self):
>          # heavy processing here
>          result = 1
>          return result
> 
>     def doMore(self):
>         return self.data.setdefault("moreresult", self._doMore())
> 
>     def _doMore(self):
>         # more heavy processing here
>         return self.doSomething() + 1
> 
> 
> This also seems kind of clunky. Is there a better way to acheive this?
> Or is this the usual idiom for this kind of thing? I've looked at
> decorators as they seem like they could be used for this, but I've no
> idea how exactly, or even if they're the appropriate tool.

No, this code is calling the heavy-processing functions
*unconditionally* -- Python's semantics is to compute every argument of
a function or method before starting to execute the code of that
function or method (almost all languages behave that way, save a very
few "pure" functional languages!), and that applies to the setdefault
methods of dictionaries too.

What you want to do is called "memoization" (sometimes "caching", but
that's a very generic term used in a bazillion different situations and
you may get overwhelmed by search results if you try searching for
it!-), and you can find plenty of recipes for it in the Python Cookbook
and elsewhere on the net.

For your needs, I would suggest first defining your class "normally"
(writing the methods without memoization) and then looping on all
methods of interest decorating them with automatic memoization; I just
suggested a similar kind of looping for decoration purposes in my very
latest post on another thread;-).  You won't use the _syntax_ of
decorators, but, hey, who cares?-)

Assuming (for simplicity but with no real loss of conceptual generality)
that all of your heavy-processing methods have names starting with 'do',
and that each of them takes only 'self' as its argument, you could for
example do something like:

def memoizeMethod(cls, n, m):
  def decorated(self):
    if n in self._memo: return self._memo[n]
    result = self._memo[n] = m(self)
    return result
  decorated.__name__ = n
  setattr(cls, n, decorated)  
    
def memoizeClass(cls):
  cls._memo = {}
  for n,m in inspect.getmembers(cls, inspect.ismethoddescriptors):
    if not n.startswith('do'): continue
      memoizeMethod(cls, n, m)

Now just write your MyClass with the doThis doThat methods you want, and
right after the class statement add

memoizeClass(MyClass)


Voila, that's it (untested details, but the general idea is sound;-).


Alex