[Python-ideas] 'Injecting' objects as function-local constants

[Arnaud Delobelle]

On 11 Jun 2011, at 14:30, Jan Kaliszewski wrote:

> == Use cases ==
> 
> A quite common practice is 'injecting' objects into a function as its
> locals, at def-time, using function arguments with default values...
> 
> Sometimes to keep state using a mutable container:
> 
>    def do_and_remember(val, verbose=False, mem=collections.Counter()):
>        result = do_something(val)
>        mem[val] += 1
>        if verbose:
>            print('Done {} times for {!r}'.format(mem[val], val))
> 
> Sometimes, when creating functions dynamically (making use of nested
> scopes), e.g. to keep some individual function features (usable within
> that functions):
> 
>    def make_my_callbacks(callback_params):
>        my_callbacks = []
>        for params in callback_params:
>            def fun1(*args, _params=params, **kwargs):
>                "...do something with args and params..."
>            def fun2(*args, _params=params, **kwargs):
>                "...do something with args and params..."
>            def fun3(*args, _fun1=fun1, _fun2=fun2, **kwargs):
>                """...do something with args and with functions fun1, fun2,
>                for example pass them as callbacks to other functions..."
>            my_callbacks.append((fun1, fun2, fun3))
>        return my_callbacks
> 
> Sometimes simply to make critical parts of code optimised...
> 
>    def do_it_quickly(fields, _len=len, _split=str.split,
>                      _sth=something):
>        return [_len(f), _split(f), _sth(f) for f in fields]
> 
> ...or even for readability -- keeping function-specific constants within
> the function definition:
> 
>    def check_value(val,
>                    VAL_REGEX=re.compile('^...$'),
>                    VAL_MAX_LEN=38):
>        return len(val) <= VAL_MAX_LEN and VAL_RE.search(val) is not None
> 
> In all that cases (and probably some other too) that technique appears
> to be quite useful.
> 
> 
> == The problem ==
> 
> ...is that it is not very elegant. We add arguments which:
> a) mess up function signatures (both in the code and in auto-generated docs);
> b) can be incidentally overriden (especially when a function has an "open"
>   signature with **kwargs).
> 
> 
> == Proposed solutions ==
> 
> I see three possibilities:
> 
> 1.
> To add a new keyword, e.g. `inject':
>    def do_and_remember(val, verbose=False):
>        inject mem = collections.Counter()
>        ...
> or maybe:
>    def do_and_remember(val, verbose=False):
>        inject collections.Counter() as mem
>        ...
> 
> 2. (which personally I would prefer)
> To add `dummy' (or `hidden') keyword arguments, defined after **kwargs
> (and after bare ** if kwargs are not needed; we have already have
> keyword-only arguments after *args or bare *):
> 
>    def do_and_remember(val, verbose=False, **, mem=collections.Counter()):
>        ...
> 
> do_and_remember(val, False, mem='something') would raise TypeError and
> `mem' shoudn not appear in help() etc. as a function argument.
> 
> 3.
> To provide a special decorator, e.g. functools.within:
>    @functools.within(mem=collections.Counter())
>    def do_and_remember(val, verbose=False):
>        ...

That's hard to do as (assuming the function is defined at the global scope), mem will be compiled as a global, meaning that you will have to modify the bytecode.  Oh but this makes me think about something I wrote a while ago (see below).


4. Use closures.

def factory(mem):
     def do_and_remember(val, verbose=False)
         result = do_something(val)
         mem[val] += 1
         if verbose:
             print('Done {} times for {!r}'.format(mem[val], val))         ....
     return do_and_remember
do_and_remember = factory(mem=collections.Counter())

Added bonus: you can create many instances of do_and_remember.


----------

Related to this, here's a "localize" decorator that I wrote some time ago for fun (I think it was from a discussion on this list).  It was for python 2.x (could easily be modified for 3.x I think, it's a matter of adapting the attribute names of the function object).  It "freezes" all non local variables in the function.  It's a hack! It may be possible to adapt it.

def new_closure(vals):
    args = ','.join('x%i' % i for i in range(len(vals)))
    f = eval("lambda %s:lambda:(%s)" % (args, args))
    return f(*vals).func_closure

def localize(f):
    f_globals = dict((n, f.func_globals[n]) for n in f.func_code.co_names)
    f_closure = ( f.func_closure and
                  new_closure([c.cell_contents for c in f.func_closure]) )
    return type(f)(f.func_code, f_globals, f.func_name,
                   f.func_defaults, f_closure)

# Examples of how localize works:

x, y = 1, 2
@localize
def f():
    return x + y

def test():
    acc = []
    for i in range(10):
        @localize
        def pr(): print i
        acc.append(pr)
    return acc

def lambdatest():
    return [localize(lambda: i) for i in range(10)]

# These examples will behave as follows:
>>> f()
3
>>> x = 3
>>> f()
3
>>> pr = test()
>>> pr[0]()
0
>>> pr[5]()
5
>>> l = lambdatest()
>>> l[2]()
2
>>> l[7]()
7
>>> 


-- 
Arnaud