[Python-ideas] Decorators on loops

[Enric Tejedor]

Thank you for your replies,


El 08/01/14 13:08, Masklinn escribió:
>> That's a nice theory, but the basic form of the decorator wouldn't
>> work. Here's how decorators work on functions:
>>
>> @foo
>> def bar():
>>    pass
>>
>> is the same as:
>>
>> def bar():
>>    pass
>> bar = foo(bar)
>>
>> It depends on there being something assigned-to. With loops, that's
>> not the case, so it's not possible to decorate them in the usual
>> sense.
>>
>> Can you turn your loop into a map() call? Something like this:
>>
>> def loop_body(i):
>>    # all the code for your loop body
>> list(map(loop_body, range(10)))
>>
>> Once you have it in that form, you can use multiprocessing.Pool() and
>> its map() method, which will parallelize the loop for you (by
>> distributing it over a pool of subprocesses). Would that cover what
>> you need?
>>     
> Alternatively, wrap the loop in a function and then do AST munging in
> the decorator. Something similar (in spirit at least) to Numba
> (http://numba.pydata.org).
>
> You could even do something like immediate function invocation in the
> decorator, and bind the result to the function name, although I'm not
> sure your coworkers will like you.
>
>   

I would use this feature as a part of a parallel programming model for
Python apps.

Ideally, the programmer would place a decorator before their loops in
order to parallelize them, similarly to OpenMP and its pragmas.

Yes, I could make the programmer wrap the body of their loops in
functions and then decorate those functions:

# decorator of my PM library
def parallel ( iterable ):
     def call ( func ):
          # parallelize the iterations here, maybe with multiprocessing
and map for local execution, or another strategy for remote execution
     return call


# user's code
@parallel ( range ( count ) )
 def loop (i):
       # loop body


But this solution requires programmers to modify the loops they want to
parallelize, and not simply place a decorator before them, like this:

@parallel
for i in range(count):
     # loop body


> In the case of function and class decorators, Python has an object that 
> can be passed to the decorator: the function or the class.  For a loop 
> decorator, how would you "have access to the loop body"?  It sounds like 
> it would have to be compiled differently, into a separate code object?
>
>
>
>
>
> --Ned.
>
>   

Yes, perhaps when a loop had a decorator, the loop body could be
encapsulated and compiled as a function (similar to the "loop" function
I wrote), and that function object would be received by the decorator,
along with an iterable object that represents the iteration space. All
this would be hidden from the programmer, who would only decorate a
regular loop.


Thanks!

Enric



WARNING / LEGAL TEXT: This message is intended only for the use of the
individual or entity to which it is addressed and may contain
information which is privileged, confidential, proprietary, or exempt
from disclosure under applicable law. If you are not the intended
recipient or the person responsible for delivering the message to the
intended recipient, you are strictly prohibited from disclosing,
distributing, copying, or in any way using this message. If you have
received this communication in error, please notify the sender and
destroy and delete any copies you may have received.

http://www.bsc.es/disclaimer