[IPython-dev] Musings: syntax for high-level expression of parallel (and other) execution control

[Edward K. Ream]

On Fri, Sep 4, 2009 at 3:31 AM, Fernando Perez <fperez.net at gmail.com> wrote:

>
>
> The code below shows an implementation of a simple for  loop directly
> and via a decorator.  Both versions do the same thing, but the point
> is that by providing such decorators, we can *trivially* provide a
> GCD-style API for users to express their parallelism and have
> execution chunks handled by ipython remotely.
>

Fascinating.  Let me ask some basic questions to see if I understand.

1.  Both loops do:  results = [None]*count

Is synchronization needed to update this array?

2. You call the decorator @for_each.  Would @parallel be more descriptive?

3. The docstring for for_each is:

   """This decorator-based loop does a normal serial run.
   But in principle it could be doing the dispatch remotely,
   or into a thread pool, etc.
   """

So you are thinking that a call might be something like:

    def call(func):
        for i in iterable:
            << create thread >>

And these threads would place their computations in the global results?

Edward
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20090904/045b42ba/attachment.html>