[IPython-dev] Musings: syntax for high-level expression of parallel (and other) execution control

[Darren Dale]

Hi Prabhu,

On Sun, Sep 6, 2009 at 3:14 AM, Prabhu
Ramachandran<prabhu at aero.iitb.ac.in> wrote:
> On 09/06/09 11:52, Fernando Perez wrote:
>>
>> But don't beat yourself up: while there's nothing complex about what
>> we're talking about, it is a slightly unusual usage of the language,
>> so it's natural to do a double-take with it.  I have the benefit of
>> having worried about this problem for a long time, but it took me
>> *many* tries to understand how to fit the pieces together.  And I had
>> the advantage of lots of help along the way:
>
> [...]
>>
>> - and finally, John Siracusa's review at Ars Technica about Apple's
>> work with anonymous blocks and Grand Central Dispatch make the whole
>> thing click.
>>
>> As you can see, if you're slow for taking a day to put it together,
>> there's simply no hope for me: it took me almost 2 years, and I needed
>> the help of some of the very brightest people in the python world to
>> push me along.  They are the ones who did all the thinking and deserve
>> the credit, I was just thick enough never to understand the ideas
>> until now!
>
> Thanks for the interesting links and thread.  Just FYI, last year I had
> occasion to solve, relatively elegantly, a set of pretty sticky problems for
> mayavi2 using decorators and generators.  See for example:
>
> https://svn.enthought.com/enthought/browser/Mayavi/trunk/enthought/mayavi/tools/show.py
> https://svn.enthought.com/enthought/browser/Mayavi/trunk/enthought/mayavi/tools/animator.py
>
> They allow us to do relatively simple but neat things very elegantly. Until
> you use decorators you often don't realize how convenient they can be.  In
> particular the pattern used in animator.py shows how convenient the
> combination of a UI dispatch mechanism plus a generator is.  All a
> programmer needs is to inject a yield suitably and the rest is automatic.
>
> The Kamaelia project (http://www.kamaelia.org) is also very interesting for
> its use of generators, microprocesses, components and very specifically
> concurrency.  They have a very neat model for exactly this and a nice though
> slightly elaborate tutorial showing how you can build their core library
> from these ideas.  If you have the time it is very interesting.
>
> My humble contribution to spending two more hours of your time. ;-)

Thanks for the pointer. The Pipeline and Graphline reminds me of
LabView, or rather how I would prefer to develop labview-like
applications in python.

I had to look in the older document structure to find a discussion on
concurrency, here it is for anyone who is interested:
http://www.kamaelia.org/Docs/Axon/Axon.Microprocess.html