[Cython] cython.parallel tasks, single, master, critical, barriers

[Dag Sverre Seljebotn]

On 10/20/2011 02:51 PM, mark florisson wrote:
> On 20 October 2011 10:35, Dag Sverre Seljebotn
> <d.s.seljebotn at astro.uio.no>  wrote:
>> On 10/20/2011 11:13 AM, mark florisson wrote:
>>>
>>> On 20 October 2011 09:42, Dag Sverre Seljebotn
>>> <d.s.seljebotn at astro.uio.no>    wrote:
>>>>
>>>> Meta: I've been meaning to respond to this thread, but can't find the
>>>> time.
>>>> What's the time-frame for implementing this? If it's hypothetical at the
>>>> moment and just is a question of getting things spec-ed, one could
>>>> perhaps
>>>> look at discussing it at the next Cython workshop, or perhaps a Skype
>>>> call
>>>> with the three of us as some point...
>>>
>>> For me this is just about getting this spec-ed, so that when someone
>>> finds the time, we don't need to discuss it for weeks first. And the
>>> implementor won't necessarily have to support everything at once, e.g.
>>> just critical sections or barriers alone would be nice.
>>>
>>> Is there any plan for a new workshop then? Because if it's in two
>>> years I think we could be more time-efficient :)
>>
>> At least in William's grant there's plans for 2-3 Cython workshops, so
>> hopefully there's funding for one next year if we want to. We should ask him
>> before planning anything though.
>>
>>>> Regarding the tasks: One of my biggest problems with Python is the lack
>>>> of
>>>> an elegant syntax for anonymous functions. But since Python has that
>>>> problem, I feel it is not necesarrily something we should fix (by using
>>>> the
>>>> with statements to create tasks). Sometimes Pythonic-ness is more
>>>> important
>>>> than elegance (for Cython).
>>>
>>> I agree it's not something we should fix, I just think tasks are most
>>> useful in inline blocks and not in separate functions or closures.
>>> Although it could certainly work, I think it restricts more, leads to
>>> more verbose code and possibly questionable semantics, and on top of
>>> that it would be a pain to implement (although that should not be used
>>> as a persuasive argument). I'm not saying there is no elegant way
>>> other than with blocks, I'm just saying that I think closures are not
>>> the right thing for it.
>>>
>>>> In general I'm happy as long as there's a chance of getting things to
>>>> work
>>>> in pure Python mode as well (with serial execution). So if, e.g., with
>>>> statements creating tasks have the same effect when running the same code
>>>> (serially) in pure Python, I'm less opposed (didn't look at it in
>>>> detail).
>>>
>>> Yes, it would have the same effect. The thing with tasks (and OpenMP
>>> constructs in general) is that usually if your compiler ignores all
>>> your pragmas, your code just runs serially in the same way. The same
>>> would be true for the tasks in with blocks.
>>
>> Short note: I like the vision of Konrad Hinsen:
>>
>> http://www.euroscipy.org/talk/2011
>>
>> The core idea is that the "task-ness" of a block of code is orthogonal to
>> the place you actually write it. That is, a block of code may often either
>> be fit for execution as a task, or not, depending on how heavy it is (=
>> values of arguments it takes in, not its contents).
>>
>> He introduces the "async" expression to drive this point through.
>>
>> I think "with task" is fine if used in this way, if you simply call a
>> function (which itself doesn't know whether it is a task or not). But once
>> you start to implement an entire function within the with-statement there's
>> a code-smell.
>
> Definitely, do you'd just call the function from the task.
>
>> Anyway, it's growing on me. But I think his "async" expression is more
>> Pythonic in the way that it forces you away from making your code smell.
>>
>> We could simply have
>>
>> async(func)(arg, arg2, somekwarg=4)
>>
>
> That looks good. The question is, does this constitute an expression
> or a statement? If it's an expression, then you expect a meaningful
> return value, which means you're going to have to wait for the task to
> complete. That would be fine if you submit multiple tasks in one
> expression, from the slides:
>
>      max(async expr1, async expr2)
>
> or even
>
>      [async expr for ... in ...]
>
> I must say, it does look really elegant and it doesn't leave the user
> to question when the task is executed (and if you need a taskwait
> directive to wait for your variables to become defined). What I don't
> see is how to do the producer consumer trick, unless you regard using
> the result of async as a taskwait, and not using it as not having a
> taskwait, e.g.
>
> async func(...) # generate a task and don't wait for it
> result = async func(...) # generate a task and wait for it.
>
> The latter is not useful unless you have multiple expressions in one
> statement, so we should also allow result1, result2 = async
> func(data=a), async func(data=b).

I think the idea is that you have a transparent, implicit future. You 
block when you use the result; you are allowed to pass the result back 
to the caller without blocking, and the caller does not need to know 
whether it is a future or not.

Implemented in Python itself, the protocol would be something like 
INCREF/DECREF does not block, but all other operations do block.

Of course, this is rather hard to implement in present-day Cython. Options:

  a) Have async(func)(x) return a future, must call result().

  b) Make async part of the type spec, such as "cdef async int x". And 
coerce it to Python using a proxy. Seems messy, and going beyond what 
current Python semantics allow. But I do like it a bit better than 
explicit futures everywhere.

Dag Sverre