[Cython] cython.parallel tasks, single, master, critical, barriers

[mark florisson]

On 12 October 2011 10:24, Dag Sverre Seljebotn
<d.s.seljebotn at astro.uio.no> wrote:
> On 10/12/2011 11:08 AM, Robert Bradshaw wrote:
>>
>> On Wed, Oct 12, 2011 at 1:36 AM, Dag Sverre Seljebotn
>>>
>>> I wouldn't resist a builtin "channel" type in Cython (since we don't have
>>> full templating/generics, it would be the only way of sending typed data
>>> conveniently?).
>>
>> zeromq seems to be a nice level of abstraction--we could probably get
>> far with a zeromq "overlay" module that didn't require the GIL. Or is
>> the C API easy enough to use if we could provide convenient mechanisms
>> to initialize the tasks/threads. I think perhaps the communication
>> model could be solved by a library more easily than the treading
>> model.
>
> Ah, zeromq even has an in-process transport, so should work nicely for
> multithreading as well.
>
> The main problem is that I'd like something like
>
> ctypedef struct Msg:
>    int what
>    double when
>
> cdef Msg msg
> cdef channel[Msg] mychan = channel[msg](blocking=True, in_process=True)
> with cython.parallel:
>    ...
>    if is_master():
>        mychan.send(what=1, when=2.3)
>    else:
>        msg = mychan.recv()
>
>
> Which one can't really do without either builtin support or templating
> support. One *could* implement it in C++...
>
> C-level API just sends char* around, e.g.,
>
> int zmq_msg_init_data (zmq_msg_t *msg, void *data, size_t size, zmq_free_fn
> *ffn, void *hint);

Actually I think fused types may be able to help here as well. E.g.
you could specify 'send' and 'recv' as cdef methods that based on the
type they get pack their data in a certain way (if you don't want
to/cannot go for the char * + sizeof(MyType) way). This means you have
to do have a branch in send and recv for every type you're going to
use, though, but it might still be more convenient than writing
different functions for every different type to pack your data.

> Dag Sverre
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>