[Cython] OpenMP support

[Sturla Molden]

Den 08.03.2011 18:00, skrev mark florisson:
> Sure, that's not what I was hinting at. What I meant was that the
> wrapper returned by the decorator would have to call the closure for
> every iteration, which introduces function call overhead.

OpenMP does this too, in addition to work-scheduling overhead. You don't 
see it in your C code, but it's there in the generated binary.


> Indeed. I guess we just have to establish what we want to do: do we
> want to support code with Python objects (and exceptions etc), or just
> C code written in Cython? If it's the latter, then I still think using
> OpenMP directly would be easier to implement and more convenient for
> the user than decorators with closures, but maybe I'm too focussed on
> OpenMP.

I can probably think of a hundred reasons why we want "OpenMP" in Cython 
and Python apart from multicore CPUs. It all comes down to the fact that 
threading.Thread, multiprocessing.Process, Java threads, pthreads, win32 
threads, .NET threads, etc., are not abstractions for concurrency that 
fits well with the human brain.

OpenMP is a nice way to "think parallel", not just for compute intensive 
code, but also I/O bound code: For example listen on a socket for a 
connection, then handle the connection. Then, put in some pragmas in the 
code (#pragma omp parallel sections, nowait, etc.), and you have a 
multithreaded server. Almost all the nastyness of parallel programming 
is deferred to the compiler. We can get concurrency by putting markups 
and small hints in the code, instead of writing code to manage everything.

Sturla