[Cython] OpenMP support

[Matej Laitl]

> On a slightly higher level, are we just trying to use OpenMP from
> Cython, or are we trying to build it into the language? If the former,
> it may make sense to stick closer than one might otherwise be tempted
> in terms of API to the underlying C to leverage the existing
> documentation. A library with a more Pythonic interface could perhaps
> be written on top of that. Alternatively, if we're building it into
> Cython itself, I'd it might be worth modeling it after the
> multiprocessing module (though I understand it would be implemented
> with threads), which I think is a decent enough model for managing
> embarrassingly parallel operations. The above code is similar to that,
> though I'd prefer the for loop implicit rather than as part of the
> worker method (or at least as an argument). If we went this route,
> what are the advantages of using OpenMP over, say, pthreads in the
> background? (And could the latter be done with just a library + some
> fancy GIL specifications?) One thing that's nice about OpenMP as
> implemented in C is that the serial code looks almost exactly like the
> parallel code; the code at http://wiki.cython.org/enhancements/openmp
> has this property too.

+1.

I'm strongly for implementing thin and low-level support for OpenMP at the 
first place instead of (ab?)using it to implement high-level threading API.

Also, code like that would have an advantage (significant for my project[1]) of 
being compilable by older cython / interpretable by python with no cython at 
all ("pure" pure-python mode):

#pragma omp parallel for private(var1) reduction(+:var2) schedule(guided)
for i in range(n):
    do_work(i)

[1] http://github.com/strohel/PyBayes

Also, the implementation should be straightforward, I use to patch generated 
.c manually and only python → c variable name translation is needed, + perhaps 
taking care of temp variables that need to be thread-local.

My five cents,
                      Matěj Laitl