[Cython] OpenMP support
Stefan Behnel
stefan_ml at behnel.de
Fri Mar 11 12:37:50 CET 2011
Dag Sverre Seljebotn, 11.03.2011 08:56:
> Basically, I'm +1 to anything that can make me
> pretend the GIL doesn't exist, even if it comes with a 2x performance hit:
> Because that will make me write parallell code (which I can't be bothered
> to do in Cython currently), and I have 4 cores on the laptop I use for
> debugging, so I'd still get a 2x speedup.
>
> Perhaps the long-term solution is something like an "autogil" mode could
> work where Cython automatically releases the GIL on blocks where it can
> (such as a typed for-loop), and acquires it back when needed (an
> exception-raising if-block within said for-loop).
I assume you mean this to become a decorator or other option written into
the code.
> And when doing
> multi-threading, GIL-requiring calls are dispatched to a master GIL-holding
> thread (which would not be a worker thread, i.e. on 4 cores you'd have 4
> workers + 1 GIL-holding support thread). So the advice for speeding up code
> is simply "make sure your code is all typed", just like before, but people
> can follow that advice without even having to learn about the GIL.
The GIL does not only protect the interpreter core. It also protects C
level data structures in user code and keeps threaded code from running
amok. Releasing and acquiring it doesn't come for free either, so besides
likely breaking code that was not specifically written to be reentrant,
releasing it automatically may also introduce a performance penalty for
many users.
I'm very happy the GIL exists, and I'm against anything that tries to
disable it automatically. Threading is an extremely dangerous programming
model. The GIL has its gotchas, too, but it still simplifies it quite a
bit. Actually, threading is so complex and easy to get wrong, that any
threaded code should always be written specifically to support threading.
Explicitly acquiring and releasing the GIL is really just a minor issue on
that path.
Stefan
More information about the cython-devel
mailing list