Status of Python threading support (GIL removal)?

[Tyler Laing]

This is a very long-running issue, that has been discussed many times. Here
are the two sides to keeping the gil or removing it:

Remove the GIL:

   - True multi-threaded programming
   - Scalable performance across a multi-core machine
   - Unfortunately, this causes a slow-down in single core/thread
   performance
   - Most of the slow down comes from Python's Reference counting, as every
   function is supposed to increase and decrease the references... which leads
   to a lot of synchronization and checking of locks.
   - It makes the core interpreter much more complex

Keeping the GIL:

   - Processes should be used instead of threads. Look at Erlang, THE model
   of concurrency. It uses a process model for concurrency and message passing,
   rather than threads. This means you avoid deadlocks, livelocks, race
   conditions(not entirely, but they are reduced)
   - C extensions can still gain really impressive multi-threaded
   performance. My C extension, a wrapper around ffmpeg, releases the GIL for
   90% or so of its processing time, so all other threads are still extremely
   responsive.
   - It allows python to stay simple, understandable, and offer good
   performance, if you know how to use it properly.

Basically the two sides are that you it ends up slowing things down, it
makes the interpreter more complex vs you should use processes instead, and
be smart about how you use the GIL.

-Tyler
On Fri, Jun 19, 2009 at 2:52 AM, Jure Erznožnik <jure.erznoznik at gmail.com>wrote:

> See here for introduction:
>
> http://groups.google.si/group/comp.lang.python/browse_thread/thread/370f8a1747f0fb91
>
> Digging through my problem, I discovered Python isn't exactly thread
> safe and to solve the issue, there's this Global Interpreter Lock
> (GIL) in place.
> Effectively, this causes the interpreter to utilize one core when
> threading is not used and .95 of a core when threading is utilized.
>
> Is there any work in progess on core Python modules that will
> permanently resolve this issue?
> Is there any other way to work around the issue aside from forking new
> processes or using something else?
> --
> http://mail.python.org/mailman/listinfo/python-list
>



-- 
Visit my blog at http://oddco.ca/zeroth/zblog
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20090619/fd983a2d/attachment-0001.html>