Kill GIL

[Dave Brueck]

Mike Meyer wrote:
> aahz at pythoncraft.com (Aahz) writes:
> 
> 
>>In article <868y5t6sal.fsf at guru.mired.org>, Mike Meyer  <mwm at mired.org> wrote:
>>
>>>Here here. I find that threading typically introduces worse problems
>>>than it purports to solve.
>>
>>Threads are also good for handling blocking I/O.
> 
> 
> Actually, this is one of the cases I was talking about. I find it
> saner to convert to non-blocking I/O and use select() for
> synchronization. That solves the problem, without introducing any of
> the headaches related to shared access and locking that come with
> threads.

This whole tangent to the original thread intrigues me - I've found that if 
you're going to use threads in any language, Python is the one to use because 
the GIL reduces so many of the problems common to multithreaded programming (I'm 
not a huge fan of the GIL, but its presence effectively prevents a pure Python 
multithreaded app from corrupting the interpreter, which is especially handy for 
those just learning Python or programming).

I've done a lot of networking applications using select/poll (usually for 
performance reasons) and found that going that route *can* in some cases 
simplify things but it requires looking at the problem differently, often from 
perspectives that seem unnatural to me - it's not just an implementation detail 
but one you have to consider during design.

One nice thing about using threads is that components of your application that 
are logically separate can remain separate in the code as well - the 
implementations don't have to be tied together at some common dispatch loop, and 
a failure to be completely non-blocking in one component doesn't necessarily 
spell disaster for the entire app (I've had apps in production where one thread 
would die or get hung but I was relieved to find out that the main service 
remained available).

Another related benefit is that a lot of application state is implicitly and 
automatically managed by your local variables when the task is running in a 
separate thread, whereas other approaches often end up forcing you to think in 
terms of a state machine when you don't really care* and as a by-product you 
have to [semi-]manually track the state and state transitions - for some 
problems this is fine, for others it's downright tedious.

Anyway, if someone doesn't know about alternatives to threads, then that's a 
shame as other approaches have their advantages (often including a certain 
elegance that is just darn *cool*), but I wouldn't shy away from threads too 
much either - especially in Python.

-Dave

* Simple case in point: a non-blocking logging facility. In Python you can just 
start up a thread that pops strings off a Queue object and writes them to an 
open file. A non-threaded version is more complicated to implement, debug, and 
maintain.