GIL in the new glossary

[Andrew Dalke]

Dave Kuhlman:
> From within Twisted-1.0.8alpha3, I did:

Yup, I was refering to the cReactor, which is
an optional performance improvement module
just like cStringIO/StringIO, cPickle/pickle, and
a few other bits.

That Twisted can be run in pure Python is
different than that Twisted contains some C
code.

>  Isn't what matters is
> whether Twisted does enough processing where it gives up control
> during the execution of C/C++ code so that other processes and
> processors can also do work in parallel?  I suppose I'd better ask
> this question over on the Twisted-users list.  Although, their
> oppinions might be just a little biased.

When the reactor gets an event (data from a socket,
GUI event, timeout, etc), it calls the associated handler.
That handler has complete control.  It may add/remove
handlers from the reactor, and modify what the event
sources are.  When it finishes, control is returned to the
reactor, which waits for the next event.

This has nothing to do with threads.  Nothing in the
above is in parallel.

Twisted also supports threads.  The handler can start
its own thread and let the thread do whatever it wants,
and in the controlling thread it finishes up and lets the
reactor carry on.

The new thread is Python code and has the same GIL
advantages and disadvantages as any other Python
library.  Twisted does nothing to change that.  If the
thread doesn't release the GIL, then the reactor will
not run.

A very common use of threads is to spin off a new
thread, let it do stuff, and return the results back to
the calling thread.  Twisted has a nice way to make
that look exactly like any other Twisted Defered,
so it's very consistent and using simple threads becomes
trivial.

What the Twisted folk will rightly point out is that
most applications are event oriented, not compute
oriented.  By recasting your design into an event-based
framework you end up with a single thread program
which can still handle many different task, and do
so without need for any explicit task management.

But if you do want to do several computations at
the same time, and they are in Python, then there's
nothing Twisted can do to give you speedup on
multiprocessor machines.

(Err, I think there's some way to Twisted to start
multiple processes and communicate between them,
letting Twisted handle the serialization and data
passing issues, but that's more than I know about
the system.  If it exists, all it does is provide a way
to simplify the solution people already use to get
around the GIL, and not a "true" solution to the
problem.)

                    Andrew
                    dalke at dalkescientific.com