[Python-Dev] Free threading

[Guido van Rossum]

> [Guido]
> > ...
> > Anyway, I don't see the point: if you want two 100% independent
> > interpreters, you can get that easily by forking two separate
> > processes.  That's what separate processes are *for*...
> 
> So what is it that multiple interpreters within a single process *are* for?
> I'm comfortable with threads, and comfortable with processes, but multiple
> interpreters not-really-sharing yet not-really-isolated seem mostly an
> opportunity to track down subtle bugs <wink>.

It was always feature number one requested by folks coming from Tcl,
where creating multiple interpreters (threaded or not) is an
acceptable pastime for people of good manners.  But it seems to be
wasted on the average Pythoneer.

In Python, there's one good use I can think of: an application may
embed Python and occasionally want to run a good chunk of Python code.
If it needs each chunk of Python code to run completely independent
from each other chunk, creating a new interpreter for each chunk is a
good way to avoid that changes to e.g. sys.path or tabnanny's globals
made by one chunk affects the next chunk.  The hosting application
might want the execution to share memory with *it*, so these
interpreters have to live in the same process.

This works well if the next chunk is only started after the previous
chunk is done; but then using Py_Initialize() and Py_Finalize() is
enough (creating one interpreter at a time).  If the chunks need to
overlap in time, creating multiple interpreters is the right solution.

But apparently this habit has fallen in disuse -- from looking at the
code I believe there's a serious problem with exceptions, where
initializing the second interpreter messes up the identities of the
exception classes stored in global variables by the first interpreter,
and nobody has ever complained about this.  Or maybe they are still
using Python 1.5.2, which may not have the same problem.  Or maybe
this is the reason why PyApache has a reptabion of instability. :(

--Guido van Rossum (home page: http://www.python.org/~guido/)