Multiple interpreters and the global lock

[Tim Peters]

[Daniel Dittmar]
> Problem:
>
> One topic that occasionally crops up is the wish to keep
> multiple interpreters completely separated.  The standard
> answer is a reference to the 'global lock', which allows
> only one interpreter to run at a time.  This of course
> limits scalability of Python on the server side, especially
> on multi processor machines.

It's easy to keep multiple interpreters completely separate, with no
scalability hassles:  if you want N separate interpreters, fire up N
distinct copies of Python!  There's no problem if you keep the distinction
at the process level.  The global lock only affects multiple interpreters
and/or multiple threads running within a single process (it's a
process-global lock, not a system-global lock).

> While searching DejaNews for the topic, I found that GvR is
> not opposed to removing the lock, provided that there is no
> negative impact on stability and performance.
>
> What I hope to get as answers:
> - is it possible?
> - would patches be accepted, welcomed, frenetically applauded?

You've already been pointed wisely to Greg Stein, and see the Thread-SIG
archives where this comes up periodically.  Greg's thrust is toward
"free-threading", though, which, while it implies no global lock, must deal
with all the thread issues.  If you run separate processes, it's irrelevant.

> - what would be the smoothest path to completition, breaking
>   the least code?

It's unclear that you're asking for free-threading.  If you are, it's not
easy (for example, the refcount issues you dismiss later can't be ignored in
free-threading); if you aren't, what's wrong with running separate processes
(in which case the code is already complete -- it comes with the OS <0.7
wink>)?

> ...
> Disclaimer:
>
>    The cause for this post is not to advance mankind, but to
> advance a commercial product.

Relax:  If the commerical product doesn't advance mankind either, it won't
sell <wink>.

writing-the-os-in-python-is-much-harder-than-writing-a-little-ipc-ly
    y'rs  - tim