[Python-Dev] Free threading

[Guido van Rossum]

> I'm asking why all of the Python interpreters in a particular process
> have to share *anything*. My naive view is that avoiding process-wide
> data is just good practice in any library. Of course processes that WANT
> to share data must share a GIL until Python is free-threaded.

They share anything that's implemented as a global resource.  This
includes the caches and free lists for small strings and ints, the
'intern' database, etc.

> For my own understanding I am curious, do "unrelated" Python
> interpreters share state:
> 
>  a) because that makes the implementation of the Python interpreter
> easier or 
>  b) because it makes it easier to write extending and embedding apps
> (don't have to pass around a context object)?
>  c) that's just how it evolved.

All of the above.  Otherwise, *every* API would have to carry an
"interp" argument around (as does Tcl/Tk).

> > ... There are plenty of other ways to deal with it,
> > including pre-forked longer-running processes -- and remember, it's
> > only necessary if you need more than one CPU just for running your
> > Python code!  (E.g. Zope, which has a perfectly good Zope-specific
> > solution if you need multiple CPUs: ZEO.)
> 
> Yes, you can always work around it. But any non-reentrant libary can be
> worked around. The only difference is that Python is a language so the
> library provides an inherent mechanism for working around the issue. I
> admit that in most cases this is a strong enough practical argument.

Sorry, I don't understand what you mean.  What does Python being a
language buy you?

> >...
> > Good question.  I've never used mmap myself. :-)  I know Unix shared
> > memory has locks and semaphores; the mmap module apparently doesn't
> > (possibly because Windows has a different philosophy there).
> 
> So do you know of any fast, portable mechanism for cross-process
> locking? That's the only thing between us and a nice queue-and-marshal
> based IPC mechanism!

Sorry, don't look at me.  On Unix, there's something referred to as
"System V IPC".  For Windows, look at the various Win32 API calls to
create locks -- I believe you can name them and the names are global.

--Guido van Rossum (home page: http://www.python.org/~guido/)