2.6, 3.0, and truly independent intepreters

[Rhamphoryncus]

On Oct 24, 1:02 pm, Glenn Linderman <v+pyt... at g.nevcal.com> wrote:
> On approximately 10/24/2008 8:42 AM, came the following characters from
> the keyboard of Andy O'Meara:
>
> > Glenn, great post and points!
>
> Thanks. I need to admit here that while I've got a fair bit of
> professional programming experience, I'm quite new to Python -- I've not
> learned its internals, nor even the full extent of its rich library. So
> I have some questions that are partly about the goals of the
> applications being discussed, partly about how Python is constructed,
> and partly about how the library is constructed. I'm hoping to get a
> better understanding of all of these; perhaps once a better
> understanding is achieved, limitations will be understood, and maybe
> solutions be achievable.
>
> Let me define some speculative Python interpreters; I think the first is
> today's Python:
>
> PyA: Has a GIL. PyA threads can run within a process; but are
> effectively serialized to the places where the GIL is obtained/released.
> Needs the GIL because that solves lots of problems with non-reentrant
> code (an example of non-reentrant code, is code that uses global (C
> global, or C static) variables – note that I'm not talking about Python
> vars declared global... they are only module global). In this model,
> non-reentrant code could include pieces of the interpreter, and/or
> extension modules.
>
> PyB: No GIL. PyB threads acquire/release a lock around each reference to
> a global variable (like "with" feature). Requires massive recoding of
> all code that contains global variables. Reduces performance
> significantly by the increased cost of obtaining and releasing locks.
>
> PyC: No locks. Instead, recoding is done to eliminate global variables
> (interpreter requires a state structure to be passed in). Extension
> modules that use globals are prohibited... this eliminates large
> portions of the library, or requires massive recoding. PyC threads do
> not share data between threads except by explicit interfaces.
>
> PyD: (A hybrid of PyA & PyC). The interpreter is recoded to eliminate
> global variables, and each interpreter instance is provided a state
> structure. There is still a GIL, however, because globals are
> potentially still used by some modules. Code is added to detect use of
> global variables by a module, or some contract is written whereby a
> module can be declared to be reentrant and global-free. PyA threads will
> obtain the GIL as they would today. PyC threads would be available to be
> created. PyC instances refuse to call non-reentrant modules, but also
> need not obtain the GIL... PyC threads would have limited module support
> initially, but over time, most modules can be migrated to be reentrant
> and global-free, so they can be used by PyC instances. Most 3rd-party
> libraries today are starting to care about reentrancy anyway, because of
> the popularity of threads.

PyE: objects are reclassified as shareable or non-shareable, many
types are now only allowed to be shareable.  A module and its classes
become shareable with the use of a __future__ import, and their
shareddict uses a read-write lock for scalability.  Most other
shareable objects are immutable.  Each thread is run in its own
private monitor, and thus protected from the normal threading memory
module nasties.  Alas, this gives you all the semantics, but you still
need scalable garbage collection.. and CPython's refcounting needs the
GIL.


> > Our software runs in real time (so performance is paramount),
> > interacts with other static libraries, depends on worker threads to
> > perform real-time image manipulation, and leverages Windows and Mac OS
> > API concepts and features.  Python's performance hits have generally
> > been a huge challenge with our animators because they often have to go
> > back and massage their python code to improve execution performance.
> > So, in short, there are many reasons why we use python as a part
> > rather than a whole.
[...]
> > As a python language fan an enthusiast, don't let lua win!  (I say
> > this endearingly of course--I have the utmost respect for both
> > communities and I only want to see CPython be an attractive pick when
> > a company is looking to embed a language that won't intrude upon their
> > app's design).

I agree with the problem, and desire to make python fill all niches,
but let's just say I'm more ambitious with my solution. ;)