C API design flaw (was: Re: Multiple independent Python interpreters in a C/C++ program?)

[Rhamphoryncus]

On Apr 12, 2:02 pm, sturlamolden <sturlamol... at yahoo.no> wrote:
> On Apr 12, 7:05 pm, sturlamolden <sturlamol... at yahoo.no> wrote:
>
> > In theory, a GIL private to each (sub)interpreter would make Python
> > more scalable. The current GIL behaves like the BKL in earlier Linux
> > kernels. However, some third-party software, notably Apache's
> > mod_python, is claimed to depend on this behaviour.
>
> I just looked into the reason why ctypes, mod_python, etc. depend on a
> shared GIL. Well ... PyGILState_Ensure() does not take an argument, so
> it does not know which interpreter's GIL to acquire. Duh!
>
> The sad fact is, the functions in Python's C API does not take the
> interpreter as an argument, so they default to the one that is
> currently active (i.e. the one that PyThreadState_Get()->interp points
> to). This is unlike Java's JNI, in which all functions take the
> "environment" (a Java VM instance) as the first argument.
>
> IMHO, I consider this a major design flaw in Python's C API. In a more
> well thought API, PyGILState_Ensure would take the interpreter
> returned by Py_NewInterpreter as an argument, and thus know the
> interpreter with which to synchronize.
>
> I complained about this before, but the answer I got was that the
> simplified GIL API would not work if interpreters has a separate GIL.
> Obviously it would not work as long as the PyGILState_Ensure does not
> take any arguments. The lack of a leading VM argument in
> PyGILState_Ensure, and almost every function in Python's C API, is the
> heart of the problem.

Alternatively, the multiple interpreter API could be ripped out, and
you could just use separate threads.  Do you have a use case that
requires it?