Stackless, thread paradigm and C extensions

[Frederic Giacometti]

"Gordon McMillan" <gmcm at hypernet.com> wrote in message
news:Xns91518EE46939Bgmcmhypernetcom at 199.171.54.215...
> Frederic Giacometti wrote:
>
> In Stackless, the Python stack and the C stack are completely separate.
> In the case where C code needs to call Python and then do something with
> the result (that is, where tail recursion doesn't apply), Christian used
> the trick of manufacturing a Python-style frame object that represents the
C
> code. As far as (Stackless) Python is concerned, it gets dispatched like
> any other frame (Stackless' Python "stack" is really a tree).

That sounds to me that the current Python thread model could also be
reengineered with stackless, profitably.

Here is a possible direction:

*** Replacement of the Python lock with a Python dedicated thread ***

Instead of having the Python VM running in multiple threads sequentially, by
means of the Pythonsynchronization lock, the Python VM would run in one
dedicated thread (on single thread build, this would just be the main
thread) - this is just a reversal of paradigm -..

Then:
  - stackless would manage the micro-threads within the (unique) python
thread
  - there is no more need for a global python lock (and performance would
boost); stackless managing on its own its continuation list, and the event
queues from other threads and C functions.
  - C code can run independently in other threads, using standard
inter-thread communication mechanisms to communicate with the Python VM
(just like the lock; but then only the 'C client functions' have to hang on
the interrupt; while Python stackless would run uninterrupted, unlike in the
current system).

On other terms:
   - the Python VM would run in one single thread (presently, the Python VM,
by means of the Python lock, already runs in a single-threaded fashion)
  - the Python thread would act as server thread to the other threads
requesting calls to Python.
  - this would get rid to the Python lock synchronization overhead, and
somehow simplify the current multithreaded approach
  - the concept of stackless microthread would be conciliated with the
current Python thread objects, that reflect the OS native threads.

This would impact the current threading/thread modules: then a Python thread
object would be a handle to the OS thread were the C code would be run;
while all Python bytecode and reference operations would be performed by
stackless in THE Python VM thread.

Does this make sense?

Thanks,

FG