Stackless, thread paradigm and C extensions

[Michael Hudson]

"Frederic Giacometti" <frederic.giacometti at arakne.com> writes:

> "Michael Hudson" <mwh at python.net> wrote in message
> news:u8zdaodu4.fsf at python.net...
> > "Frederic Giacometti" <frederic.giacometti at arakne.com> writes:
[...]
> > So when Python code executes
> >
> >    sock.recv(data)
> >
> > the interpreter would take a thread form the pool, and in effect say
> > "here, run this function".  Then the interpreter thread would go off
> > and execute pending Python threads, and when the sock.recv call
> > returned, it would add the thread that called it back to the set of
> > pending interpreter threads?  (I think this would be easier to discuss
> > with paper and pencil...)
> >
> > What does this buy us again?  It still makes writing C code that calls
> > Python code a bit of a pain, doesn't it?  Oh, maybe not.  More
> > thinking required...
> 
> A possible implementation would consist in using the method flag to mark C
> python methods/functions to run outside the interpreter thread. This way,
> the entire function runs in a separate thread.

method flag?

> The programmer does not have to explicitely manipulate OS threads at any
> point; the continuation/OS thread correspondance would be taken care of
> automatically by the threaded stackless implementation.
> One would have parallel threads of execution through just continuations.
> C extensions would run entirely either in a parallel thread, or in the
> interpreter thread. There would be no more interpreter lock; and when a C
> thread needs some service by the interpreter, it would just post a
> continuation to the interpreter scheduler.

This is what I thought you meant.  Good.

I think you could write code that would use an OS thread for every
other level of recursion (think of printing deeply nested data
structures that have Python __str__ methods, for instance).  I'm not
sure that's a good idea.  I also think of other situations in which
you'd end up with a lot of OS threads sitting around.  Is having a lot
of blocked OS threads around a problem?

> This will simplify programming when using Python callbacks, too;
> since the currently active Python continuation would be the
> continuation active in the thread.

I think there are too many definite articles in that sentence for me
to make head or tail of it.

Interesting idea.  I wonder if anyone has the time to implement it...

Cheers,
M.

-- 
  My hat is lined with tinfoil for protection in the unlikely event
  that the droid gets his PowerPoint presentation working.
                               -- Alan W. Frame, alt.sysadmin.recovery