[concurrency] Reworking the GIL

[Michael Sparks]

On Sun, Nov 1, 2009 at 1:29 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> Hello,
>
> I've just noticed that this list exists (not that it seems very active).

FWIW, It's not been active because the list moved from google groups
to python.org.

(Rather more annoyingly the original group was also destroyed for no
good reason either).

As far as I can tell that killed discussion. [1] (It certainly put me
off using this list for anything :-( )

> So I thought I'd send you a link to the recent work I've done to improve
> the behaviour of the GIL, the results of which you can find in the SVN
> sandbox :

Can you explain what effect you're aiming to achieve with this change?

The reason I'm asking is because using kamaelia[2] it should be
possible to craft a bunch of useful test cases or benchmarks  - as
long as I know what I'm supposed to be testing :-)

[2] http://www.kamaelia.org/Home

For example, we mix threads in our code safely and happily at the
moment, what benefit should I expect to see from this change ?
(primarily a message passing setup, though we do use a form of
simplified software transactional memory as well)

Primarily we use co-operative scheduling of generator based
components, but threaded components are common as well to enable use
of blocking code, and to also allow the system as a whole to sleep.
(For example in network code, select runs in its own thread, which
wakes other parts of the system, and can itself block.

Similarly the generator scheduler sleeps in a thread - either
foreground or background - until something needs running) Similar
points go for GUI code etc as well. As a result having mixed workloads
is common, so if this reworking should have a benefit, we should be
able to practically test for in with kamaelia :-)

(The reason for using generators has historically good reasons
incidentally, but also has the side effect of encouraging the creation
of small reusable things and some handy side effects: primarily
because you can put lots of generators in threads, and threads in
processes, but doing it the other way is harder :-)

If there's anything you'd like me to test, please let me know. I can't
promise anything, but willing to help if I can.

Regards,


Michael.
--
[1] I find python.org mailing lists a real pain in the neck because
the mail server throws away perfectly valid mails from my mail server,
so I have to really want to send something if I want to send anything
- because it means I end up having to use google's web gmail interface
(rather than a decent mail client) which is a real pain for anything
other than occasional discussion.