[concurrency] Issues with the current concurrency mechanisms in Python

[Mike Meyer]

On Wed, 15 Feb 2012 09:47:37 +0200
Christopher Reay <christopherreay at gmail.com> wrote:
> I think the core of this discussion (which has lost all steam, verve, fun
> and substance since it was kicked out of python-ideas) is:

Yes, it has lost that. I think all the people who want to spout off
but not do work left the discussion.

From my viewpoint, there are four concurrent (sorry) issues. This is
one of them.

>    1. How can python support multiple external processes interacting with a
>    single set of states (i.e. an interpreter instance) given that the GIL
>    exists?
>    2. How can we improve sharing of objects between interpreter instances?
> Does this about sum up where we are?

That's the "processes" part of the discussion.

The second one is - can we "fix" threading to not have the problems it
has?

Summarizing that problem:

1) Memory management is so hard that so few programmers get it right
   that most modern languages simply don't let them do it.
2) Concurrent access to mutable objects is *much*, *much* harder than
   memory management.
3) Threading (whether done with OS or userland threads) expose every object
   in the program to concurrent access.

The languages I've dealt with that fixed this did so by making mutable
objects rare, and requiring special tools to mutate them. I don't
think that's going to fly in Python.

Which leaves tweaking the interpreter so that threads no longer expose
most objects to concurrent access. My start on this is a tool to do
static analysis to figure out which objects can't be exposed to
concurrent access at all, since that 1) provides a start on auditing
the standard library for thread safety, and 2) gives us a handle how
big the problem really is. All I have to do is find time to work on
it...

    <mike
-- 
Mike Meyer <mwm at mired.org>		http://www.mired.org/
Independent Software developer/SCM consultant, email for more information.

O< ascii ribbon campaign - stop html mail - www.asciiribbon.org