[Python-Dev] Pythonic concurrency

[Phillip J. Eby]

At 09:30 AM 9/29/2005 -0600, Bruce Eckel wrote:
>This paper looks very interesting and promises some good ideas. It
>also looks like it will require time and effort to digest.
>
>I've only read the first few pages, but one thing that does leap out
>is at the beginning of section 3, they say:
>
>"... a purely-declarative language is a perfect setting for
>transactional memory."
>
>What's not clear to me from this is whether STM will work in a
>non-declarative language like Python.

I spent a few weekends studying that paper earlier this year in order to 
see if anything could be stolen for Python; my general impression was "not 
easily" at the least.  One notable feature of the presented concept was 
that when code would otherwise block, they *rolled it back* to the last 
nonblocking execution point.  In a sense, they ran the code backwards, 
albeit by undoing its effects.  They then suspend execution until there's a 
change to at least one of the variables read during the forward execution, 
to avoid repeated retries.

It was a really fascinating idea, because it was basically a way to write 
nonblocking code without explicit yielding constructs.  But, it depends 
utterly on having all changes to data being transactional, and on being 
able to guarantee it in order to prevent bugs that would be just as bad as 
the ones you get from threading.

Oddly enough, this paper actually demonstrates a situation where having 
static type checking is in fact a solution to a non-trivial problem!  It 
uses static type checking of monads to ensure that you can't touch 
untransacted things inside a transaction.