python concurrency proposal

[Corey Coughlin]

Mike Meyer wrote:

> [Rest elided]
> 
> This really has a lot in common with SCOOP. SCOOP makes concurrent
> stuff easier, but i'm not sure it fits well with Python. I'll describe
> it, on the off chance you may get some ideas from it. See <URL:
> http://archive.eiffel.com/doc/manuals/technology/concurrency/ > for
> full details.

Eiffel!  I should have thought of that.  I may have to dig all my old 
Meyer books out of storage......

> 
> Like pardef, you use SCOOP by creating objects that are going to run
> concurrently. Unlike pardef, the objects are - well, objects, not just
> restricted to functions. This has deep implications. For instance, one
> of the things that sets SCOOP apart from other concurrency systems
> that I know if is that the program doesn't have code to create
> "processors" (threads, process, whatever - places for the code to
> run).  From my reading of your description, this isn't the case for
> pardef. Invoking a pardef causes a new processor to be created, and
> when the pardef returns, the processor is finished.

Well, not quite, the pardef is actually an object, and you can certainly 
put all kinds of objects inside it if you want to.  The pardef object is 
really more of an interface specification, where the actual 
implementation of most of the methods is really defined in the base 
class.  The body of the pardef really just defines what happens during a 
.run() call.  The whole 'def' in pardef and the use of returns (and 
yields) may be kind of misleading, the return (and yield) are really 
just ways to send info back to the calling scope without having to come 
up with some wierd way of referencing it.  What I'm envisioning can 
return a single value, or stay resident and return multiple values 
(through yield or .send calls) or whatever.

> 
> Like pardef, SCOOP separates the specification of the kind of
> processor from the rest of the code. It carries things further than
> pardef though. You specify a set of processors at build time, and the
> support system takes care of getting objects on the right processor.
> 
> Synchronization and protection is all handled via method
> invocation. There are no new primitives for locking objects, explicit
> synchronization, or communications.

Yeah, I've been trying to think of some way to setup channels in a 
better way.  Tagged channels can certainly be used, but method (or 
method-like) invocations would be more elegant in more than a few 
situations.  On the other hand, I've also been trying to avoid locking 
people into a fully OO style in order to use concurrency, it would be 
nice to allow people who don't want to define everything as an object to 
use parallelism.  But it can get ugly that way.  Maybe something like a 
"property" declaration to optionally simplify things, hmm....

> 
> Calls to methods of a separate object are non-blocking. The calls are
> queued, and executed later. Reading an attribute from a separate
> object is a blocking action.  Calling a method with a separate object
> as an argument causes the call to block until the separate arguments
> are all locked. There are rules - enforceable by the compiler - about
> what you can do with separate objects that prevent a lot of the things
> that cause headaches when doing concurrent programming.

Undoubtedly enforced by contracts, Eiffel is nice that way.  Some things 
I do miss from stricter languages.  But then again, having to define 
everything as an object does drive me nuts.

> 
> So instead of "run", you instantiate a separate object. Instead of
> ".send", you just invoke a method on the seperate object, and the
> arguments get sent to it. Instead of ".receive", you read an attribute
> from a separate object. Instead of ."kill", you let the object be
> garbage collected (or maybe those aren't the same thing).  Return is
> missing, because it's an object, not a function. On the other hand,
> you could define an attribute that's the "return value", and reading
> it fetches that value.
> 
> The problems with Python is that "separate" is really a property of a
> variable, not an object. To see the difference, if I declare an object
> separate, and it creates an attribute, and I read that attribute, then
> that object is separate for me. But in the object that created it, it
> isn't. This is one major clash with Python. The other is the
> critical distinction that SCOOP places on reading vs. calling a
> feature. What does:
> 
>          a = Seperate_Object()
>          x = a.thing
>          x()
> 
> do with SCOOP semantics?
> 
> Oh well. I think SCOOP does the job you want of "making things
> easier", and fits well in an OO language. Maybe you can find good
> ideas in it.
> 
>       <mike

Yeah, the dynamic nature of python makes a straight translation of SCOOP 
pretty problematic.  But it's good stuff to consider, definitely some 
great ideas in there.  I'm not really sure I want a solution that gets 
as complicated as SCOOP, but it may not be something I can avoid.  Ah 
well, some thinking to do.  But just out of curiosity, would you support 
a PEP with some language additions to support and simplify concurrency?

		------ Corey