python concurrency proposal

[Mike Meyer]

corey.coughlin at comcast.net writes:
> Alright, so I've been following some of the arguments about enhancing
> parallelism in python, and I've kind of been struck by how hard things
> still are.  It seems like what we really need is a more pythonic
> approach.

I certainly agree, and have thought about it some myself.

> pardef <Name>(self, <par type>, arguments...):
> 	self.send(<dest pardef>, <tag>, arguments)
> 	self.receive(<tag>, arguments)
> 	return arguments
> 	yield arguments

[Rest elided]

This really has a lot in common with SCOOP. SCOOP makes concurrent
stuff easier, but i'm not sure it fits well with Python. I'll describe
it, on the off chance you may get some ideas from it. See <URL:
http://archive.eiffel.com/doc/manuals/technology/concurrency/ > for
full details.

Like pardef, you use SCOOP by creating objects that are going to run
concurrently. Unlike pardef, the objects are - well, objects, not just
restricted to functions. This has deep implications. For instance, one
of the things that sets SCOOP apart from other concurrency systems
that I know if is that the program doesn't have code to create
"processors" (threads, process, whatever - places for the code to
run).  From my reading of your description, this isn't the case for
pardef. Invoking a pardef causes a new processor to be created, and
when the pardef returns, the processor is finished.

Like pardef, SCOOP separates the specification of the kind of
processor from the rest of the code. It carries things further than
pardef though. You specify a set of processors at build time, and the
support system takes care of getting objects on the right processor.

Synchronization and protection is all handled via method
invocation. There are no new primitives for locking objects, explicit
synchronization, or communications.

Calls to methods of a separate object are non-blocking. The calls are
queued, and executed later. Reading an attribute from a separate
object is a blocking action.  Calling a method with a separate object
as an argument causes the call to block until the separate arguments
are all locked. There are rules - enforceable by the compiler - about
what you can do with separate objects that prevent a lot of the things
that cause headaches when doing concurrent programming.

So instead of "run", you instantiate a separate object. Instead of
".send", you just invoke a method on the seperate object, and the
arguments get sent to it. Instead of ".receive", you read an attribute
from a separate object. Instead of ."kill", you let the object be
garbage collected (or maybe those aren't the same thing).  Return is
missing, because it's an object, not a function. On the other hand,
you could define an attribute that's the "return value", and reading
it fetches that value.

The problems with Python is that "separate" is really a property of a
variable, not an object. To see the difference, if I declare an object
separate, and it creates an attribute, and I read that attribute, then
that object is separate for me. But in the object that created it, it
isn't. This is one major clash with Python. The other is the
critical distinction that SCOOP places on reading vs. calling a
feature. What does:

         a = Seperate_Object()
         x = a.thing
         x()

do with SCOOP semantics?

Oh well. I think SCOOP does the job you want of "making things
easier", and fits well in an OO language. Maybe you can find good
ideas in it.

      <mike
-- 
Mike Meyer <mwm at mired.org>			http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.