Parallel processing

[Josiah Carlson]

corey.coughlin at attbi.com (Corey Coughlin) wrote:
> 
> Josiah Carlson <jcarlson at uci.edu> wrote in message news:<mailman.6501.1100717187.5135.python-list at python.org>...
> > 
> > There are various other reasons why Python is not as parallelizable as
> > you would think.  Among them is the semantics of scoping, and whether
> > there is shared or unshared scope among the processors/nodes.  If shared,
> > then any operation that could change scopes would need to be distributed
> > (ick), or if unshared, then you are basically looking at an
> > automatically distributed tuplespace (LINDA).  It gets even uglier with
> > certain kinds of generators.
> > 
> > Regardless of which one is the case, heavy modifications to Python would
> > necessarily need to be done in order to make them happen.
> > 
> > 
> >  - Josiah
> 
> Well, I'm not sure it's necessarily that grim.  Generally, taking an
> inference engine something like that proposed for Starkiller, tracking
> variable types as closely as possible, adding some data flow
> capability to follow the execution path, you could probably do
> something useful.  With Python's for..in loop syntax, the prohibition
> against chaning loop variables there, and list comprehensions,
> parallelized loop unrolling, done intelligently, could help out a lot
> on parallel architectures.  Sure, it'd be hard to do effectively in a
> strictly interpreted environment, but if Starkiller ever comes out, it
> seems almost inevitable.  Then again, that project seems to be getting
> later all the time.  :(


One could also just add a debugger that distributes data via pickles as
information changes, so data types don't really matter.

The real issue is that /anything/ can have an arbitrary side-effect, and
in order for 'for i in j' parallelization to occur consistantly, those
side-effects must be handled properly.  Those side-effects can be
horribly ugly.


While I say "huzzah" for new languages (or preprocessors for older
languages) that make parallelization occur 'automatically', there is
something to be said about manually parallelizing your algorithms with
MPI, Linda, PVM, etc.  At least then you can be explicit with your
communication and not be afraid that your desired changes may or may not
be transferred (a waste of bandwidth, scope overwriting, not getting
updated data, etc).

 - Josiah