Comments appreciated on Erlang inspired Process class.

George Sakkis george.sakkis at gmail.com
Fri Jun 1 21:45:46 EDT 2007


On Jun 1, 2:20 pm, "Brian L. Troutwine" <goofyheadedp... at gmail.com>
wrote:

> >http://wiki.python.org/moin/ParallelProcessing
>
> Ah, I'd forgotten about that page of the wiki; I hadn't seen it for a few
> months.
>
> > Do you have any opinions about those projects listed on the above page
> > that are similar to your own? My contribution (pprocess), along with
> > others (processing, pp...), can offer similar facilities, but the
> > styles of interfacing with spawned processes may be somewhat
> > different.
>
> The interface was my most important design goal, in that I wanted it to be a
> dead simple "Drop in some code and forget about it for a while. Retrieve the
> results later as if you'd called the function yourself." sort of thing.
> Secondly I wanted share nothing parallelism in order to avoid the nastier
> bits of implementing concurrent code.

Funny, I've been working on a similar library these days with the same
primary goal, a minimal intuitive API. The closest to what I had in
mind that I found in the parallel processing wiki was the
Scientific.DistributedComputing package (actually there are no
dependencies with the rest Scientific.* packages, the library
comprises two modules and one script all in all). It's worth checking
out and there's at least one idea I plan to copy (the "watchdog"
thread monitoring if a process has died), but overall I found the API
a bit clunky for my taste.

So I rolled yet another parallel processing package and I'm glad to
have a first working version ready at this point, exposing a single
lightweight API and three distinct platform-independent
implementations: one using multiple threads, one using multiple
processes in one or more hosts (through PYRO) and one singlethreaded
(for the sake of completeness, probably not very useful). I'll write
up some docs and I'll announce it, hopefully  within the week.

George




More information about the Python-list mailing list