Help me use my Dual Core CPU!

[Paul Boddie]

Simon Wittber wrote:
> Michael wrote:
> > Also, Paul Boddie posted a module for parallel systems a while back as well
> > which might be useful (at least for ideas):
> >    * http://cheeseshop.python.org/pypi/parallel
> >
>
> I've checked this out, it looks like a good idea which I could build
> further on.

Feel free to expand on what I've done. The tricky part was making some
kind of communications mechanism with asynchronous properties, and
although I suppose I could have used asyncore, Medusa or Twisted, I was
interested in the exercise of learning more about the poll system call
(and without adding dependencies which dwarf the pprocess module
itself).

> I've just noticed that os.fork is not available on Win32. Ouch.

Sorry! My motivation was to support operating systems that I personally
care about and which also have solutions for transparent process
migration, which I believe could be used in conjunction with the
pprocess module for better-than-thread parallelisation. In other words,
you get processes running independently on potentially many CPUs with
only the communications as overhead, and with access to shared data
either through the communications channels or via global variables
which are read only due to the nature of the fork semantics. This
doesn't necessarily play well with threaded-style programs wanting to
modify huge numbers of shared objects, but I'd argue that the benefits
of parallelisation (for performance) are somewhat reduced in such
programs anyway.

> Does that mean there is _no_ way for a single Python program to use
> multiple CPU/core systems on Windows? (other than writing an extension
> module in C (which is completely unacceptable for me!))

Rumour has it that recent versions of Windows provide fork-like
semantics through a system call. Your mission is to integrate this
transparently into the standard library's os.fork function. ;-)

Paul