Multi-threading on Multi-CPU machines

[Tim Churches]

Garry Taylor wrote:
> 
> Joseph A Knapka <jknapka at earthlink.net> wrote in message news:<3D29FC43.C239BD5F at earthlink.net>...
> > Garry Taylor wrote:
> > >
> > > Hello,
> > > I am attempting to make a multi-threading function in one of my
> > > programs in an effort to gain a speed increase, but I'm getting quite
> > > the opposite, even on a dual-CPU Intel/Linux box. Can anyone enlighten
> > > me as to why,
> >
> > Yes. CPython threads cannot utilize multiple CPUs, due to the
> > Global Interpreter Lock, which may only be acquired by one
> > thread at a time. Apparently Jython threads do not have
> > this limitation, as the GIL doesn't exist in Jython, or so
> > I'm told. So if you simply ran your program under Jython
> > you might see an improvement.
> >
> > Cheers,
> >
> > -- Joe
> 
> Thank you both for your anwsers, unfortunatly running under Jython is
> not an option, as the whole program which I am writing runs to about
> 5,000 lines and use lots of Python modules which I don't really fancy
> trying to get to work under Jython.
> 
> So, am I correct in thinking that there is nothing I can do about
> this, and still use standard Python? I understand that Solaris has a
> very good threading library, but from the comments above, I assume
> that this would make no difference? Do you have any tips/ideas how I
> could make use of multiple processors in a Python program?

As someone else suggested, consider using MPI, which can be used to 
parallelise code on shared memory SMP machines as well as networked
clusters. Installing user-mode LAM/MPI is very easy, although other
forms of MPI such as MPICH may be a bit more difficult. However, once 
you have MPI installed, there are a number of Python MPI modules around
which make using it a cinch. May I recommend PyPar, by Ole Nielsen
at the Austrlian National University, as being particularly easy to use?
See http://datamining.anu.edu.au/~ole/pypar/  I am pretty sure Ole has 
been using MPI and Pypar on multi-CPU Solaris machines as well as Linux
Beowulf clusters and the hybrid shared/distributed memory APAC
supercomputer
at ANU.

Tim C

> 
> Thanks again
> 
> Garry
> --
> http://mail.python.org/mailman/listinfo/python-list