Parallelization on muli-CPU hardware?

[Steve Holden]

P.M. wrote:

> Alan Kennedy <alanmk at hotmail.com> wrote in message news:<fmy8d.32815$Z14.12044 at news.indigo.ie>...
> 
>>I agree that it could potentially be a serious hindrance for cpython if 
>>"multiple core" CPUs become commonplace. This is in contrast to jython 
>>and ironpython, both of which support multiple-cpu parallelism.
>>
>>Although I completely accept the usual arguments offered in defense of 
>>the GIL, i.e. that it isn't a problem in the great majority of use 
>>cases, I think that position will become more difficult to defend as 
>>desktop CPUs sprout more and more execution pipelines.
> 
The GIL has been a problem to me (probably a "fairly typical" Python 
user) precisely once, when I tried to run a lengthy database process on 
my brand-new H-P desktop-replacement laptop. The task seemed to be 
running a little slower than I would have expected from the quoted clock 
speed, so I opened up the Windows task manager and lo and behold, the 
performance monitor was showing two CPUs, one hitting the stop at 100% 
and the other basically idle.

Until that time I hadn't realized that the laptop was built on a 
multi-core processor. Fortunately the task was easily partitioned into 
two independent processes, each dealing with a separate set of database 
rows, so the run completed well under the necessary 24 hours.
> 
> I wonder if Python could be changed to use thread local storage?  That
> might allow for multiple interpreter instances without the GIL (I've
> never looked at the actual code so I'm just hypothesizing).  I took a
> quick look at Lua today and it has no problems with creating multiple
> instances of the interpreter, so it definately is a solvable problem.

Thread synchronization through a common namespace would likely be the 
problem. The last time this was discussed on c.l.py someone who knows 
far more than I about the interpreter's structure (Tim Peters, unless I 
mistake my guess) opined that removing the GIL from CPython was far from 
being a simple undertaking. Whether that's changed since I dunno.

regards
  Steve
-- 
http://www.holdenweb.com
http://pydish.holdenweb.com
Holden Web LLC +1 800 494 3119