Parallelization on muli-CPU hardware?
Steve Holden
steve at holdenweb.com
Wed Oct 6 04:49:50 EDT 2004
P.M. wrote:
> Alan Kennedy <alanmk at hotmail.com> wrote in message news:<fmy8d.32815$Z14.12044 at news.indigo.ie>...
>
>>I agree that it could potentially be a serious hindrance for cpython if
>>"multiple core" CPUs become commonplace. This is in contrast to jython
>>and ironpython, both of which support multiple-cpu parallelism.
>>
>>Although I completely accept the usual arguments offered in defense of
>>the GIL, i.e. that it isn't a problem in the great majority of use
>>cases, I think that position will become more difficult to defend as
>>desktop CPUs sprout more and more execution pipelines.
>
The GIL has been a problem to me (probably a "fairly typical" Python
user) precisely once, when I tried to run a lengthy database process on
my brand-new H-P desktop-replacement laptop. The task seemed to be
running a little slower than I would have expected from the quoted clock
speed, so I opened up the Windows task manager and lo and behold, the
performance monitor was showing two CPUs, one hitting the stop at 100%
and the other basically idle.
Until that time I hadn't realized that the laptop was built on a
multi-core processor. Fortunately the task was easily partitioned into
two independent processes, each dealing with a separate set of database
rows, so the run completed well under the necessary 24 hours.
>
> I wonder if Python could be changed to use thread local storage? That
> might allow for multiple interpreter instances without the GIL (I've
> never looked at the actual code so I'm just hypothesizing). I took a
> quick look at Lua today and it has no problems with creating multiple
> instances of the interpreter, so it definately is a solvable problem.
Thread synchronization through a common namespace would likely be the
problem. The last time this was discussed on c.l.py someone who knows
far more than I about the interpreter's structure (Tim Peters, unless I
mistake my guess) opined that removing the GIL from CPython was far from
being a simple undertaking. Whether that's changed since I dunno.
regards
Steve
--
http://www.holdenweb.com
http://pydish.holdenweb.com
Holden Web LLC +1 800 494 3119
More information about the Python-list
mailing list