using threads with for-loops
Peter Hansen
peter at engcorp.com
Tue Sep 28 16:00:44 EDT 2004
Mathias Waack wrote:
> Peter Hansen wrote:
>>Klaus Neuner wrote:
>>>I wrote a program that does essentially the following:
>>>
>>>for rule in rules:
>>>for line in line_list:
>>>line = my_apply(rule, line)
>>>
>>>line_list contains the lines of some input text.
>>>
>>>To "apply a rule" always means to
>>>
>>> 1. do some regex matches on line
>>> 2. substitute something in line
>>>
>>>My question is: Given this "architecture", does it make sense
>>>to use threads? And if so, how?
>>
>>The code is (based on what you give above) "CPU bound",
>>which means you will not see any advantage in using
>>threads.
>
> Have you ever seen machines with more than one CPU?
Why yes, I have! And have *you* seen an implementation
of Python which will effectively use those multiple CPUs
in code like that above which runs in a single process?
And do you think it likely that the OP is dealing with
a multiple CPU situation, but managed to forget to
mention it? I didn't think it likely, which is why when
I considered the multi-CPU situation I discarded the
idea. Perhaps, however, I was too quick to judge...
> The code above is perfectly suited for parallelization.
Yes, it is. Once you take into account implementation
issues (e.g. the GIL), would you still think so?
-Peter
More information about the Python-list
mailing list