threading a 10 lines out of a file

[Steve Holden]

alex goretoy wrote:
> Steve,
> 
> Are you referring to the endless GIL debate? I googled about what you
> said. I'll look into it here shortly. I just know one thing, is that I
> need a comparison for regular code and something about how GIL and
> threading fit in the picture. It makes it easier for me to understand
> what is going on. Can you post an example? something I can use to truely
> thread a for loop? The example previously posted show that it works 10
> times faster than original for loop. My current program runs for 4.5
> hours. I think it will be good to minimize that by 10 times. But if It
> can do more/faster, than that would be great. TIA for all your help.
> 
Sadly my example doesn't show that. I did, once upon a time, write code
that used several hundred threads to send emails, and gave a dramatic
speed-up (because of the network-bound nature of the task). Can I
presume that your original inquiry was a toy, and that your real problem
is also IO-bound? Otherwise I am unsure how you will benefit by
threading - if your line-processing tasks don't contain any IO then
using a threaded approach will not yield any speed-up at all.

The example you quoted achieved its speed-up because a thread releases
the GIL while waiting for a network response, allowing other threads to
process. Thus it effectively ran all the pings in parallel.

Se we need to know a bit more about your 4.5-hour program before we can
determine whether threads can help. There is light at the end of the
tunnel, however, since even if threads don't work it's possible that the
multiprocessing module will (assuming you have multi-processor hardware
at your disposal).

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/