Threading problem

sdistefano sdistefano at gmail.com
Sun Apr 25 15:55:18 EDT 2010


I have the following issue:

My program runs a thread called the MainThread, that loops trough a
number of URLs and decides when it's the right time for one to be
fetched.  Once a URL has to be fetched, it's added to a Queue object,
where the FetchingThread picks up and does the actual work. Often,
URLs have to be fetched with frequencies of 100ms, so the objects will
get added to the queue repeatedly. Even though it takes more than
100ms to get the URL and process it, ideally what would happen is:
ms0: Request 1 is sent
ms100: request 2 is sent
ms150: request 1 is processed
ms200: request 3 is sent
ms250: request 2 is processed

and so on... The problem is that for some reason python runs the main
thread considerably more than the checking thread. If I ask them both
to 'print' when run, this becomes obvious ; even if I create more
check threads than main threads (I even tried 50 check threads and 1
main thread). Despite my terminology, both the mainthread and
fetchthread are created in exactly the same way.

what part of threading in python am I not properly understanding?

thanks!



More information about the Python-list mailing list