Threading / Web Page Download

[brueckd at tbye.com]

On Thu, 15 Aug 2002, David wrote:

> Hello.  In the app that I (newbie) am building, I was finding that I would
> occasionally encounter lockups when downloading pages with urllib2 (and
> ClientCookie, which is based on urllib2).  To get past the lockups, I've
> implemented threading (below).  Problem is, it's an inelegant solution:
> When a given download exceeds the timeout threshold, my program effectively
> abandons it, starting another download attemp in another thread (up to some
> max number of thereads, n).

Hi David,

There's no clean/safe way to kill a thread. Google for timeoutsocket; 
it'll make sockets raise an exception after awhile instead of hanging 
forever. Also (looking ahead), Python 2.3 has socket timeouts built-in.

-Dave