httplib hung on read

[Eric Lorenzo]

ftian at cs.wisc.edu writes:
> I am using the httplib to get some webpage, it hungs sometimes
> in the following senario,
> 
> http = httplib.HTTP('myhost')
> http.putrequest('GET', myurl)
> http.putheader(...)
> http.endheaders()
> 
> errcode, errmsg, header = http.getreply()
> myfile = http.getfile()
> # "Still OK here"
> page = myfile.read()
> # Sometimes Cannot Reach here
> 
> the page can be opened with IE, and it only hungs sometimes.
> Anyone know the reason?  Thanks.

I use httplib heavily in one of my programs, and I found that it would
occasionally hang for no apparent reason.  Similarly, occasionally I
find that some sites occasionally hang when I try to load pages in a
browser - I connect okay, but never get any response.  I'm assuming
that httplib is hanging for the same reason as my browser - except
that my browser eventually times out.

I handled this in my app by forking off a seperate thread to do the
HTTP requests, and have the main thread do a '.join(60)' on the
request thread - so that if the request thread hasn't completed in 60
seconds, the main thread will continue, and I'll treat this as a
failure to connect.  Maybe not the cleanest solution, but it worked
okay for my purposes.

Eric