Timeout in urllib2

[Dan M]

I'm writing a system monitor script that needs to load web pages. I'm
using urllib2.urlopen to get the pages, and I'm attempting to set the
timeout value using socket.defaulttimeout.

Specifically, I'm calling socket.defaultttimeout(10), then calling
urllib2.urlopen to fetch a web page that never gets delivered. My code
waits about 30 seconds before terminating.

I am about to add threading to my app so that delays on a few servers
won't be a critical issue, but I'd still like to understand why the call
to socket.defaulttimeout doesn't affect the timeout on my urlopen calls.

My code follows.

#!/usr/local/bin/python
import socket, time

socket.setdefaulttimeout(10)
import urllib2

def doHttpTest():
	url = "http://url.that.never.returns"
	t_start = time.time()
	if httptest(url):
		print "Error on site ", url
	t_end = time.time()
	t_diff = t_end - t_start

def httptest(url):
	timeout = 10
	socket.setdefaulttimeout(timeout)
	try:
		req = urllib2.Request(url)
		urllib2.urlopen(req)
	except urllib2.HTTPError, e:
		if e.code == 401:
			return 1
		elif e.code == 404:
			return 1
		elif e.code == 503:
			return 1
		else:
			return 1
	except urllib2.URLError, e:
		return 1
	else:
		return 0



if __name__ == '__main__':
	try:
		doHttpTest()
	except KeyboardInterrupt:
		print "Exiting..."