[urllib2 + Tor] How to handle 404?

Gilles Ganault nospam at nospam.com
Fri Nov 7 03:05:20 EST 2008


Hello

	I'm using the urllib2 module and Tor as a proxy to download data
from the web.

Occasionnally, urlllib2 returns 404, probably because of some issue
with the Tor network. This code doesn't solve the issue, as it just
loops through the same error indefinitely:

=====
for id in rows:
	url  = 'http://www.acme.com/?code=' + id[0]
	while True:
		try:
			req = urllib2.Request(url, None, headers)
			response = urllib2.urlopen(req).read()
		except HTTPError,e:
			print 'Error code: ', e.code
			time.sleep(2)
			continue
=====

Any idea of what I should do to handle this error properly?

Thank you.



More information about the Python-list mailing list