urllib2 rate limiting

[Dimitrios Apostolou]

On Thu, 10 Jan 2008, Rob Wolfe wrote:

> Dimitrios Apostolou <jimis at gmx.net> writes:
>
>> P.S. And something simpler: How can I disallow urllib2 to follow
>> redirections to foreign hosts?
>
> You need to subclass `urllib2.HTTPRedirectHandler`, override
> `http_error_301` and `http_error_302` methods and throw
> `urllib2.HTTPError` exception.

Thanks! I think for my case it's better to override redirect_request 
method, and return a Request only in case the redirection goes to the 
same site. Just another question, because I can't find in the docs the 
meaning of (req, fp, code, msg, hdrs) parameters. To read the URL I get 
redirected to (the 'Location:' HTTP header?), should I check the hdrs 
parameter or there is a better way?


Thanks,
Dimitris


>
> http://diveintopython.org/http_web_services/redirects.html
>
> HTH,
> Rob
> --
> http://mail.python.org/mailman/listinfo/python-list
>