Urllib2 upgrade is breaking urlopen()

[John J. Lee]

jjl at pobox.com (John J. Lee) writes:

> dj00302003 at yahoo.com (Jay Davis) writes:
> 
> > We have a lot of simple code using 'urllib', that basically
> > just goes out and opens and read()'s some http url.  Very basic,
> > ten line scripts, and they work fine with urllib.
> > 
> > We want to use urllib2 for everything, though, but when we
> > do we get: 
> [...]
> > urllib2.HTTPError: HTTP Error 302: The HTTP server returned a redirect
> > error that would lead to an infinite loop.
> > The last 30x error message was:
> > Found
> 
> Which version of Python are you using?
> 
> It might also help if you can give a publicly-accessible URL that
> triggers the problem (mail it to me privately if you prefer).

Jay sent me a URL, and it turns out both urllib and urllib2 are
functioning as intended.  There's a 302 loop caused by the server
expecting cookies to get returned.  urllib goes into a loop until it
notices it's redirected 10 times, then just returns the 302 as a
normal response (which doesn't seem very nice, but appears to be what
was intended).  urllib2 detects the loop and returns the 302 as an
HTTPError exception (which can be caught and .read() if necessary).


John