urllib2 - 403 that _should_ not occur.

[Chris Mellon]

On Sun, Jan 11, 2009 at 9:05 PM, James Mills
<prologic at shortcircuit.net.au> wrote:
> On Mon, Jan 12, 2009 at 12:58 PM, Philip Semanchuk <philip at semanchuk.com> wrote:
>>
>> On Jan 11, 2009, at 8:59 PM, James Mills wrote:
>>
>>> Hey all,
>>>
>>> The following fails for me:
>>>
>>>>>> from urllib2 import urlopen
>>>>>> f =
>>>>>> urlopen("http://groups.google.com/group/chromium-announce/feed/rss_v2_0_msgs.xml")
>>>
>>> Traceback (most recent call last):
>>>  File "<stdin>", line 1, in <module>
>>>  File "/usr/lib/python2.6/urllib2.py", line 124, in urlopen
>>>   return _opener.open(url, data, timeout)
>>>  File "/usr/lib/python2.6/urllib2.py", line 389, in open
>>>   response = meth(req, response)
>>>  File "/usr/lib/python2.6/urllib2.py", line 502, in http_response
>>>   'http', request, response, code, msg, hdrs)
>>>  File "/usr/lib/python2.6/urllib2.py", line 427, in error
>>>   return self._call_chain(*args)
>>>  File "/usr/lib/python2.6/urllib2.py", line 361, in _call_chain
>>>   result = func(*args)
>>>  File "/usr/lib/python2.6/urllib2.py", line 510, in http_error_default
>>>   raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
>>> urllib2.HTTPError: HTTP Error 403: Forbidden
>>>>>>
>>>
>>> However, that _same_ url works perfectly fine on the
>>> same machine (and same network) using any of:
>>> * curl
>>> * wget
>>> * elinks
>>> * firefox
>>>
>>> Any helpful ideas ?
>>
>> The remote server doesn't like your user agent?
>>
>> It'd be easier to help if you post a working sample.
>
> That was a working sample! The "User-Agent" didn't
> occur to me :) Thanks - I think that might be it.
>
> Why Google would deny access to services by
> unknown User Agents is beyond me - especially
> since in most cases User Agents strings are not
> strict.

If you look at the actual response text and not just the error code,
you will get a clear explanation of why you were forbidden.