HTTPError sometimes when using urllib2.urlopen

[Magnus.Moraberg at gmail.com]

On 27 Maj, 11:03, Magnus.Morab... at gmail.com wrote:
> Hi,
>
> I have the following code -
>
> import urllib2
> from BeautifulSoup import BeautifulSoup
>
> proxy_support = urllib2.ProxyHandler({"http":"http://
> 999.999.999.999:8080"})
> opener = urllib2.build_opener(proxy_support)
> urllib2.install_opener(opener)
>
> page = urllib2.urlopen('http://www.cornish-language.org/CORNISH/
> membership.asp')
> soup = BeautifulSoup(page)
>
> pageText = soup.findAll(text=True)
> print pageText
>
> Sometimes when I run this code I get the exception -
>
> HTTPError: HTTP Error 407 Proxy authentication required
>
> I seem to get this error repeatidly for a while, then I go a surf for
> a moment and when I run the code again the exception might have gone
> away. No idea why...
>
> I have no problems with firefox which uses the same proxy. The proxy
> requires no authentication...
>
> What might the issue be?
>
> Thanks,
>
> Barry.

I'm running with Python 2.5 with PythonWin

Heres the full exception -

Traceback (most recent call last):
  File "C:\Python25\Lib\site-packages\pythonwin\pywin\framework
\scriptutils.py", line 409, in ImportFile
    reload(sys.modules[modName])
  File "C:\Documents and Settings\konbgn\Desktop\parse.py", line 8, in
<module>
    page = urllib2.urlopen('http://www.cornish-language.org/CORNISH/
membership.asp')
  File "C:\Python25\lib\urllib2.py", line 121, in urlopen
    return _opener.open(url, data)
  File "C:\Python25\lib\urllib2.py", line 380, in open
    response = meth(req, response)
  File "C:\Python25\lib\urllib2.py", line 491, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Python25\lib\urllib2.py", line 418, in error
    return self._call_chain(*args)
  File "C:\Python25\lib\urllib2.py", line 353, in _call_chain
    result = func(*args)
  File "C:\Python25\lib\urllib2.py", line 499, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 407: Proxy Authentication Required