urllib and urllib2, with proxies

[Ali.Sabil at gmail.com]

hello all,

I just maybe hit a bug in both urllib and urllib2, actually urllib
doesn't support proxy authentication, and if you setup the http_proxy
env var to http_proxy=http://user:passwd@host:port/ and
https_proxy=$http_proxy i get a traceback :

Traceback (most recent call last):
  File "test_urllib.py", line 2, in ?
    urllib.urlopen("https://sf.net/")
  File "/usr/lib/python2.4/urllib.py", line 82, in urlopen
    return opener.open(url)
  File "/usr/lib/python2.4/urllib.py", line 190, in open
    return getattr(self, name)(url)
  File "/usr/lib/python2.4/urllib.py", line 313, in open_http
    h.endheaders()
  File "/usr/lib/python2.4/httplib.py", line 798, in endheaders
    self._send_output()
  File "/usr/lib/python2.4/httplib.py", line 679, in _send_output
    self.send(msg)
  File "/usr/lib/python2.4/httplib.py", line 646, in send
    self.connect()
  File "/usr/lib/python2.4/httplib.py", line 614, in connect
    socket.SOCK_STREAM):
IOError: [Errno socket error] (-2, 'Name or service not known')


now with urllib2, it goes beyond that and connect and authenticate to
the proxy, however, instead of using CONNECT with https, it simply does
a GET, which result in a error 501 with a squid proxy:

Traceback (most recent call last):
  File "test_urllib.py", line 2, in ?
    urllib2.urlopen("https://sf.net/")
  File "/usr/lib/python2.4/urllib2.py", line 130, in urlopen
    return _opener.open(url, data)
  File "/usr/lib/python2.4/urllib2.py", line 358, in open
    response = self._open(req, data)
  File "/usr/lib/python2.4/urllib2.py", line 376, in _open
    '_open', req)
  File "/usr/lib/python2.4/urllib2.py", line 337, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.4/urllib2.py", line 573, in <lambda>
    lambda r, proxy=url, type=type, meth=self.proxy_open: \
  File "/usr/lib/python2.4/urllib2.py", line 597, in proxy_open
    return self.parent.open(req)
  File "/usr/lib/python2.4/urllib2.py", line 364, in open
    response = meth(req, response)
  File "/usr/lib/python2.4/urllib2.py", line 471, in http_response
    response = self.parent.error(
  File "/usr/lib/python2.4/urllib2.py", line 402, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.4/urllib2.py", line 337, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.4/urllib2.py", line 480, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 501: Not Implemented


this is with python-2.4.3

thank you for your help

--
Ali Sabil