[New-bugs-announce] [issue23448] urllib2 needs to remove scope from IPv6 address when creating Host header

[Neil Gierman]

New submission from Neil Gierman:

Using a scoped IPv6 address with urllib2 creates an invalid Host header that Apache will not accept.

        IP = "fe80::0000:0000:0000:0001%eth0"
        req = urllib2.Request("http://[" + IP + "]/")
        req.add_header('Content-Type', 'application/json')
        res = urllib2.urlopen(req, json.dumps(data))

Apache will reject the above request because the Host header is "[fe80::0000:0000:0000:0001%eth0]". This behavior was reported to Apache at https://issues.apache.org/bugzilla/show_bug.cgi?id=35122 and the Apache devs will not fix this as there are new RFCs prohibiting scopes in the Host header. Firefox had the same issue and their fix was to strip out the scope from the Host header: https://bugzilla.mozilla.org/show_bug.cgi?id=464162 and http://hg.mozilla.org/mozilla-central/rev/bb80e727c531.

My suggestion is to change urllib2.py's do_request_ method from:

        if not request.has_header('Host'):
            request.add_unredirected_header('Host', sel_host)

to:

        if not request.has_header('Host'):
            request.add_unredirected_header('Host', re.compile(r"%.*$").sub("", sel_host, 1))

I have not tested this patch to urllib2.py however I am now using similar logic in my code to override the Host header when I create my request:

        IP = "fe80::0000:0000:0000:0001%eth0"
        req = urllib2.Request("http://[" + IP + "]/")
        req.add_header('Host', '[' + re.compile(r"%.*").sub("", IP, 1) + ']')
        req.add_header('Content-Type', 'application/json')
        res = urllib2.urlopen(req, json.dumps(data))

----------
components: Library (Lib)
messages: 235762
nosy: ngierman
priority: normal
severity: normal
status: open
title: urllib2 needs to remove scope from IPv6 address when creating Host header
type: behavior
versions: Python 2.7

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue23448>
_______________________________________