Impersonating other broswers...

[Skip Montanaro]

    sboyle> I'm using urlopen, and it works fine.  But I'd like to be able
    sboyle> to change my browser string from "Python-urllib/1.15" to instead
    sboyle> impersonate Internet Explorer.

    sboyle> I know this can be done very easily with Perl, so I'm assuming
    sboyle> it's also easy in Python.  How do I do it?

Easy is in the eye of the beholder I suppose.  It doesn't look as
straightforward as I would have thought.  You can subclass the
FancyURLopener class like so:

    class MSIEURLopener(urllib.FancyURLopener):
        version = "Internet Exploder"

then set urllib._urlopener to it:

    urllib._urlopener = MSIEURLopener

After that, urllib.urlopen() should spit out your user-agent string.

Seems like FancyURLopener should support setting the user agent string
directly.  You can accomplish that with something like this:

    class FlexibleUAopener(urllib.FancyURLopener):
        def set_user_agent(self, user_agent):
            ua = [(hdr, val) for (hdr, val) in self.addheaders
                     if hdr == "User-agent"]
            while ua:
                self.addheaders.remove(ua[0])
                ua.pop()
            self.addheader(("User-agent", user_agent))

You'd then be able to set the user agent, but have to use your new opener
class directly:

    opener = FlexibleUAopener(...)
    opener.set_user_agent("Internet Exploder")
    f = opener.open(url)
    print f.read()

It doesn't look any easier to do this using urllib2.  Seems like a
semi-obvious oversight for both modules.  That suggests few people have ever
desired this capability.

Skip