Help extending BaseHandler from urllib2

Ray Van Dolson leovd at pacbell.net
Fri Nov 9 13:43:41 EST 2001


I'm trying to extend the BaseHandler class from urllib2 to handle 302 
redirects in a different way.  I need to grab the cookies from the page 
and send them after being redirected.  Should be easy enough right?  Well 
I'm running into a problem.  Here's my class:

from urllib2 import BaseHandler, Request, urlopen

class DotsterHandler(BaseHandler):
    def default_open(self, req):
        print "Executing custom thingy"
        return urlopen(req)

    def http_error_302(self, req, fp, code, msg, hdrs):
        """Hopefully this function will handle a 302 redirect properly.
        We need to maintain any cookies we receive."""
        print "Another custom thingy..."
        if hdrs.has_key('location'):
            newurl = hdrs['location']
        elif hdrs.has_key('uri'):
            newurl = hdrs['uri']
        else:
            return
        nil = fp.read()
        fp.close()

        newurl = urlparse.urljoin(req.get_full_url(), newurl)

        if hdrs.has_key('set-cookie'):
            cookie = hdrs['set-cookie']

        new = Request(newurl)
        new.add_header('Cookie', cookie)
        return self.parent.open(new)

And the test program:

import DotsterHandler
import urllib2
import urllib

def main():
    """The main function"""
    # First, create the DotsterHandler
    dh = DotsterHandler.DotsterHandler()
    # Now create the OpenerDirector object and add dh as its handler
    od = urllib2.OpenerDirector()
    od.add_handler(dh)
    # Construct our request
    req = urllib2.Request \
        ("https://www.dotster.com/account/login/login.asp")
    req.add_data(urllib.urlencode({'Acct_Name': 'user', 'Password': \
        'pass'}))
    # Now let's try opening this with our OpenerDirector...
    a = od.open(req)
    # Did it work?
    print a

if __name__ == '__main__':
    main()

I notice that while my custom default_open() method is called (from 
DotsterHandler), but once the 302 page is received, the standard 302 
handler from urllib2 is called instead of my custom one.  Is there 
something wrong with the way I'm doing this?

Thanks for any help.



More information about the Python-list mailing list