Help extending BaseHandler from urllib2
Ray Van Dolson
leovd at pacbell.net
Fri Nov 9 13:43:41 EST 2001
I'm trying to extend the BaseHandler class from urllib2 to handle 302
redirects in a different way. I need to grab the cookies from the page
and send them after being redirected. Should be easy enough right? Well
I'm running into a problem. Here's my class:
from urllib2 import BaseHandler, Request, urlopen
class DotsterHandler(BaseHandler):
def default_open(self, req):
print "Executing custom thingy"
return urlopen(req)
def http_error_302(self, req, fp, code, msg, hdrs):
"""Hopefully this function will handle a 302 redirect properly.
We need to maintain any cookies we receive."""
print "Another custom thingy..."
if hdrs.has_key('location'):
newurl = hdrs['location']
elif hdrs.has_key('uri'):
newurl = hdrs['uri']
else:
return
nil = fp.read()
fp.close()
newurl = urlparse.urljoin(req.get_full_url(), newurl)
if hdrs.has_key('set-cookie'):
cookie = hdrs['set-cookie']
new = Request(newurl)
new.add_header('Cookie', cookie)
return self.parent.open(new)
And the test program:
import DotsterHandler
import urllib2
import urllib
def main():
"""The main function"""
# First, create the DotsterHandler
dh = DotsterHandler.DotsterHandler()
# Now create the OpenerDirector object and add dh as its handler
od = urllib2.OpenerDirector()
od.add_handler(dh)
# Construct our request
req = urllib2.Request \
("https://www.dotster.com/account/login/login.asp")
req.add_data(urllib.urlencode({'Acct_Name': 'user', 'Password': \
'pass'}))
# Now let's try opening this with our OpenerDirector...
a = od.open(req)
# Did it work?
print a
if __name__ == '__main__':
main()
I notice that while my custom default_open() method is called (from
DotsterHandler), but once the 302 page is received, the standard 302
handler from urllib2 is called instead of my custom one. Is there
something wrong with the way I'm doing this?
Thanks for any help.
More information about the Python-list
mailing list