Slurping Web Pages

Tony Dunn tdunn at lynxxsolutions.com
Sat Jan 25 13:54:33 EST 2003


I've started a new project where I need to slurp web pages from a site that
use cookies to authenticate access.  I've used *urllib* in the past to grab
*public* web pages, but I'm not sure the best way to go about dealing with
the cookie issue.

I found some code to drive IE via COM, but I can't find a method to save the
current web page to a file so I can *slurp* it later.  I've wandered through
the file generated by makepy.py for the *Internet Control* COM object, but I
don't see what I'm looking for.  I know I can grab the files from the local
*Internet* cache, but I'd like the option to specify a file location and
file name for each page downloaded.

Has anyone done this with IE?

-Tony






More information about the Python-list mailing list