How to download a web page just like a web browser do ?

Bo Yang struggleyb at gmail.com
Wed Aug 23 10:34:45 EDT 2006


Hi ,
     It is about one month passed since I post to this list
last time . Yes , I use python , I used it in every day normal
work , whenever I need to do some scripts or other little-scale
works , python is the first one I took consideration in . I must
say it is a powerful tool for me , and what is more important
is there is a friendly and flourish community here .
     Oh , I must stop appriciation and go to my question now .

    Everyday , I receive a newsletter from NYTimes , but I
didn't want to read the news in evening the time the letter
came in . So , I am about to download the web page
contains the news and read them next morning ! I decide to
use python to write a tool , which should be feeded with a
URL , and then download the page to my disk . This
function just like the Browser's "save as..." function . I
know what shoud I do to accomplish that , I need to parse
the web page , and download all pages in the page , and
modify all the links to conrespond local disk links and ...

   So , is there any similar function any one have impelment?
Does anyone can share some code with me ? I really don't
want to some confusing code to process such as text findings
and substitutions .

     Thanks in advance !


                                                        Bo




More information about the Python-list mailing list