http request doesn't work!

f29 f29 at gazeta.pl
Fri Mar 26 04:12:08 EST 2004


> 
> The output of the above is different if you go to the following in
> your browser:
> http://adsl.internode.on.net/htm/un-metered-sites-ip-list.htm
> 
> 
> Whats my problem?!?!??!

Try adding User-Agent header of some popular browser (e.g.
"Mozilla/5.0 (Windows; U; Windows NT; en-US; rv:1.6) Gecko") so that
remote site could not prevent fetching their content with a robot.

Moreover, try looking at the urllib2 module, it has great power.

rgrds,
f29



More information about the Python-list mailing list