Web-crawling

Rene Pijlman reply.in.the.newsgroup at my.address.is.invalid
Sat Oct 4 09:51:40 EDT 2003


John Bradbury:
>where do you set up the User-agent header you suggested.

Its an HTTP header:
http://www.ietf.org/rfc/rfc2616.txt

See also:
http://www.google.com/search?q=urllib+python+%22user%2Dagent%22

which leads you to... tada... the documentation! :-)
http://www.python.org/doc/current/lib/module-urllib.html

-- 
René Pijlman




More information about the Python-list mailing list