Can not get urllib.urlopen to work

[Andrew Dalke]

Pater Maximus wrote:

> I am trying to implement the recipe listed at
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/211886
> 
> However, I can not get to first base. When I try to run
> 
> import urllib
> fo=urllib.urlopen("http://www.dictionary.com/")
> page = fo.read()

I can't even connect to it with my web browser.  Can you?
If you can, they are probably checking the user-agent sent
by urllib, to make it harder to do this sort of automated
screen scraping.

See the docs at
  http://www.python.org/doc/current/lib/module-urllib.html

for an example of how to change the default user-agent.

Here's one for MSIE under Win2K

   Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)

				Andrew
				dalke at dalkescientific.com