mysteries of urllib/urllib2

[Ben Cartwright]

On Jul 3, 11:14 am, Adrian Smith <adrian_p_sm... at yahoo.com> wrote:
> > > The following (pinched
> > > from Dive Into Python) seems to work perfectly in Idle, but
> > > falls at the final hurdle when run as a cgi script
> > Put this at the top of your cgi script:
>
> > import cgitb; cgitb.enable()

Did you even try this?  Asking for Python help without posting the
traceback is like phoning your mechanic and saying, "My car is making
a generic rattling noise, can you tell me what the problem is without
looking under the hood?"

> Apparently there's a way to change the user-agent string
> by subclassing urllib's URLopener class, but that's beyond my comfort
> zone at present.

Untested:

import urllib
url = 'http://groups.google.com/group/Google-AJAX-Search-API/
browse_thread/thread/a0eb87ad13b11762'
opener = urllib.FancyURLopener()
opener.addheaders = [('User-Agent', 'Fauxzilla 4.0')]
data = opener.open(url).read()

Hope that helps,
--Ben