mysteries of urllib/urllib2

[Adrian Smith]

On Jul 3, 11:25 pm, Ben Cartwright <benc.nos... at gmail.com> wrote:
> On Jul 3, 9:43 am, Adrian Smith <adrian_p_sm... at yahoo.com> wrote:
>
> > The following (pinched
> > from Dive Into Python) seems to work perfectly in Idle, but
> > falls at the final hurdle when run as a cgi script - can
> > anyone suggest anything I may have overlooked?
>
> > request = urllib2.Request(some_URL)
> > request.add_header('User-Agent', 'some_plausible_string')
> > opener = urllib2.build_opener()
> > data = opener.open(request).read()
>
> Most likely the account that cgi script is running as does not
> have permissions to access the net. Check the traceback to be
> sure. Put this at the top of your cgi script:
>
> import cgitb; cgitb.enable()

Well, it worked with urllib (resulting in a G**gle 403 your-client-
does-not-have-permission-to-get-urlX page), so I think it must have
some access. Apparently there's a way to change the user-agent string
by subclassing urllib's URLopener class, but that's beyond my comfort
zone at present.