Python equivalent of LWP and HTTP in Perl

[David M. Cooke]

"Erik Johnson" <spam at nospam.org> writes:

> "Irmen de Jong" wrote
>
>> >   my($ua) = LWP::UserAgent->new;
>> >   my $req = GET "$uri";
>> >   $req->header(Referer => "$referer");
>> >   $ua->proxy('http', 'http://localhost:8080/'); # Proxomitron
>> >   $response = $ua->request($req);
>> >   $respcode = $response->code;
>>
>>
>> import urllib
>> proxies = {'http': 'http://localhost:8080/'}
>> response = urllib.urlopen("some_url", proxies=proxies).read()
>
> ... which is short and clean, but ignores the Referer header.
> I have worked with httplib some, but not urllib. Maybe urllib can do this,
> but may I suggest checking out httplib:
> http://docs.python.org/lib/httplib-examples.html

Try urllib2:

import urllib2

proxy_support = urllib2.ProxyHandler({"http" : "http://localhost:8080/"})
opener = urllib2.build_opener(proxy_support)
urllib2.install_opener(opener)

req = urllib2.Request("some_url")
req.add_header("Referer", referer)
response = urllib2.urlopen(req)


There's fancier handlers too, if you need digest authentication, for
example. If you have the environment variable 'http_proxy' set to your
proxy, you don't need to do the install_opener jazz.

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca