fetch a html page via proxy?

[curtis001 at my-deja.com]

In article <387978ff.3083491 at news.jps.net>,
  bayernmREMOVEME at hotmail.com wrote:
> I need to fetch a page from the web but the linux machine doesn't have
> direct connection to the net.  It needs to go thru a proxy to connect
> to the web.  How can I do this with Python?  I know how to write
> Python code that fetch the page directly, but I can't find any
> information about the proxy part.

Ricardo -

The best info I can tell you is to use the URLLIB module, and to set an
environment variable beforehand named "http_proxy" containing the
string "http://<proxyserver>:<socket>/", which URLLIB will try to use
in getting out.  (In fact, it looks for any environment variables named
<service>_proxy and use it for that service -- ftp, gopher, http, etc.)

That's where my assistance ends, I'm afraid.  I haven't been able to
learn the exact format(s) needed (or available) for the http_proxy
variable, and I can't get Python through my proxy server either --
though my error messages seem to be getting fewer each time I re-read
the URLLIB module to determine what it needs from me.  <wry grin>

The module also offers a function named FancyURLopener, which is
supposed to do more in the way of using proxies, but I haven't been
able to find any docs on how it works within URLLIB to replace the
regular URLLIB.URLOpen function.  Apparently it uses a manually-preset
dictionary named "proxies", in which you set a key named for the
service you are proxying with the string value of the proxy server
(like the environment variable above).

There's pitiful little documentation [none] on how to use proxy servers
with Python, and in general, questions about it go uanswered.  (Python
folks are normally good at helping out, so this must be a topic most
aren't familiar with.)

I hope this helps you out a litle bit at least, and if you should find
out more, feel free to share it with another proxy-snared Python
user.  :)

Curtis


Sent via Deja.com http://www.deja.com/
Before you buy.