Client side network programming

[Anand Pillai]

Try downloading HarvestMan from http://harvestman.freezope.org .

-Anand


fishboy <fishboy at spamspamspam.com> wrote in message news:<qslrb0h9bstsob5056jfdnkvcqk7c8p0go at 4ax.com>...
> On Wed, 02 Jun 2004 06:23:20 -0400, "Roysun_rohit"
> <roysun_rohit at rediffmail.com> wrote:
> 
> >I am interested in making a search engine which takes web sites
> >iteratively, and downloads the web page or has to perform some search
> >across the web pages.
> >I am unsucessful to do so. My machine works through a proxy server and the
> >internet connection is through 24 hour lease line. when ever i try the
> >code it gives :
> >
> >Traceback (innermost last):
> >  File "http-getfile-urllib2.py", line 19, in ?
> >    urllib.urlretrieve(remoteaddr, localname)
> >  File "/usr/lib/python1.5/urllib.py", line 66, in urlretrieve
> >    return _urlopener.retrieve(url, filename, reporthook)
> >  File "/usr/lib/python1.5/urllib.py", line 186, in retrieve
> >    fp = self.open(url)
> >  File "/usr/lib/python1.5/urllib.py", line 159, in open
> >    return getattr(self, name)(url)
> >  File "/usr/lib/python1.5/urllib.py", line 260, in open_http
> >    h = httplib.HTTP(host)
> >  File "/usr/lib/python1.5/httplib.py", line 53, in __init__
> >    if host: self.connect(host, port)
> >  File "/usr/lib/python1.5/httplib.py", line 81, in connect
> >    self.sock.connect(host, port)
> >IOError: [Errno socket error] (101, 'Network is unreachable')
> >============================================================
> >The code is like this:-
> >#!/usr/bin/env python
> >import os, sys, urllib, urlparse, socket
> >showlines = 6
> >try:
> >    servername, filename = sys.argv[1:3]
> >except:
> >    servername, filename = 'www.igib.res.in', '/sarsanalysis.html'
> >    
> >remoteaddr = 'http://%s%s' % (servername, filename)
> >if len(sys.argv) == 4:
> >   localname = sys.argv[3]
> >else:
> >   (scheme, server, path, parms, query, frag) =
> >urlparse.urlparse(remoteaddr)
> >   localname = os.path.split(path)[1]
> >   
> >print remoteaddr, localname
> >urllib.urlretrieve(remoteaddr, localname)
> >remotedata = open(localname).readlines()
> >for line in remotedata[:showlines]: print line,
> >============================================================
> >I am new to the internet programming as well as python. please guide me,
> >how to solve this one.
> 
> urllib will work with proxies.  Just set your environment to point at
> it before you start python.  Like this (copied from the urllib doc):
> 
> % http_proxy="http://www.someproxy.com:3128"
> % export http_proxy
> % python
> 
> hth,
> ><{{{*>