url directory listing

[Paul Magwene]

Curtis Jensen wrote:
> 
> Remco Gerlich wrote:
> >
> > Curtis Jensen wrote in comp.lang.python:
> > > I little while ago, there was a post about websucker.  I'm writing a
> > > similar script. What is the socket command or url command to get a
> > > directory listing?  Especialy if there is an index.html file in the
> > > directory.  Thanks.
> >
> > This is not possible with HTTP.
> >
> > --
> > Remco Gerlich,  scarblac at pino.selwerd.nl
> > Hi! I'm a .sig virus! Join the fun and copy me into yours!
> 
> It is possible to get the files off of a site.  softbytelabs did it:
> http://www.softbytelabs.com/
> Their black widow program does a fine job of copying site files.  Maybe
> it's not done with HTTP, but does anyone know how it's done?  What does
> WebSucker do?  Thanks.
> 

websucker.py is just a convenient utility for sucking up websites.
Look in the Tools/webchecker subdirectory of your python installation. 
You can check out the code for yourself.

--Paul