web searching scripts

[Jorgen Grahn]

On Fri, 04 Aug 2006 18:11:18 +0200, Avell Diroll <avelldiroll at yahoo.fr> wrote:
> julien.lord at gmail.com wrote:
>> Does anyone know of a freely available script that can take a given URL
>> and follow every link within it?
>> 
>> Ideally, I would like to start with this to build a quick application
>> to grab all the content off a website to publish it to a CD.
...
> If you just want to download websites (i.e. not necessarily writing a
> program yourself to do that), you may try Httrack, it might suite your
> needs.

The well-known Gnu wget is what I always use.

(IMHO, this is a situation where it's a /good/ idea to glue together existing
software, rather than joining many bits of code to a mirror-from-http-to-cdr
application.)

/Jorgen

-- 
  // Jorgen Grahn <grahn@        Ph'nglui mglw'nafh Cthulhu
\X/     snipabacken.dyndns.org>  R'lyeh wgah'nagl fhtagn!