Recursing through a website

[Aahz Maruch]

In article <2C9S5.140107$78.43737982 at news3.rdc1.on.home.com>,
Colin Meeks <colin at programmedintegration.com> wrote:
>
>I want to write a program to recurse through my web site checking links
>and checking performance.  Does anybody have any examples of this.  I
>realise I'm probably going to need to use some kind of recursion, but
>must admit this isn't one of my best subjects.

Actually, you probably don't want recursion for something like this,
except possibly for developing your algorithm.  The problem is that
recusion takes up space on the stack, and you're quite likely to blow
your stack in a Real Life [tm] situation, unless you have some way of
controlling recursion depth (e.g. if you have a balanced B-Tree,
recursion is fine because you'd need literally billions of elements to
blow the depth).

If you want to see an implementation of a spider, take a look at
http://starship.python.net/crew/aahz/
(you'll also learn about threads along the way ;-)
-- 
                      --- Aahz (Copyright 2000 by aahz at pobox.com)

Androgynous poly kinky vanilla queer het    <*>     http://www.rahul.net/aahz/
Hugs and backrubs -- I break Rule 6

"as long as we like the same operating system, things are cool." -- piranha