Is Python good for web crawlers?

[John J. Lee]

aleaxit at yahoo.com (Alex Martelli) writes:

> Magnus Lycka <lycka at carmen.se> wrote:
>    ...
> > I dunno, but there are these two guys, Sergey Brin and Lawrence Page,
> > who wrote a web crawler in Python. As far as I understood, they were
> > fairly successful with it. I think they called their system Koogle,
> > Bugle, or Gobble or something like that. Goo...can't remember.
> > 
> > See http://www-db.stanford.edu/~backrub/google.html
> 
> Yeah, I've heard of them, too.

I wonder if that little outfit has considered open-sourcing any of
their web client code?

(Declaring my interest: I'm maintaining, and very slowly developing,
some open-source libraries for web scraping and testing)


John