Is Python good for web crawlers?

[Diez B. Roggisch]

Tempo wrote:

> Why do you say that the bottleneck of the crawler will always be
> downloading the page? Is it becasue there isn't already a modual to do
> this and I will have to start from scratch? Or a bandwidth issue?

Because of bandwidth - not necessarily yours directly, but the maximum flow
between your uplink and the site in question. It will always take at least
a fractioin of a second up to several seconds until the data is there - in
that time, lots of python code can run.

Diez