web crawler in python or C?

[Paul Rubin]

"abhinav" <abhinavduggal at gmail.com> writes:
> It is DSL broadband 128kbps.But thats not the point.

But it is the point.

> What i am saying is that would python be fine for implementing fast
> crawler algorithms or should i use C.Handling huge
> data,multithreading,file handling,heuristics for ranking,and
> maintaining huge data structures.What should be the language so as
> not to compromise that much on speed.What is the performance of
> python based crawlers vs C based crawlers.Should I use both the
> languages(partly C and python).How should i decide what part to be
> implemented in C and what should be done in python?  Please guide
> me.Thanks.

I think if you don't know how to answer these questions for yourself,
you're not ready to take on projects of that complexity.  My advice
is start in Python since development will be much easier.  If and when
you start hitting performance problems, you'll have to examine many
combinations of tactics for dealing with them, and switching languages
is just one such tactic.