Web Crawler - Python or Perl?

[subeen]

On Jun 13, 1:26 am, Chuck Rhode <CRh... at LacusVeris.com> wrote:
> On Mon, 09 Jun 2008 10:48:03 -0700, disappearedng wrote:
> > I knowPythonbut notPerl, and I am interested in knowing which of
> > these two are a better choice.
>
> I'm partial to *Python*, but, the last time I looked, *urllib2* didn't
> provide a time-out mechanism that worked under all circumstances.  My
> client-side scripts would usually hang when the server quit
> responding, which happened a lot.
>

You can avoid the problem using the following code:
import socket

timeout = 300 # seconds
socket.setdefaulttimeout(timeout)

regards,
Subeen.
http://love-python.blogspot.com/