simple httplib and urllib timeout question

[Amit Patel]

  <darrell at rubina.funkomite.com> wrote:

| Just a quick question, im writing a small webbot for a python
| learning experience and it comes across some sites that will make it
| hang for a while, if the site is down or extremely slow etc, is
| there a way to set a faster timeout for httplib . Or is there a
| better way to handle it ?


You can set an alarm using signal.alarm, and then you can throw an
exception in the signal handler.  I used to do this, but then I
discovered asyncore.  If you want to crawl multiple pages at once, you
might want to switch to asyncore.  I believe the asyncore documentation page
has an example HTTP client.

    - Amit

--
Amit J Patel, Computer Science Department, Stanford University
http://www-cs-students.stanford.edu/~amitp/