Alarms in threads - how to get/mimic?

[Lloyd Zusman]

Luca de Alfaro <dealfaro at eecs.berkeley.edu> writes:

> I am trying to write a multi-threaded program to read web pages.  The
> program spawns (say) 10 threads, assigning to each thread a file to
> read with the ftp or http protocol; each thread reads it, and returns
> the info.
> 
> The problem is that the threads sometimes hang for a very long time
> waiting for a connection (httplib seems to have very long timeouts??),
> whereas instead I don't care for the answer if it comes later than
> (say) a few seconds.
> 
> [ ... ]

Perhaps Timothy O'Malley's `timeoutsocket' class will help you.  It
allows you to set timeouts for selected socket operations, such as the
httplib connect calls you're referring to.  You can get it here ...

  http://www.timo-tasi.org/python/

Here's an excerpt from Timothy O'Malley's documentation:

  """Timeout Socket

  This module enables a timeout mechanism on all TCP connections.  It
  does this by inserting a shim on top of the socket module.  After
  this module has been imported, all socket creation goes through this
  shim.  As a result, every TCP connection will support a timeout.

  [ ... ]

  The timeout applies to the socket functions that normally block on
  execution:  read, write, connect, and accept.  If any of these 
  operations exceeds the specified timeout, the exception Timeout
  will be raised.

  [ ... ]

  """

Using this module, you can cause each httplib connection statement
which times out to throw an exception, and each thread can then
intelligently respond to its own exception.

I've been using this module for a similar purpose, and I'm quite
happy with it.


And by the way, I want to take this opportunity to express my public
thanks and congratulations to Timothy O'Malley for having created this
much-needed module.


Question to the Python community: the lack of a timeout mechanism
seems to be a glaring hole within the standard `socket' module's API.
Are there any plans to add this functionality to this module?  If so,
it should then be quite straightforward to encapsulate timeout
capabilities within the all the standard socket-based libraries such
as `httplib', `pop3lib', `imaplib', `smtplib', etc.


> [ ... ]
>
> Luca de Alfaro


-- 
 Lloyd Zusman
 ljz at asfast.com