socket setdefaulttimeout

[Sheila King]

On 08/12/2005 22:37:22 Bryan Olson <fakeaddress at nowhere.org> wrote:

> Sheila King wrote:

>> I'm doing DNS lookups [...] it is important to make sure that the socket
>> doesn't just hang there waiting for a response.

>> After a recent system upgrade to Python 2.4.1 (from 2.2.2) I thought I
>> could take advantage of the setdefaulttimeout in the socket module, to
>> limit the amount of time the sockets take for a lookup.

>> As a test, I set the default timout ridiculously low.  But it doesn't
>> seem to be having any effect.

> The timeout applies to network communication on that socket, but not to
> calls such as socket.gethostbyname.  The gethostbyname function actually
> goes to the operating system, which can look up the name in a cache, or a
> hosts file, or query DNS servers on sockets of its own.

> Modern OS's generally have reasonably TCP/IP implementations, and the OS
> will handle applying a reasonable timeout.  Still gethostbyname and its
> brethren can be a pain for single- threaded event-driven programs, because
> they can block for significant time.

> Under some older threading systems, any system call would block every
> thread in the process, and gethostbyname was notorious for holding things
> up.  Some systems offer an asynchronous gethostbyname, but that doesn't
> help users of Python's library. Some programmers would keep around a few
> extra processes to handle their hosts lookups.  Fortunately, threading
> systems are now much better, and should only block the thread waiting for
> gethostbyname.

Thanks, Bryan. I'm not doing any threading. But we are running this script on 
incoming email as it arrives at the SMTP server, and scripts have a 16 second 
max time of execution. Generally they run in much less time. However, we have 
seen incidents where, due to issues with the DNS servers for the blacklists, 
that the script exceed it's max time to run and the process was killed by 
the OS. This results in the email being placed back into the mail queue for 
attempted re-delivery later. Of course, if this issue goes undetected, the 
mail can eventually be "returned to sender". There's no effective way to check 
from within the running filter script that the time is not exceeded if the 
gethostbyname blocks and doesn't return. :(

As I said, normally this isn't a problem. But there have been a handful of 
incidents where it did cause issues briefly over a few days. I was hoping to 
address it. :/

Sounds like I'm out of luck.

-- 
Sheila King
http://www.thinkspot.net/sheila/