gethostbyname blocking

[Jean-Paul Calderone]

On Wed, 22 Apr 2009 08:08:51 -0700 (PDT), marc wyburn <marc.wyburn at googlemail.com> wrote:
>Hi, I am writing an asynchronous ping app to check if 1000s of hosts
>are alive very quickly.  Everything works extremely quickly unless the
>host name doesn't have a DNS record.
>
>when calling socket.gethostbyname if there is no record for the host
>the result seems to block all other threads.  As an example I have 6
>threads running and if I pass the class below a Queue with about 30
>valid addresses with one invalid address in the middle the thread that
>the exception occurs in seems to block the others.

If your system doesn't have the gethostbyname_r function, then you can
only make one gethostbyname_ex call at a time - a call to this function
will block any other calls to this function, since it is not safe to
call gethostbyname reentrantly.

I'm not sure what the easiest way to determine whether Python has found
gethostbyname_r or not on your system is.  The configure script used to
build Python will probably tell, but I doubt you have that lying around.
You could just assume this is the case, since you're observing behavior
consistent with it. ;)

An alternate approach would be to use a DNS library which doesn't have
this restriction.  For example, Twisted includes a DNS library which
is not implemented in terms of threads.  There are also a couple other
Python DNS client libraries around.

This isn't /quite/ the same as using gethostbyname, though, since
gethostbyname is controlled by system configuration in various ways,
and may do more than just DNS lookups (in fact, it may even be
configured not to use DNS at all!).  But it's pretty close, and may
be close enough for your purposes.

Jean-Paul