Real-world use of concurrent.futures

[Chris Angelico]

On Fri, May 9, 2014 at 4:55 AM, Andrew McLean <lists at andros.org.uk> wrote:
> Because of latency in the DNS lookup this could
> benefit from multithreading.

Before you go too far down roads that are starting to look
problematic: A DNS lookup is a UDP packet out and a UDP packet in
(ignoring the possibility of TCP queries, which you probably won't be
doing here). Maybe it would be easier to implement it as asynchronous
networking? I don't know that Python makes it easy for you to
construct DNS requests and parse DNS responses; that's something more
in Pike's line of work. But it may be more possible to outright do the
DNS query asynchronously. TBH I haven't looked into it; but it's
another option to consider.

Separately from your programming model, though, how are you handling
timeouts? Any form of DNS error (NXDOMAIN being the most likely), and
the sort-of-error-but-not-error state of getting a response with no
answer, indicates that the address is invalid; but what if you just
don't hear back from the server? Will that mark addresses off as dead?

ChrisA