CPython thread starvation

[John Nagle]

On 4/27/2012 9:55 PM, Paul Rubin wrote:
> John Nagle<nagle at animats.com>  writes:
>>     I may do that to prevent the stall.  But the real problem was all
>> those DNS requests.  Parallizing them wouldn't help much when it took
>> hours to grind through them all.
>
> True dat.  But building a DNS cache into the application seems like a
> kludge.  Unless the number of requests is insane, running a caching
> nameserver on the local box seems cleaner.

    I know.  When I have a bit more time, I'll figure out why
CentOS 5 and Webmin didn't set up a caching DNS resolver by
default.

    Sometimes the number of requests IS insane.  When the
system hits a page with a thousand links, it has to resolve
all of them.  (Beyond a thousand links, we classify it as
link spam and stop.  The record so far is a page with over
10,000 links.)

				John Nagle