urllib2 timeout issue

[Jérôme]

Hi.

Thank you all for your answers.

--------------------------------------------------
Context:

The problem I want to address is the code being stuck too long when the
network is down.

I'm working on a software gateway running on a Raspberry Pi, that forwards
data received through a radio link to the network.

https://github.com/Jerome-github/oem_gateway

This can be sending HTTP requests every 3 seconds, so a 10 secondes timeout
is an issue.
--------------------------------------------------

I was not at home when I wrote my last message. Now back home, I could try on
my own Debian Jessie machine (Python 2.7.5+) and I get the same results as
with the RaspberryPi (long timeout). So there seems to be something going on
with the router/modem.

Wed, 16 Oct 2013 12:04:22 +0200
Vincent Vande Vyvre a écrit:

> The url "dumdgdfgdgmyurl.com/" is just an example ?

Yes.

I wrote dummyurl.com as an example but unfortunately, this actually existed,
so I added gibberish to get a non-existing URL.

Wed, 16 Oct 2013 13:22:34 +0200
Peter Otten a écrit:

> The problem might be ipv6-related. 

I do have a inet6 line in ifconfig on my desktop, but nothing on my Pi:

wlan0     Link encap:Ethernet  HWaddr 80:1f:02:af:36:e4  
          inet addr:192.168.1.7  Bcast:192.168.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2644 errors:0 dropped:42297 overruns:0 frame:0
          TX packets:982 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:362298 (353.8 KiB)  TX bytes:127069 (124.0 KiB)

(I need to connect via Wi-Fi because my ethernet socket is dead thunder
struck...)

And I did this test:
(http://tldp.org/HOWTO/Linux+IPv6-HOWTO/systemcheck-kernel.html):

test -f /proc/net/if_inet6 && echo "Running kernel is IPv6 ready"

IPv6 works on my desktop only, not on the Pi. So it shouldn't be the
culprit, right ?

Wed, 16 Oct 2013 09:06:29 -0700
Tobiah a écrit:

> What happens when you use ping to resolve that address.  Do you get
> the same results?  If so, I'd say you have a DNS problem.  Maybe
> you have two DNS servers listed in /etc/resolv.conf or similar, and
> the first one is unavailable, so it takes 10 seconds to fail over
> to the second working server.

Good point. Same symptoms with ping. Long timeout on machine behind my router,
be it the Pi or the Debian PC.

/etc/resolv.conf:
domain home
search home
nameserver 192.168.1.254

I found this on my server's config:

options timeout:1

I added it to /etc/resolv.conf on my Pi and PC and I get a much shorter
timeout, yet still longer than the value I enter, longer than I expect, and
longer that what I have on other machines (instantaneous).

But at least I see something happening.

--------------------------------------------------

Should I conclude that the issue is in the DNS resolution ?

What can I do then ?

The /etc/resolv.conf trick looks more like a workaround. Besides, this file
is auto-modified. I know I can try to prevent modifications, either by
modifying privileges, or by adding "nohook resolv.conf" to /etc/dhcpcd.conf,
but that does not sound satisfying.

Anyway, it is still not instantaneous.

I don't see anything I can change in my router/modem config (it is closed and
belongs to my ISP, so I can only change what i'm allowed to).

Any other recommandation ?

I'm afraid I'm being out of topic, here.

Back to python, is it normal that urllib2's timeout does not apply to those
DNS delays ?

Is there absolutely nothing I can do from the software side to limit this
delay and give up if no answer is received before a second has passed ?

-- 
Jérôme