[issue18907] urllib2.open FTP open times out at 20 secs despite timeout parameter

[Charles-François Natali]

Charles-François Natali added the comment:

Actually, I think urllib (actually ftplib) handles the timeout correctly.

Here's the result on my Linux box (replacing the server with localhost, with a firewall rule to drop packets to ftp port):

$ ./python ~/edgartimeouttest3.py 
Open of ftp://localhost/daily-index failed after 60.09 seconds: <urlopen error ftp error: timeout('timed out',)>

And indeed, here's what strace shows:

$ strace -ttT ./python ~/edgartimeouttest3.py 
23:15:46.953116 connect(3, {sa_family=AF_INET, sin_port=htons(21), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress) <0.000066>
23:15:46.953257 poll([{fd=3, events=POLLOUT}], 1, 60000) = 0 (Timeout) <60.011497>

See, we - correctly - pass 60s to poll()/select().

Now, I think I know what's going on in your case.
It's likely that your TCP/IP stack settings have shorter timeouts than mine, so the kernel reports a timeout on the socket *before the timeout expires*:

If I reduce the SYN retries limit (on Linux):
# sysctl net.ipv4.tcp_syn_retries=1
net.ipv4.tcp_syn_retries = 1

I now get:
$ ./python ~/edgartimeouttest3.py 
Open of ftp://localhost/daily-index failed after 3.03 seconds: <urlopen error ftp error: TimeoutError(110, 'Connection timed out')>

3s timeout, and yet, we still pass 60 seconds to poll():

23:19:43.756823 connect(3, {sa_family=AF_INET, sin_port=htons(21), sin_addr=inet_addr("127.0.0.1")}, 16) = -1 EINPROGRESS (Operation now in progress) <0.000099>
23:19:43.757034 poll([{fd=3, events=POLLOUT}], 1, 60000) = 1 ([{fd=3, revents=POLLOUT|POLLERR|POLLHUP}]) <3.003306>

In short, the kernel reports a timeout on the socket because it reached the maximum number of retries for SYN packets.

And actually, you can check that the timeouts are correctly processed by passing a smaller value: if it's smaller than the TCP/IP retries limit, you'll get exactly the expected timeout.

So I think it's not a Python bug, but rather an issue with your TCP/IP stack settings.

On Linux, can you try to increase the following sysctls:
# sysctl net.ipv4.tcp_syn_retries=7
# net.ipv4.tcp_retries1=7

And see if this solves your problem?

(BTW, are you "the" John Nagle?)

----------
nosy: +neologix

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18907>
_______________________________________