urllib2 FTP Weirdness

[Cameron Simpson]

On 07Feb2013 02:43, Steven D'Aprano <steve+comp.lang.python at pearwood.info> wrote:
| On Thu, 07 Feb 2013 10:06:32 +1100, Cameron Simpson wrote:
| > Timing. (Let me say I consider this scenario unlikely, very unlikely.
| > But...)
| > If the latter is consistently slightly slower 
| 
| On my laptop, the difference is of the order of 10 microseconds.

Like I said, I do not consider this likely.

| > then the firewall may be
| > an issue if active FTP is being used. "Active" FTP requires the FTP
| > server to connect to you to deliver the data: your end opens a listening
| > TCP socket and says "get", supplying the socket details.
| 
| If you are thinking that the socket gets closed if the read is delayed 
| too much, that doesn't explain the results you are getting. The read 
| succeeds when there is a delay, not when there is no delay. Almost as if 
| something is saying "oh, the read request came in too soon after the 
| connection was made, must block".

Exactly so.

For active FTP the firewall must accept an _inbound_ connection
from the server. If that connection's opening SYN packet comes in
_before_ the firewall has set up the special purpose rule to accept this
(remember, the fw is not the FTP client) then the firewall will quite
possibly _reject_ the inbound SYN packet, causing the server to see
"connection refused".

  client:
    open listening socket for data
    say "GET foo" to server, with socket details
  server:
    connect to the socket
    send data...

The firewall's in the middle, watching for the socket details. When it
sees them it must create an inbound forwarding rule to let the server's
inbound DATA connection through to the client. But the server believes
the socket is already available (because it is - the client makes it
before supplying the socket details) and may dispatch the DATA
connection before the firewall gets its rule in place.

| What can I say? I cannot reproduce the issue you are having. If you can 
| reproduce it, try again without the firewall. If bypassing the firewall 
| makes the issue go away, then go and yell at your network admins until 
| they fix it.

If it is a speed thing, it may not be fixable. The fix is to use PASV
mode FTP or better still to avoid FTP entirely. I certainly don't support
active FTP on the firewalls I administer.

Cheers,
-- 
Cameron Simpson <cs at zip.com.au>

Symbol?  What's a symbol?  This is a rose.
        - R.A. MacAvoy, _Tea with the Black Dragon_