[Python-Dev] Adding timeout to socket.py and httplib.py

[Alan Kennedy]

[Facundo]
> So, I have two modifications to make to the patch:
>
> - change the name to "create_connection"
> - make timeout obligatory named

I was going to suggest a third change: for orthogonality with the API
for socket objects, add a blocking parameter as well, i.e.

def create_connection(address, timeout=sentinel, blocking=sentinel):
  [snip]
    if timeout != sentinel:
      new_socket.settimeout(timeout)
    if blocking != sentinel:
      new_socket.setblocking(blocking)
  [snip]

but that may be taking it too far.

But there is still an issue remaining, relating to non-blocking IO.

With or without a blocking parameter, the user can still set
non-blocking behaviour on a socket by setting a timeout of 0. The
following snippet illustrates the issue.

#-=-=-=-=-=-=-=-=-=-=-=-=-=
import socket

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.settimeout(0)
s.connect( ("localhost", 80) )
#-=-=-=-=-=-=-=-=-=-=-=-=-=

If you run this, it is very likely to generate an exception, but not
guaranteed: you may have to run it a few times. Or try a host that is
slower to respond.

The problem is now that the connect call is now a non-blocking
connect, which means that it may throw a socket.error, even after a
successful connect, as follows

socket.error: (10035, 'The socket operation could not complete without
blocking')

The standard mechanism in C for doing a non-blocking connect is to
issue the connect call, and check the return value for a non-zero
error code. If this error code is errno.EAGAIN (code 10035), then the
call succeeded, but you should check back later for completion of the
operation.

It was for this reason that the connect_ex method was introduced to
python socket objects. Instead of raising an exception, it directly
returns the error code from the socket operation, so that it can be
checked, as in C.

So in the case of the new create_connection function, either

A: The user should be prepared to handle an exception if they use a
zero timeout (i.e. set non-blocking mode)

or

B: Detect the non-blocking case inside the function implementation and
return the value of the connect_ex method instead of the connect
method, as would be standard in a non-blocking app.

This could be implemented as follows

def create_connection(address, timeout=sentinel):
  [snip]
     if timeout != sentinel:
       new_socket.settimeout(timeout)
     if new_socket.gettimeout() == 0:
       result = new_socket.connect_ex(address)
     else:
       new_socket.connect(address)
       result = new_socket
  [snip]

I know that this makes it all more complex, and I'm *not* saying the
new function should be modified to include these concerns.

The new function is designed to address straightforward usability
cases, so it's perhaps appropriate that the API be restricted to those
cases, i.e. to supporting timeout values > 0. Perhaps this limit could
be coded into the function?

Also, people who want explicitly do non-blocking socket IO will likely
use the socket API directly, so it may not be worth supporting that
use in this function.

Regards,

Alan.