[issue9641] httplib/ftplib: timeout parameter not applied correctly

[Anders Sandvig]

New submission from Anders Sandvig <anders.sandvig at gmail.com>:

>From <http://mail.python.org/pipermail/python-dev/2010-July/101266.html>:

Consider the following code for retreieving a web page using httplib:

   def get_url(hostname, port, url, timeout=5):
       con = httplib.HTTPConnection(hostname, port, timeout)
       con.request("GET", url)
       res = con.getresponse()
       data = res.read()
       return res, data

As expected, this will raise a socket.error if the client is unable to
connect before the timeout has expired. However, once the connection
has been made, the timeout parameter no longer has any effect and
con.getresponse() will block forever if the server does not send any
data.

I think the reason for this is that the socket object created in
HTTPConnection.connect() has a default timeout of 0 (i.e. it is never
set explicitly):

   def connect(self):
       """Connect to the host and port specified in __init__."""
       self.sock = socket.create_connection((self.host,self.port),
                                            self.timeout)


For now I have been able to work around this by manually setting the
timeout of the (private) socket object after calling con.request(),
like this:

       ...
       con.request("GET", url)
       con.sock.settimeout(timeout)
       res = con.getresponse()
       ...

However, I don't think this is a very good solution as it relies on
knowledge about the inner workings of httplib (and I had to read the
library source code to come up with it).

>From the top of my head, I can come up with three (four) ways of
properly solving the issue:

1) Documenting the timeout behavior and describing the above hack in
the httplib documentation.

2) Modify HTTPConnection.connect() to set the timeout on the socket
object after it has been  created (using the same timeout as given on
the HTTPConnection constructor).

3) Adding (optional) timeout parameters to
HTTPConnection.getresponse() and HTTPResponse.read() (and possibly
other functions with the same problem).

4) A combination of 2) and 3).

[...]

----------
components: Library (Lib)
messages: 114343
nosy: anders.sandvig
priority: normal
severity: normal
status: open
title: httplib/ftplib: timeout parameter not applied correctly
type: behavior
versions: Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9641>
_______________________________________