requests.{get,post} timeout

[Chris Angelico]

On Wed, Aug 23, 2017 at 5:10 AM, MRAB <python at mrabarnett.plus.com> wrote:
> On 2017-08-22 19:43, Chris Angelico wrote:
>>
>> On Wed, Aug 23, 2017 at 4:14 AM, Jon Ribbens <jon+usenet at unequivocal.eu>
>> wrote:
>>>
>>> On 2017-08-22, Chris Angelico <rosuav at gmail.com> wrote:
>>>>
>>>> On Wed, Aug 23, 2017 at 2:58 AM, Jon Ribbens <jon+usenet at unequivocal.eu>
>>>> wrote:
>>>>>
>>>>> Yes. There is no timeout feature that can be used to limit the total
>>>>> time a 'requests' request takes. Some people might think that this is
>>>>> a serious flaw in the requests library that would need urgent
>>>>> rectification in order to make the library safe and useful to use in
>>>>> almost any situation, but the 'requests' developers are apparently not
>>>>> among those people.
>>>>
>>>>
>>>> I'm not either. The idea of a timeout is to detect when something's
>>>> completely not working, not to limit the overall time to process.
>>>
>>>
>>> We appear to have different understandings of the word "timeout".
>>> I think it means a time, which if it runs out, will stop the operation.
>>>
>>> I am somewhat surprised that anyone might have a different definition
>>> - not least because, from a human being's point of view, they care
>>> about the overall time something takes to happen and telling them that
>>> nothing's wrong because technically we are still "successfully" receiving
>>> the expected 10 kilobytes of data 3 hours later is unlikely to make
>>> them happy.
>>
>>
>> You start downloading a file from a web page. It stalls out.
>>
>> Is it merely slow, and continuing to wait will get you a result?
>>
>> Or has it actually stalled out and you should give up?
>>
>> The low-level timeout will distinguish between those. If you want a
>> high-level timeout across the entire job, you can do that too, but
>> then you have to figure out exactly how long is "too long". Let's say
>> you set a thirty-second timeout. Great! Now someone uses your program
>> on a midrange connection to download a 100MB file, or on a poor
>> connection to download a 5MB file, or on dial-up to download a 10KB
>> file. Data is constantly flowing, but at some point, the connection
>> just dies, because it's hit your timeout. This is EXTREMELY
>> frustrating.
>>
>> You can always add in the overall timeout separately. If the low-level
>> timeout were implemented that way, there would be no way to externally
>> add the other form of timeout. Therefore the only sane way to
>> implement the request timeout is a between-byte limit.
>>
> You might want to have a way of setting the minimum data rate in order to
> defend against a slowloris attack.

That assumes that that's an attack - it often isn't. But if that's
what you want, then add that as a separate feature - it's distinct
from a timeout.

ChrisA