requests.{get,post} timeout

[Chris Angelico]

On Thu, Aug 24, 2017 at 9:43 PM, Jon Ribbens <jon+usenet at unequivocal.eu> wrote:
> On 2017-08-23, Chris Angelico <rosuav at gmail.com> wrote:
>> On Thu, Aug 24, 2017 at 8:54 AM, Jon Ribbens <jon+usenet at unequivocal.eu> wrote:
>>> On 2017-08-23, Chris Angelico <rosuav at gmail.com> wrote:
>>>> Yes and no. If requests provided a 'cancel query' feature, it would
>>>> play nicely with everything else, but (a) the entire concept here is
>>>> that the request has stalled, so you COULD just ignore the pending
>>>> query and pretend it's failed without actually cancelling it; and (b)
>>>> you could just close the underlying socket without help, but it might
>>>> mess up future queries that end up getting put onto the same socket.
>>>> It's not that you CAN'T do this without help (which is the case for a
>>>> "time between bytes" timeout), but that having help would allow
>>>> requests *itself* to benefit.
>>>
>>> I don't understand - in the above paragraph you first explain how
>>> it cannot be done without help from requests, then you state that it
>>> can be done without help from requests. Was your first explanation
>>> wrong?
>>
>> Not quite. I first explain that it can be done WITH help, and then
>> state that it can be done WITHOUT help. That it can be done with help
>> does not imply that it cannot be done without help.
>
> Where did you explain how it can be done without help? As far as I'm
> aware, you can't close the socket without help since you can't get
> access to it, and as you mentioned even if you were to do so the
> effect it would have on requests is completely undefined.

In a single-threaded program, just hit Ctrl-C. Job done. No need to
know ANYTHING about the internals of the requests module, beyond that
it has correct handling of signals. Want that on a clock? SIGALRM. You
only need the more complicated solutions if you can't do this (eg if
you want multithreading for othre reasons, or if SIGALRM doesn't work
on Windows - which is probably the case).

>> Help is nice but it mainly helps for *subsequent* requests; an
>> external abort might leave internal state somewhat messed up, which
>> would result in resources not being released until the next query, or
>> perhaps a query failing and getting retried. But even without help, it
>> would all work.
>
> We now appear to have different understandings of the word "work",
> which you are defining to include things that are clearly not working.
>
> Even in a situation where there is a user constantly watching over the
> operation, there is still no solution as, while the user might well
> indicate their running out of patience by pressing 'cancel' or
> something, the process has no way to cancel the request in response to
> the user's command.
>

Since any process-level signal will have the effect I describe, the
requests module HAS to cope with it. Hence, it WILL all work. It might
not be quite as efficient ("resources not released until next query"),
but it will be fully functional. I don't know what you mean by "not
working". Basically, you have the potential for a dangling socket that
isn't being used for anything but is still an open file. Unless you're
creating a billion of them in quick succession, that's not going to
break your program.

> Basically the only way to use requests safely is to fork an individual
> process for every request, which is of course spectacularly inefficient.

Yes. Spectacularly inefficient and almost certainly unnecessary.

ChrisA