requests.{get,post} timeout

[Chris Angelico]

On Fri, Aug 25, 2017 at 12:17 AM, Jon Ribbens <jon+usenet at unequivocal.eu> wrote:
> On 2017-08-24, Chris Angelico <rosuav at gmail.com> wrote:
>> On Thu, Aug 24, 2017 at 9:43 PM, Jon Ribbens <jon+usenet at unequivocal.eu> wrote:
>>> Where did you explain how it can be done without help? As far as I'm
>>> aware, you can't close the socket without help since you can't get
>>> access to it, and as you mentioned even if you were to do so the
>>> effect it would have on requests is completely undefined.
>>
>> In a single-threaded program, just hit Ctrl-C.
>
> By that, do you mean "kill the process"? That's obviously not
> a sensible answer in general, especially given we were including
> processes which have no terminal or user sitting there watching
> them.

Only in the sense that "kill" is the Unix term for "send signal".
Python catches the signal, the system call terminates with EINTR, and
the exception is raised. Give it a try.

(Caveat: I have no idea how this works on Windows. I do expect,
though, that it will abort the connection without terminating the
process, just like it does on Unix.)

>> Job done. No need to know ANYTHING about the internals of the
>> requests module, beyond that it has correct handling of signals.
>> Want that on a clock? SIGALRM.
>
> Doesn't work with threading.

How many of your programs have threads in them? Did you not read my
point where I said that the bulk of programs can use these simple
techniques?

>>> Even in a situation where there is a user constantly watching over the
>>> operation, there is still no solution as, while the user might well
>>> indicate their running out of patience by pressing 'cancel' or
>>> something, the process has no way to cancel the request in response to
>>> the user's command.
>>
>> Since any process-level signal will have the effect I describe, the
>> requests module HAS to cope with it.
>
> Receiving a signal is the same as closing the socket? What?
> (And as I already mentioned, you can't close the socket anyway.)

Not as such, but try it and see what actually happens. The signal
aborts the syscall; the exception causes the stack to be unwound. TRY
IT. It works.

>> Hence, it WILL all work. It might not be quite as efficient
>> ("resources not released until next query"), but it will be fully
>> functional. I don't know what you mean by "not working".
>
> Resources not released, subsequent operations failing, the library
> possibly left in a state from which it cannot recover. This is
> pretty obviously stuff that "is not working". Although even then
> you still haven't explained how we can abort the operation (even
> with all these side-effects) in the first place.

Not released UNTIL NEXT QUERY. Everything is recoverable. TRY IT. It works.

>> Basically, you have the potential for a dangling socket that isn't
>> being used for anything but is still an open file. Unless you're
>> creating a billion of them in quick succession, that's not going to
>> break your program.
>
> It would take many orders of magnitude fewer than a billion to break
> the program. This is not a responsible or sensible way to write a
> computer program - to deliberately let it leak stuff and invoke
> undefined behaviour all over the place and hope that somehow it'll
> muddle along regardless.

Leaking until the next call? A billion.

>>> Basically the only way to use requests safely is to fork an individual
>>> process for every request, which is of course spectacularly inefficient.
>>
>> Yes. Spectacularly inefficient and almost certainly unnecessary.
>
> You haven't suggested any workable alternative so far.

Have you tried any of what I said?

ChrisA