requests.{get,post} timeout

[Marko Rauhamaa]

Chris Angelico <rosuav at gmail.com>:

> On Fri, Aug 25, 2017 at 12:17 AM, Jon Ribbens
> <jon+usenet at unequivocal.eu> wrote:
>> By that, do you mean "kill the process"? That's obviously not a
>> sensible answer in general, especially given we were including
>> processes which have no terminal or user sitting there watching them.
>
> Only in the sense that "kill" is the Unix term for "send signal".
> Python catches the signal, the system call terminates with EINTR, and
> the exception is raised. Give it a try.

Signals are an arcane Unix communication method. I strongly recommend
against using signals for anything but terminating a process, and even
then you have to be extra careful.

I have seen code that uses signals for runtime communication, but none
of it was free from race conditions.

>>>> Basically the only way to use requests safely is to fork an
>>>> individual process for every request, which is of course
>>>> spectacularly inefficient.
>>>
>>> Yes. Spectacularly inefficient and almost certainly unnecessary.

Processes are a nice way to exercise multiple CPUs and also a way to
deal with obnoxious synchronous function calls.

However, you don't want to fork a process (or a thread, for that matter)
for each context. Rather, you should have a pool of processes in the
order of, say, 2 * CPU count, and the processes should fetch work from a
queue.

And if a process should get stuck, killing it is trivial.


Marko