requests.{get,post} timeout

[Jon Ribbens]

On 2017-08-23, Chris Angelico <rosuav at gmail.com> wrote:
> On Wed, Aug 23, 2017 at 10:52 PM, Jon Ribbens <jon+usenet at unequivocal.eu> wrote:
>> Yes, what I was interested to learn was how the monitoring thread can
>> "cut off" the requesting thread.
>
> Ah, I see. That partly depends on your definition of "cut off", and
> how it needs to interact with other things. I'm not sure about the
> requests module specifically, but one very effective method of
> terminating a network query is to close the socket (since resources
> are owned by processes, not threads, any thread can close the
> underlying socket connection); it won't instantly terminate the
> thread, but it will result in an end-of-connection read shortly
> afterwards. You'd have to do some sort of high-level "hey, I'm
> cancelling this request" for the user's benefit, too - or maybe the
> user initiated it in the first place. For example, in a web browser,
> you can hit Esc to cancel the current page download; that can
> immediately close the socket, and it probably has to tell the cache
> subsystem not to hold that data, and maybe some logging and stuff, but
> in terms of aborting the download, closing the socket is usually
> sufficient.
>
> How this interacts with the actual specific details of the requests
> module I'm not sure, especially since a lot of the work usually
> happens inside requests.get() or one of its friends; but if you
> explicitly construct a Request object before sending it [1], it would
> be conceivable to have a "req.close()" or "req.abort()" method that
> closes the underlying socket. (Or possibly that goes on the
> PreparedRequest. Or maybe the Session. I don't know; normally, when I
> use requests, I use the higher level interfaces.) It would need to be
> a feature provided by requests ("abort this request"), as it would
> potentially interact with connection pooling and such. But at the
> simplest level, closing the socket WILL abort the connection.

OK cool, so circling back to where you were - which is the same place
that the 'requests' developers are - which is the claim that requests
does not need to provide an "overall timeout" feature because you
can cancel the request yourself is untrue since, as you explain above,
you cannot in fact cancel the request yourself without some sort of
support for this in the module itself. Sure, requests *could* provide
a "cancel" feature, just the same as it *could* provide an "overall
timeout" feature, but it doesn't actually provide either, and this
is a problem.