Question about asyncio and blocking operations

[Chris Angelico]

On Thu, Jan 28, 2016 at 10:11 PM, Frank Millman <frank at chagford.com> wrote:
>>
>> The other risk is that the wrong result will be queried (two async
>> tasks put something onto the queue - which one gets the first
>> result?), which could either be coped with by simple sequencing (maybe
>> this happens automatically, although I'd prefer a
>> mathematically-provable result to "it seems to work"), or by wrapping
>> the whole thing up in a function/class.
>>
>
> I *think* I have this one covered. When the caller makes a request, it
> creates an instance of an asyncio.Queue, and includes it with the request.
> The db handler uses this queue to send the result back.
>
> Do you see any problem with this?

Oh, I get it. In that case, you should be safe (at the cost of
efficiency, presumably, but probably immeasurably so).

The easiest way to thrash-test this is to simulate an ever-increasing
stream of requests that eventually get bottlenecked by the database.
For instance, have the database "process" a maximum of 10 requests a
second, and then start feeding it 11 requests a second. Monitoring the
queue length should tell you what's happening. Eventually, the queue
will hit its limit (and you can make that happen sooner by creating
the queue with a small maxsize), at which point the queue.put() will
block. My suspicion is that that's going to lock your entire backend,
unlocking only when the database takes something off its queue; you'll
end up throttling at the database's rate (which is fine), but with
something perpetually blocked waiting for the database (which is bad -
other jobs, like socket read/write, won't be happening).

As an alternative, you could use put_nowait or put(item, block=False)
to put things on the queue. If there's no room, it'll fail
immediately, which you can spit back to the user as "Database
overloaded, please try again later". Tuning the size of the queue
would then be an important consideration for real-world work; you'd
want it long enough to cope with bursty traffic, but short enough that
people's requests don't time out while they're blocked on the database
(it's much tidier to fail them instantly if that's going to happen).

ChrisA