Question about asyncio and blocking operations

[Ian Kelly]

On Thu, Jan 28, 2016 at 2:23 PM, Maxime S <maxischmeii at gmail.com> wrote:
>
> 2016-01-28 17:53 GMT+01:00 Ian Kelly <ian.g.kelly at gmail.com>:
>>
>> On Thu, Jan 28, 2016 at 9:40 AM, Frank Millman <frank at chagford.com> wrote:
>>
>> > The caller requests some data from the database like this.
>> >
>> >    return_queue = asyncio.Queue()
>> >    sql = 'SELECT ...'
>> >    request_queue.put((return_queue, sql))
>>
>> Note that since this is a queue.Queue, the put call has the potential
>> to block your entire event loop.
>>
>
> Actually, I don't think you actually need an asyncio.Queue.
>
> You could use a simple deque as a buffer, and call fetchmany() when it is
> empty, like that (untested):

True. The asyncio Queue is really just a wrapper around a deque with
an interface designed for use with the producer-consumer pattern. If
the producer isn't a coroutine then it may not be appropriate.

This seems like a nice suggestion. Caution is advised if multiple
cursor methods are executed concurrently since they would be in
different threads and the underlying cursor may not be thread-safe.