Question about asyncio and blocking operations

[Ian Kelly]

On Tue, Jan 26, 2016 at 7:15 AM, Frank Millman <frank at chagford.com> wrote:
> I am making some progress, but I have found a snag - possibly unavoidable,
> but worth a mention.
>
> Usually when I retrieve rows from a database I iterate over the cursor -
>
>    def get_rows(sql, params):
>        cur.execute(sql, params)
>        for row in cur:
>            yield row
>
> If I create a Future to run get_rows(), I have to 'return' the result so
> that the caller can access it by calling future.result().
>
> If I return the cursor, I can iterate over it, but isn't this a blocking
> operation? As far as I know, the DB adaptor will only actually retrieve the
> row when requested.
>
> If I am right, I should call fetchall() while inside get_rows(), and return
> all the rows as a list.
>
> This seems to be swapping one bit of asynchronicity for another.
>
> Does this sound right?

You probably want an asynchronous iterator here. If the cursor doesn't
provide that, then you can wrap it in one. In fact, this is basically
one of the examples in the PEP:
https://www.python.org/dev/peps/pep-0492/#example-1