[Python-Dev] API design: where to add async variants of existing stdlib APIs?

[Michel Desmoulin]

Last week I had to download a CSV from an FTP and push any update on it
using websocket so asyncio was a natural fit and the network part went well.

The surprise was that the CSV part would not work as expected. Usually I
read csv doing:

import csv

file_like_object = csv_crawler.get_file()
for row in csv.DictReader(file_like_object)

But it didn't work because file_like_object.read() was a coroutine which
the csv module doesn't handle.

So I had to do:

import csv
import io

	raw_bytes = await stream.read(10000000)
        wrapped_bytes = io.BytesIO(raw_bytes)
        text = io.TextIOWrapper(wrapped_bytes, encoding=encoding,
errors='replace')

        for i, row in enumerate(csv.DictReader(text)):

Turns out I used asyncio a bit, and I now the stdlib, the io AIP, etc.
But for somebody that doesn't, it's not very easy to figure out. Plus
it's not as elegant as traditional Python. Not to mention it loads the
entire CSV in memory.

So I wondered if I could fix the csv module so it accept async. But the
question arised. Where should I put it ?

- Create AsyncDictReader and AsyncReader ?
- Add inspect.iscoroutine calls widh it in the regular Readers and some
__aiter__ and __aenter__ ?
- add a csv.async namespace ?

What API design are we recommanding for expose both sync and async
behaviors ?


Le 07/03/2017 à 03:08, Guido van Rossum a écrit :
> On Mon, Mar 6, 2017 at 5:57 PM, Raymond Hettinger
> <raymond.hettinger at gmail.com <mailto:raymond.hettinger at gmail.com>> wrote:
> 
>     Of course, it makes sense that anything not specific to asyncio
>     should go outside of asyncio.
> 
>     What I'm more concerned about is what the other places actually
>     are.   Rather than putting async variants of everything sprinkled
>     all over the standard library, I suggest collecting them all
>     together, perhaps in a new asynctools module.
> 
> 
> That's a tough design choice. I think neither extreme is particularly
> attractive -- having everything in an asynctools package might also
> bundle together thing that are entirely unrelated. In the extreme it
> would be like proposing that all metaclasses should go in a new
> "metaclasstools" package. I think we did a reasonable job with ABCs:
> core support goes in abc.py, support for collections ABCs goes into the
> collections package (in a submodule), and other packages and modules
> sometimes define ABCs for their own users.
> 
> Also, in some cases I expect we'll have to create a whole new module
> instead of updating some ancient piece of code with newfangled async
> variants to its outdated APIs.
> 
> -- 
> --Guido van Rossum (python.org/~guido <http://python.org/~guido>)
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/desmoulinmichel%40gmail.com
>