[Async-sig] APIs for high-bandwidth large I/O?

[Ludovic Gasc]

Hi Antoine,

First, thanks a lot to share with us the results of your research, it's
interesting, even if, for me, I'm not the first concerned with that: Most
of our network data we handle with AsyncIO have enough space with a simple
UDP packet ;-)

Anyway, I have questions about your suggestions, see below:

2017-10-18 20:04 GMT+02:00 Antoine Pitrou <solipsis at pitrou.net>:

>
> Also, since asyncio is the de facto standard now, I wonder if asyncio
> might grow such a new API.  That may be troublesome: asyncio already
> has Protocols and Streams, and people often complain about its
> extensive API surface that's difficult for beginners :-)
>

>From my point of view, I like a lot to code TCP servers with the Streams
API of AsyncIO.
When I see your piece of code for Tornado:
https://gist.github.com/pitrou/0f772867008d861c4aa2d2d7b846bbf0
It looks very similar of the Streams API.

To my understanding, some AsyncIO libraries, like aiohttp, don't
use Streams API of AsyncIO and implement a specific implementation,
especially to have a full control on the buffers: based on the information
provided inside the protocol, you are able to know if a small payload or a
big payload will arrive on the wire, like Content-Length header with HTTP.

My question is: Do you think it's possible to have a simple API to fit at
the same time small payloads and big payloads, without impacts on
efficiency for all use cases ?
Or it's too much integrated with the protocol implementation itself to
converge until a simple solution for all use cases ?

Maybe before to think about an AsyncIO improvement, it might be a new
library at the first step ?
To let decide people to use this new buffering algorithm ?

Thanks for your responses.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/async-sig/attachments/20171022/e756bd9b/attachment.html>