[Async-sig] SansIO (Was: PEP: asynchronous generators)

[Brett Cannon]

On Mon, 8 Aug 2016 at 11:12 Cory Benfield <cory at lukasa.co.uk> wrote:

>
> > On 8 Aug 2016, at 17:33, Yury Selivanov <yselivanov at gmail.com> wrote:
> >
> >
> >> On Aug 8, 2016, at 12:16 PM, Cory Benfield <cory at lukasa.co.uk> wrote:
> >>
> >>
> >>> On 8 Aug 2016, at 17:06, Yury Selivanov <yselivanov at gmail.com> wrote:
> >>>>
> >>>> As to having a PEP or putting something in PEP 8, I feel pretty
> lukewarm to those ideas. If the core Python team was able to legislate good
> coding practices via PEPs I think the world would be a very different place.
> >>>
> >>> What does this mean?  A lot of people, in any serious project at
> least, follow PEP 8 and coding with linters is a wide-spread practice.
> >>
> >> Sure, but that only works for things that linters can enforce, and I
> sincerely doubt that this is one of them.
> >
> > Correct.  I don’t think it should be PEP 8 either.  I think Guido’s idea
> on including h11 to the stdlib is cool, and that’s the better way to send
> the message.  We can also add a *new* informational PEP instructing people
> on why (and how) they should write IO free protocol implementations.  It
> would be great if you have time to champion such a PEP (I’d be glad to help
> if needed.)
>
> I am certainly happy to take the lead on a PEP like that if we believe
> there is value in it. I suspect I’d want to run it past this group quite a
> few times, because there are many others in this SIG (Brett, Nathaniel,
> Dave, and Glyph all jump to mind) that would have lots of useful things to
> say.
>

The lighter option is we put the equivalent of a PEP on sans-io.rtfd.io
instead of my dinky paragraph or three quickly explaining why this is an
important concept. That way there's no python-dev discussion and we can
update it w/o issue (I'm happy to add more contributors to the GH repo).


>
> > The other problem with sans-IO approach is that it takes longer to
> implement properly.  You need at least a synchronous and an asynchronous
> versions to make sure that you got the design parts “right”.
>
> Nah, that’s not necessary, at least for just the protocol part. You just
> need a test suite that tests it entirely in memory.


Right, so in the case of asyncpg it would be pulling out the binary
protocol parser and having that stand on its own. Then the asyncio part
that uses the binary protocol parser you wrote would be what asyncpg is w/
the protocol parser becomings pgparser or something.


> Necessarily, if you can test it with tests that don’t involve writing
> mocks for anything in the socket, select, selectors, or asyncio modules,
> you’re probably in a pretty good place to be arguing that you’re I/O free.
> If your tests only use the public interfaces, then you’re totally set.
>

Which "public interfaces" are you referring to? For me, any I/O-free
library shouldn't be driving the I/O, just a producer/consumer of data.
That means even if something follows e.g. the public interface of a socket
that it wouldn't qualify as that suggests the library gets to make the call
on when the I/O occurs and that you expose a socket for it to use.


>
> The *outer* layers are where you need to duplicate work, but you don’t
> need both of those right away. If you have the low-level no-I/O layer in
> place, you can start with just writing (say) the asyncio implementation and
> leave the sync implementation for later in the day.
>

The way I think of it is you make the I/O-free library do as much as
possible that allows it to be used in synchronous code, asyncio, and curio.
Then it's up to people who are interested in supporting those protocols
using those frameworks to wrap it as appropriate (if ever, e.g. maybe no
one ever cares about SMTP on curio).

-Brett


>
> I strongly, strongly advocate enforcing this distinction by splitting the
> no-I/O layer out into an entirely separate Python package that is
> separately versioned, and that you treat as third-party code from the
> perspective of your other modules. This has a nice tendency of getting into
> the headspace of thinking of it as an entirely separate module. In the case
> of OSS code, it also lets you push out the no-I/O version early: if your
> implementation really is faster than psycopg2, you might be able to
> convince someone else to come along and write the sync code instead!
>
> Anyway, I’ll stop now, because at a certain point I’ll just start writing
> that PEP in this email.
>
> Cory
>
>
> _______________________________________________
> Async-sig mailing list
> Async-sig at python.org
> https://mail.python.org/mailman/listinfo/async-sig
> Code of Conduct: https://www.python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/async-sig/attachments/20160809/cde25562/attachment.html>