[Async-sig] PEP: asynchronous generators

Mon Aug 8 06:16:25 EDT 2016

2016-08-08 8:56 GMT+02:00 Yarko Tymciurak <yarkot1 at gmail.com>:

>
>
> On Monday, August 8, 2016, Yarko Tymciurak <yarkot1 at gmail.com> wrote:
>
>> I still have to wonder, though, how an async repl, from the
>> inside-out,which handles a single task by default (synchronous equivalent)
>> would be anything less than explicit, or would complicate much (if anything
>> - I suspect a significant amount of the opposite).
>>
>> Regardless, I am grateful for the discussions.
>>
>> - Yarko
>>
>> On Sunday, August 7, 2016, Ludovic Gasc <gmludo at gmail.com> wrote:
>>
>>> +1 for PEP, nothing more to add from technical point of view.
>>> An extra step to the right direction, at least to me.
>>> Thank you Yury for that :-)
>>>
>>> About side conversation on sync/async split world, except to force
>>> coroutines pattern usage like in Go, I don't see how we can become more
>>> implicit.
>>> Even if the zen of Python recommands to prefer an explicit approach, I
>>> see more explicit/implicit as a balance you must adjust between
>>> Simplicity/Flexibility than a binary choice.
>>>
>>> To me, the success of Python as language is also because you have a
>>> good balance between theses approaches, and the last move from "yield from"
>>> to "await" illustrates that: Hide the internal mechanisms of
>>> implementation, but keep the explicit way to declare that.
>>>
>>> Like Andrew Svetlov, I don't believe a lot in the implicit approach of
>>> Gevent, because very quickly, you need to understand the extra tools, like
>>> synchronization primitives. The fact to know if you need to prefix with
>>> "await" or not the call of the functions is the tree that hides the forest.
>>>
>>> With the async pattern, it's impossible to hide everything and
>>> everything will work automagically: You must understand a little bit what's
>>> happening, or it will be very complicated to debug.
>>>
>>> To me, you can hide everything only if you are really sure it will work
>>> 100% of time without human intervention, like with autonomous Google cars.
>>>
>>> However, it might be interesting to have an async "linter", that should
>>> list all blocking I/O code in async coroutines, to help new comers to find
>>> this type of bugs.
>>> But with the dynamic nature of Python, I don't know if it's realistic to
>>> try to implement that.
>>> To me, it should be a better answer than to try to remove all sync/async
>>> code differences.
>>>
>>> Moreover, I see the need of async libs as an extra opportunity to
>>> challenge and simplify the Python toolbox.
>>>
>>> For now, with aiohttp, you have an unified API for HTTP in general,
>>> contrary in sync world with requests and flask for example.
>>> At least to me, a client and a server are only two sides of the same
>>> piece. More true with p2p protocols.
>>>
>>> As discussed several times, the next level might be more code reuse like
>>> suggested by David Beazley with SansIO, split protocol and I/O handling:
>>> https://twitter.com/dabeaz/status/761599925444550656?lang=fr
>>>
>>> https://github.com/brettcannon/sans-io
>>>
>>
> Question: isn't SansIO / Corey's work just a specific instance of Bob
> Martin's "Clean Architecture"?  It sounds familiar to me, when thinking of
> Brandon Rhode's 2014 PyOhio talk, and his recast of the topic in Warsaw in
> 2015.  It seems like it...  If so, then perhaps async aspects are just a
> second aspect.
>

Maybe that sync/async dichotomy might be a concrete use case to justify
this split.
Nevertheless, it shouldn't be the first case where, theoretically, it's
better to split layers, and finally, it's counter-productive during the
implementation.
Only one method to know that: Try to code to see what's happen.
Certainly some protocols/transports should be easier to have this split
than others: Interesting to know if somebody has already tried to have QUIC
and HTTP/2 in the same time with Python.

>
> I don't know yet if the benefit to share more code between implementations
>>> will be more important than the potential complexity code increase.
>>>
>>> The only point I'm sure for now: I'm preparing the pop-corn to watch the
>>> next episodes: curious to see what are the next ideas/implementations will
>>> emerge ;-)
>>> At least to me, it's more interesting than follow a TV serie, thank you
>>> for that :-)
>>>
>>> Have a nice week.
>>>
>>> Ludovic Gasc (GMLudo)
>>> http://www.gmludo.eu/
>>>
>>> On 29 Jul 2016 20:50, "Yarko Tymciurak" <yarkot1 at gmail.com> wrote:
>>>
>>>>
>>>>
>>>> On Friday, July 29, 2016, Yury Selivanov <yselivanov at gmail.com> wrote:
>>>>
>>>>> Comments inlined:
>>>>>
>>>>>
>>>>> > On Jul 29, 2016, at 2:20 PM, Yarko Tymciurak <yarkot1 at gmail.com>
>>>>> wrote:
>>>>> >
>>>>> > Hmm...  I think we need to think about a future where,
>>>>> programmatically, there's little-to no distinction between async and
>>>>> synchronous functions. Pushing this down deeper in the system is the way to
>>>>> go. For one, it will serve simple multi-core use once gilectomy is
>>>>> completed (it, or something effectively equivalent will complete).  For
>>>>> another, this is the path to reducing the functionally "useless" rewrite
>>>>> efforts of libraries (e.g. github.com/aio-libs), which somehow
>>>>> resemble all the efforts of migrating libraries from 2-3 (loosely).  The
>>>>> resistance and unexpected time that 2-3 migration experienced won't readily
>>>>> be mimicked in async tasks - too much effort to get computer and I/O bound
>>>>> benefits?  Maintain two versions of needed libraries, or jump languages is
>>>>> what will increasingly happen in the distributed (and more so IOT) world.
>>>>>
>>>>> When and *if* gilectomy is completed (or another project to remove the
>>>>> GIL), we will be able to do this:
>>>>>
>>>>> 1) Run existing async/await applications as is, but instead of running
>>>>> a process per core, we will be able to run a single process with many
>>>>> threads.  Likely one asyncio (or other) event loop per thread.  This is
>>>>> very speculative, but possible in theory.
>>>>>
>>>>> 2) Run existing blocking IO applications in several threads in one
>>>>> process.  This is something that only sounds like an easy thing to do, I
>>>>> suspect that a lot of code will break (or dead lock) when the GIL is
>>>>> removed.  Even if everything works perfectly well, threads aren’t answer to
>>>>> all problems — try to manage 1000s of them.
>>>>>
>>>>> Long story short, even if we had no GIL at all, having async/await
>>>>> (and non-blocking IO) would make sense.  And if you have async/await, with
>>>>> GIL or without, you will inevitably have different APIs and different IO
>>>>> low-level libs that drive them.
>>>>>
>>>>> There are ways to lessen the pain.  For instance, I like Cory’s
>>>>> approach with hyper - implement protocols separately from IO, so that it’s
>>>>> easy to port them to various sync and async frameworks.
>>>>>
>>>>> >
>>>>> > Time to think about paving the way to async-as first class citizen
>>>>> world.
>>>>> >
>>>>> > That's probably too much for this PEP, but the topic (a- prefixing)
>>>>> is a good canary for the bigger picture we need to start mulling over.
>>>>> >
>>>>> > So in this context (and in general w/ async) asking the question
>>>>> "can we make it so it doesn't matter?" is a good one to always be asking -
>>>>> it will get is there.
>>>>> >
>>>>>
>>>>> Unfortunately there is no way to use the same APIs for both
>>>>> async/await and synchronous world.  At least for CPython builtin types they
>>>>> have to have different names.
>>>>>
>>>>> I’m fine to discuss the ‘a’ prefix, but I’m a bit afraid that focusing
>>>>> on it too much will distract us from the PEP and details of it that really
>>>>> matter.
>>>>>
>>>>> Yury
>>>>
>>>>
>>>> To keep it simple, try thinking like this (and yes, Yury, apologies -
>>>> this is now a side discussion, and not about this pep):  everything in
>>>> CPython is async, and if you don't want async, you don't need to know
>>>> about, you run a single async task and don't need to know more...
>>>>
>>>> Can we get there?
>>>> That would be cool...
>>>>
>>>> - Yarko
>>>>
>>>> _______________________________________________
>>>> Async-sig mailing list
>>>> Async-sig at python.org
>>>> https://mail.python.org/mailman/listinfo/async-sig
>>>> Code of Conduct: https://www.python.org/psf/codeofconduct/
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/async-sig/attachments/20160808/2d3ba780/attachment.html>