[Python-Dev] PEP 492: What is the real goal?

[Yury Selivanov]

Hi Paul,

On 2015-04-29 2:26 PM, Paul Moore wrote:
> On 29 April 2015 at 18:43, Jim J. Jewett <jimjjewett at gmail.com> wrote:
>>> Rationale and Goals
>>> ===================
>>>
>>> Current Python supports implementing coroutines via generators (PEP
>>> 342), further enhanced by the ``yield from`` syntax introduced in PEP
>>> 380. This approach has a number of shortcomings:
>>>
>>> * it is easy to confuse coroutines with regular generators, since they
>>>    share the same syntax; async libraries often attempt to alleviate
>>>    this by using decorators (e.g. ``@asyncio.coroutine`` [1]_);
>> So?  PEP 492 never says what coroutines *are* in a way that explains
>> why it matters that they are different from generators.
> I agree. While I don't use coroutines/asyncio, and I may never do so,
> I will say that I find Python's approach very difficult to understand.
>
> I'd hope that the point of PEP 492, by making await/async first class
> language constructs, would be to make async programming more
> accessible in Python. Whether that will actually be the case isn't
> particularly clear to me. And whether "async programming" and
> "coroutines" are the same thing, I'm even less sure of. I haven't
> really followed the discussions here, because they seem to be about
> details that are completely confusing to me.

It will make it more accessible in Python.  asyncio is getting
a lot of traction, and with this PEP accepted I can see it
only becoming easier to work with it (or any other async
frameworks that start using the new syntax/protocols).

>
> In principle, I support the PEP, on the basis that working towards
> better coroutine/async support in Python seems worthwhile to me. But
> until the whole area is made more accessible to the average
> programmer, I doubt any of this will be more than a niche area in
> Python.
>
> For example, the PEP says:
>
> """
> New Coroutine Declaration Syntax
>
> The following new syntax is used to declare a coroutine:
>
> async def read_data(db):
>      pass
> """
>
> Looking at the Wikipedia article on coroutines, I see an example of
> how a producer/consumer process might be written with coroutines:
>
> var q := new queue
>
> coroutine produce
>      loop
>          while q is not full
>              create some new items
>              add the items to q
>          yield to consume
>
> coroutine consume
>      loop
>          while q is not empty
>              remove some items from q
>              use the items
>          yield to produce
>
> (To start everything off, you'd just run "produce").
>
> I can't even see how to relate that to PEP 429 syntax. I'm not allowed
> to use "yield", so should I use "await consume" in produce (and vice
> versa)? I'd actually expect to just write 2 generators in Python, and
> use .send() somehow (it's clunky and I can never remember how to write
> the calls, but that's OK, it just means that coroutines don't have
> first-class syntax support in Python). This is totally unrelated to
> asyncio, which is the core use case for all of Python's async support.
> But it's what I think of when I see the word "coroutine" (and
> Wikipedia agrees).

That Wikipedia page is very generic, and the pseudo-code
that it uses does indeed look confusing.

Here's how it might look like (this is the same pseudo-code
but tailored for PEP 492, not a real something)

   q = asyncio.Queue(maxsize=100)

   async def produce():
       # you might want to wrap it all in 'while True'

       while not q.full():
           item = create_item()
           await q.put(item)

   async def consume():
       while not q.empty():
           item = await q.get()
           process_item(item)


Thanks!
Yury