[Python-checkins] cpython (3.6): Issue #28091: Document PEP 525 & PEP 530.
ned.deily
python-checkins at python.org
Fri Dec 16 02:44:40 EST 2016
https://hg.python.org/cpython/rev/5a11263bb612
changeset: 105661:5a11263bb612
branch: 3.6
user: Yury Selivanov <yury at magic.io>
date: Thu Dec 15 17:36:05 2016 -0500
summary:
Issue #28091: Document PEP 525 & PEP 530.
Patch by Eric Appelt.
(grafted from 78c8f450b84ca1864123ec487d363eb151f61a4a)
files:
Doc/glossary.rst | 28 +++
Doc/library/asyncio-eventloop.rst | 18 +
Doc/library/inspect.rst | 21 ++
Doc/library/sys.rst | 36 +++
Doc/library/types.rst | 8 +
Doc/reference/compound_stmts.rst | 2 +-
Doc/reference/datamodel.rst | 19 ++
Doc/reference/expressions.rst | 166 +++++++++++++++++-
Doc/reference/simple_stmts.rst | 4 +
9 files changed, 298 insertions(+), 4 deletions(-)
diff --git a/Doc/glossary.rst b/Doc/glossary.rst
--- a/Doc/glossary.rst
+++ b/Doc/glossary.rst
@@ -74,6 +74,34 @@
:keyword:`async with` statement by defining :meth:`__aenter__` and
:meth:`__aexit__` methods. Introduced by :pep:`492`.
+ asynchronous generator
+ A function which returns an :term:`asynchronous generator iterator`. It
+ looks like a coroutine function defined with :keyword:`async def` except
+ that it contains :keyword:`yield` expressions for producing a series of
+ values usable in an :keyword:`async for` loop.
+
+ Usually refers to a asynchronous generator function, but may refer to an
+ *asynchronous generator iterator* in some contexts. In cases where the
+ intended meaning isn't clear, using the full terms avoids ambiguity.
+
+ An asynchronous generator function may contain :keyword:`await`
+ expressions as well as :keyword:`async for`, and :keyword:`async with`
+ statements.
+
+ asynchronous generator iterator
+ An object created by a :term:`asynchronous generator` function.
+
+ This is an :term:`asynchronous iterator` which when called using the
+ :meth:`__anext__` method returns an awaitable object which will execute
+ that the body of the asynchronous generator function until the
+ next :keyword:`yield` expression.
+
+ Each :keyword:`yield` temporarily suspends processing, remembering the
+ location execution state (including local variables and pending
+ try-statements). When the *asynchronous generator iterator* effectively
+ resumes with another awaitable returned by :meth:`__anext__`, it
+ picks-up where it left-off. See :pep:`492` and :pep:`525`.
+
asynchronous iterable
An object, that can be used in an :keyword:`async for` statement.
Must return an :term:`asynchronous iterator` from its
diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst
--- a/Doc/library/asyncio-eventloop.rst
+++ b/Doc/library/asyncio-eventloop.rst
@@ -88,6 +88,24 @@
This is idempotent and irreversible. No other methods should be called after
this one.
+
+.. coroutinemethod:: AbstractEventLoop.shutdown_asyncgens()
+
+ Schedule all currently open :term:`asynchronous generator` objects to
+ close with an :meth:`~agen.aclose()` call. After calling this method,
+ the event loop will issue a warning whenever a new asynchronous generator
+ is iterated. Should be used to finalize all scheduled asynchronous
+ generators reliably. Example::
+
+ try:
+ loop.run_forever()
+ finally:
+ loop.run_until_complete(loop.shutdown_asyncgens())
+ loop.close()
+
+ .. versionadded:: 3.6
+
+
.. _asyncio-pass-keywords:
Calls
diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst
--- a/Doc/library/inspect.rst
+++ b/Doc/library/inspect.rst
@@ -318,6 +318,27 @@
.. versionadded:: 3.5
+.. function:: isasyncgenfunction(object)
+
+ Return true if the object is an :term:`asynchronous generator` function,
+ for example::
+
+ >>> async def agen():
+ ... yield 1
+ ...
+ >>> inspect.isasyncgenfunction(agen)
+ True
+
+ .. versionadded:: 3.6
+
+
+.. function:: isasyncgen(object)
+
+ Return true if the object is an :term:`asynchronous generator iterator`
+ created by an :term:`asynchronous generator` function.
+
+ .. versionadded:: 3.6
+
.. function:: istraceback(object)
Return true if the object is a traceback.
diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst
--- a/Doc/library/sys.rst
+++ b/Doc/library/sys.rst
@@ -594,6 +594,24 @@
.. versionchanged:: 3.6
Added *platform_version*
+
+.. function:: get_asyncgen_hooks()
+
+ Returns an *asyncgen_hooks* object, which is similar to a
+ :class:`~collections.namedtuple` of the form `(firstiter, finalizer)`,
+ where *firstiter* and *finalizer* are expected to be either ``None`` or
+ functions which take an :term:`asynchronous generator iterator` as an
+ argument, and are used to schedule finalization of an asychronous
+ generator by an event loop.
+
+ .. versionadded:: 3.6
+ See :pep:`525` for more details.
+
+ .. note::
+ This function has been added on a provisional basis (see :pep:`411`
+ for details.)
+
+
.. function:: get_coroutine_wrapper()
Returns ``None``, or a wrapper set by :func:`set_coroutine_wrapper`.
@@ -1098,6 +1116,24 @@
implementation platform, rather than part of the language definition, and
thus may not be available in all Python implementations.
+.. function:: set_asyncgen_hooks(firstiter, finalizer)
+
+ Accepts two optional keyword arguments which are callables that accept an
+ :term:`asynchronous generator iterator` as an argument. The *firstiter*
+ callable will be called when an asynchronous generator is iterated for the
+ first time. The *finalizer* will be called when an asynchronous generator
+ is about to be garbage collected.
+
+ .. versionadded:: 3.6
+ See :pep:`525` for more details, and for a reference example of a
+ *finalizer* method see the implementation of
+ ``asyncio.Loop.shutdown_asyncgens`` in
+ :source:`Lib/asyncio/base_events.py`
+
+ .. note::
+ This function has been added on a provisional basis (see :pep:`411`
+ for details.)
+
.. function:: set_coroutine_wrapper(wrapper)
diff --git a/Doc/library/types.rst b/Doc/library/types.rst
--- a/Doc/library/types.rst
+++ b/Doc/library/types.rst
@@ -104,6 +104,14 @@
.. versionadded:: 3.5
+.. data:: AsyncGeneratorType
+
+ The type of :term:`asynchronous generator`-iterator objects, created by
+ asynchronous generator functions.
+
+ .. versionadded:: 3.6
+
+
.. data:: CodeType
.. index:: builtin: compile
diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst
--- a/Doc/reference/compound_stmts.rst
+++ b/Doc/reference/compound_stmts.rst
@@ -697,7 +697,7 @@
Functions defined with ``async def`` syntax are always coroutine functions,
even if they do not contain ``await`` or ``async`` keywords.
-It is a :exc:`SyntaxError` to use :keyword:`yield` expressions in
+It is a :exc:`SyntaxError` to use ``yield from`` expressions in
``async def`` coroutines.
An example of a coroutine function::
diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst
--- a/Doc/reference/datamodel.rst
+++ b/Doc/reference/datamodel.rst
@@ -627,6 +627,25 @@
as well as :keyword:`async with` and :keyword:`async for` statements. See
also the :ref:`coroutine-objects` section.
+ Asynchronous generator functions
+ .. index::
+ single: asynchronous generator; function
+ single: asynchronous generator; asynchronous iterator
+
+ A function or method which is defined using :keyword:`async def` and
+ which uses the :keyword:`yield` statement is called a
+ :dfn:`asynchronous generator function`. Such a function, when called,
+ returns an asynchronous iterator object which can be used in an
+ :keyword:`async for` statement to execute the body of the function.
+
+ Calling the asynchronous iterator's :meth:`aiterator.__anext__` method
+ will return an :term:`awaitable` which when awaited
+ will execute until it provides a value using the :keyword:`yield`
+ expression. When the function executes an empty :keyword:`return`
+ statement or falls off the end, a :exc:`StopAsyncIteration` exception
+ is raised and the asynchronous iterator will have reached the end of
+ the set of values to be yielded.
+
Built-in functions
.. index::
object: built-in function
diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst
--- a/Doc/reference/expressions.rst
+++ b/Doc/reference/expressions.rst
@@ -172,7 +172,7 @@
.. productionlist::
comprehension: `expression` `comp_for`
- comp_for: "for" `target_list` "in" `or_test` [`comp_iter`]
+ comp_for: [ASYNC] "for" `target_list` "in" `or_test` [`comp_iter`]
comp_iter: `comp_for` | `comp_if`
comp_if: "if" `expression_nocond` [`comp_iter`]
@@ -186,6 +186,17 @@
Note that the comprehension is executed in a separate scope, so names assigned
to in the target list don't "leak" into the enclosing scope.
+Since Python 3.6, in an :keyword:`async def` function, an :keyword:`async for`
+clause may be used to iterate over a :term:`asynchronous iterator`.
+A comprehension in an :keyword:`async def` function may consist of either a
+:keyword:`for` or :keyword:`async for` clause following the leading
+expression, may contan additonal :keyword:`for` or :keyword:`async for`
+clauses, and may also use :keyword:`await` expressions.
+If a comprehension contains either :keyword:`async for` clauses
+or :keyword:`await` expressions it is called an
+:dfn:`asynchronous comprehension`. An asynchronous comprehension may
+suspend the execution of the coroutine function in which it appears.
+See also :pep:`530`.
.. _lists:
@@ -315,6 +326,14 @@
The parentheses can be omitted on calls with only one argument. See section
:ref:`calls` for details.
+Since Python 3.6, if the generator appears in an :keyword:`async def` function,
+then :keyword:`async for` clauses and :keyword:`await` expressions are permitted
+as with an asynchronous comprehension. If a generator expression
+contains either :keyword:`async for` clauses or :keyword:`await` expressions
+it is called an :dfn:`asynchronous generator expression`.
+An asynchronous generator expression yields a new asynchronous
+generator object, which is an asynchronous iterator
+(see :ref:`async-iterators`).
.. _yieldexpr:
@@ -330,9 +349,22 @@
yield_atom: "(" `yield_expression` ")"
yield_expression: "yield" [`expression_list` | "from" `expression`]
-The yield expression is only used when defining a :term:`generator` function and
+The yield expression is used when defining a :term:`generator` function
+or an :term:`asynchronous generator` function and
thus can only be used in the body of a function definition. Using a yield
-expression in a function's body causes that function to be a generator.
+expression in a function's body causes that function to be a generator,
+and using it in an :keyword:`async def` function's body causes that
+coroutine function to be an asynchronous generator. For example::
+
+ def gen(): # defines a generator function
+ yield 123
+
+ async def agen(): # defines an asynchronous generator function (PEP 525)
+ yield 123
+
+Generator functions are described below, while asynchronous generator
+functions are described separately in section
+:ref:`asynchronous-generator-functions`.
When a generator function is called, it returns an iterator known as a
generator. That generator then controls the execution of the generator function.
@@ -496,6 +528,134 @@
For examples using ``yield from``, see :ref:`pep-380` in "What's New in
Python."
+.. _asynchronous-generator-functions:
+
+Asynchronous generator functions
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The presence of a yield expression in a function or method defined using
+:keyword:`async def` further defines the function as a
+:term:`asynchronous generator` function.
+
+When an asynchronous generator function is called, it returns an
+asynchronous iterator known as an asynchronous generator object.
+That object then controls the execution of the generator function.
+An asynchronous generator object is typically used in an
+:keyword:`async for` statement in a coroutine function analogously to
+how a generator object would be used in a :keyword:`for` statement.
+
+Calling one of the asynchronous generator's methods returns an
+:term:`awaitable` object, and the execution starts when this object
+is awaited on. At that time, the execution proceeds to the first yield
+expression, where it is suspended again, returning the value of
+:token:`expression_list` to the awaiting coroutine. As with a generator,
+suspension means that all local state is retained, including the
+current bindings of local variables, the instruction pointer, the internal
+evaluation stack, and the state of any exception handling. When the execution
+is resumed by awaiting on the next object returned by the asynchronous
+generator's methods, the function can proceed exactly as if the yield
+expression were just another external call. The value of the yield expression
+after resuming depends on the method which resumed the execution. If
+:meth:`~agen.__anext__` is used then the result is :const:`None`. Otherwise, if
+:meth:`~agen.asend` is used, then the result will be the value passed in to
+that method.
+
+In an asynchronous generator function, yield expressions are allowed anywhere
+in a :keyword:`try` construct. However, if an asynchronous generator is not
+resumed before it is finalized (by reaching a zero reference count or by
+being garbage collected), then a yield expression within a :keyword:`try`
+construct could result in a failure to execute pending :keyword:`finally`
+clauses. In this case, it is the responsibility of the event loop or
+scheduler running the asynchronous generator to call the asynchronous
+generator-iterator's :meth:`~agen.aclose` method and run the resulting
+coroutine object, thus allowing any pending :keyword:`finally` clauses
+to execute.
+
+To take care of finalization, an event loop should define
+a *finalizer* function which takes an asynchronous generator-iterator
+and presumably calls :meth:`~agen.aclose` and executes the coroutine.
+This *finalizer* may be registered by calling :func:`sys.set_asyncgen_hooks`.
+When first iterated over, an asynchronous generator-iterator will store the
+registered *finalizer* to be called upon finalization. For a reference example
+of a *finalizer* method see the implementation of
+``asyncio.Loop.shutdown_asyncgens`` in :source:`Lib/asyncio/base_events.py`.
+
+The expression ``yield from <expr>`` is a syntax error when used in an
+asynchronous generator function.
+
+.. index:: object: asynchronous-generator
+.. _asynchronous-generator-methods:
+
+Asynchronous generator-iterator methods
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This subsection describes the methods of an asynchronous generator iterator,
+which are used to control the execution of a generator function.
+
+
+.. index:: exception: StopAsyncIteration
+
+.. coroutinemethod:: agen.__anext__()
+
+ Returns an awaitable which when run starts to execute the asynchronous
+ generator or resumes it at the last executed yield expression. When an
+ asynchronous generator function is resumed with a :meth:`~agen.__anext__`
+ method, the current yield expression always evaluates to :const:`None` in
+ the returned awaitable, which when run will continue to the next yield
+ expression. The value of the :token:`expression_list` of the yield
+ expression is the value of the :exc:`StopIteration` exception raised by
+ the completing coroutine. If the asynchronous generator exits without
+ yielding another value, the awaitable instead raises an
+ :exc:`StopAsyncIteration` exception, signalling that the asynchronous
+ iteration has completed.
+
+ This method is normally called implicitly by a :keyword:`async for` loop.
+
+
+.. coroutinemethod:: agen.asend(value)
+
+ Returns an awaitable which when run resumes the execution of the
+ asynchronous generator. As with the :meth:`~generator.send()` method for a
+ generator, this "sends" a value into the asynchronous generator function,
+ and the *value* argument becomes the result of the current yield expression.
+ The awaitable returned by the :meth:`asend` method will return the next
+ value yielded by the generator as the value of the raised
+ :exc:`StopIteration`, or raises :exc:`StopAsyncIteration` if the
+ asynchronous generator exits without yielding another value. When
+ :meth:`asend` is called to start the asynchronous
+ generator, it must be called with :const:`None` as the argument,
+ because there is no yield expression that could receive the value.
+
+
+.. coroutinemethod:: agen.athrow(type[, value[, traceback]])
+
+ Returns an awaitable that raises an exception of type ``type`` at the point
+ where the asynchronous generator was paused, and returns the next value
+ yielded by the generator function as the value of the raised
+ :exc:`StopIteration` exception. If the asynchronous generator exits
+ without yielding another value, an :exc:`StopAsyncIteration` exception is
+ raised by the awaitable.
+ If the generator function does not catch the passed-in exception, or
+ raises a different exception, then when the awaitalbe is run that exception
+ propagates to the caller of the awaitable.
+
+.. index:: exception: GeneratorExit
+
+
+.. coroutinemethod:: agen.aclose()
+
+ Returns an awaitable that when run will throw a :exc:`GeneratorExit` into
+ the asynchronous generator function at the point where it was paused.
+ If the asynchronous generator function then exits gracefully, is already
+ closed, or raises :exc:`GeneratorExit` (by not catching the exception),
+ then the returned awaitable will raise a :exc:`StopIteration` exception.
+ Any further awaitables returned by subsequent calls to the asynchronous
+ generator will raise a :exc:`StopAsyncIteration` exception. If the
+ asynchronous generator yields a value, a :exc:`RuntimeError` is raised
+ by the awaitable. If the asynchronous generator raises any other exception,
+ it is propagated to the caller of the awaitable. If the asynchronous
+ generator has already exited due to an exception or normal exit, then
+ further calls to :meth:`aclose` will return an awaitable that does nothing.
.. _primaries:
diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst
--- a/Doc/reference/simple_stmts.rst
+++ b/Doc/reference/simple_stmts.rst
@@ -492,6 +492,10 @@
value (if any) is used as an argument to construct :exc:`StopIteration` and
becomes the :attr:`StopIteration.value` attribute.
+In an asynchronous generator function, an empty :keyword:`return` statement
+indicates that the asynchronous generator is done and will cause
+:exc:`StopAsyncIteration` to be raised. A non-empty :keyword:`return`
+statement is a syntax error in an asynchronous generator function.
.. _yield:
--
Repository URL: https://hg.python.org/cpython
More information about the Python-checkins
mailing list