[Python-checkins] peps: Add PEP 492 by Yury Selivanov.
berker.peksag
python-checkins at python.org
Fri Apr 17 22:43:46 CEST 2015
https://hg.python.org/peps/rev/e532f0629fc3
changeset: 5761:e532f0629fc3
user: Berker Peksag <berker.peksag at gmail.com>
date: Fri Apr 17 23:43:55 2015 +0300
summary:
Add PEP 492 by Yury Selivanov.
files:
pep-0492.txt | 1014 ++++++++++++++++++++++++++++++++++++++
1 files changed, 1014 insertions(+), 0 deletions(-)
diff --git a/pep-0492.txt b/pep-0492.txt
new file mode 100644
--- /dev/null
+++ b/pep-0492.txt
@@ -0,0 +1,1014 @@
+PEP: 492
+Title: Coroutines with async and await syntax
+Version: $Revision$
+Last-Modified: $Date$
+Author: Yury Selivanov <yselivanov at sprymix.com>
+Status: Draft
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 09-Apr-2015
+Python-Version: 3.5
+
+
+Abstract
+========
+
+This PEP introduces new syntax for coroutines, asynchronous ``with``
+statements and ``for`` loops. The main motivation behind this proposal is to
+streamline writing and maintaining asynchronous code, as well as to simplify
+previously hard to implement code patterns.
+
+
+Rationale and Goals
+===================
+
+Current Python supports implementing coroutines via generators (PEP 342),
+further enhanced by the ``yield from`` syntax introduced in PEP 380.
+This approach has a number of shortcomings:
+
+* it is easy to confuse coroutines with regular generators, since they share
+ the same syntax; async libraries often attempt to alleviate this by using
+ decorators (e.g. ``@asyncio.coroutine`` [1]_);
+
+* it is not possible to natively define a coroutine which has no ``yield``
+ or ``yield from`` statements, again requiring the use of decorators to
+ fix potential refactoring issues;
+
+* support for asynchronous calls is limited to expressions where ``yield`` is
+ allowed syntactically, limiting the usefulness of syntactic features, such
+ as ``with`` and ``for`` statements.
+
+This proposal makes coroutines a native Python language feature, and clearly
+separates them from generators. This removes generator/coroutine ambiguity,
+and makes it possible to reliably define coroutines without reliance on a
+specific library. This also enables linters and IDEs to improve static code
+analysis and refactoring.
+
+Native coroutines and the associated new syntax features make it possible
+to define context manager and iteration protocols in asynchronous terms.
+As shown later in this proposal, the new ``async with`` statement lets Python
+programs perform asynchronous calls when entering and exiting a runtime
+context, and the new ``async for`` statement makes it possible to perform
+asynchronous calls in iterators.
+
+
+Specification
+=============
+
+This proposal introduces new syntax and semantics to enhance coroutine support
+in Python, it does not change the internal implementation of coroutines, which
+are still based on generators.
+
+It is strongly suggested that the reader understands how coroutines are
+implemented in Python (PEP 342 and PEP 380). It is also recommended to read
+PEP 3156 (asyncio framework).
+
+From this point in this document we use the word *coroutine* to refer to
+functions declared using the new syntax. *generator-based coroutine* is used
+where necessary to refer to coroutines that are based on generator syntax.
+
+
+New Coroutine Declaration Syntax
+--------------------------------
+
+The following new syntax is used to declare a coroutine::
+
+ async def read_data(db):
+ pass
+
+Key properties of coroutines:
+
+* Coroutines are always generators, even if they do not contain ``await``
+ expressions.
+
+* It is a ``SyntaxError`` to have ``yield`` or ``yield from`` expressions in
+ an ``async`` function.
+
+* Internally, a new code object flag - ``CO_ASYNC`` - is introduced to enable
+ runtime detection of coroutines (and migrating existing code).
+ All coroutines have both ``CO_ASYNC`` and ``CO_GENERATOR`` flags set.
+
+* Regular generators, when called, return a *generator object*; similarly,
+ coroutines return a *coroutine object*.
+
+* ``StopIteration`` exceptions are not propagated out of coroutines, and are
+ replaced with a ``RuntimeError``. For regular generators such behavior
+ requires a future import (see PEP 479).
+
+
+types.async_def()
+-----------------
+
+A new function ``async_def(gen)`` is added to the ``types`` module. It
+applies ``CO_ASYNC`` flag to the passed generator's code object, so that it
+returns a *coroutine object* when called.
+
+This feature enables an easy upgrade path for existing libraries.
+
+
+Await Expression
+----------------
+
+The following new ``await`` expression is used to obtain a result of coroutine
+execution::
+
+ async def read_data(db):
+ data = await db.fetch('SELECT ...')
+ ...
+
+``await``, similarly to ``yield from``, suspends execution of ``read_data``
+coroutine until ``db.fetch`` *awaitable* completes and returns the result
+data.
+
+It uses the ``yield from`` implementation with an extra step of validating its
+argument. ``await`` only accepts an *awaitable*, which can be one of:
+
+* A *coroutine object* returned from a coroutine or a generator decorated with
+ ``types.async_def()``.
+
+* An object with an ``__await__`` method returning an iterator.
+
+ Any ``yield from`` chain of calls ends with a ``yield``. This is a
+ fundamental mechanism of how *Futures* are implemented. Since, internally,
+ coroutines are a special kind of generators, every ``await`` is suspended by
+ a ``yield`` somewhere down the chain of ``await`` calls (please refer to PEP
+ 3156 for a detailed explanation.)
+
+ To enable this behavior for coroutines, a new magic method called
+ ``__await__`` is added. In asyncio, for instance, to enable Future objects
+ in ``await`` statements, the only change is to add ``__await__ = __iter__``
+ line to ``asyncio.Future`` class.
+
+ Objects with ``__await__`` method are called *Future-like* objects in the
+ rest of this PEP.
+
+ Also, please note that ``__aiter__`` method (see its definition below) cannot
+ be used for this purpose. It is a different protocol, and would be like
+ using ``__iter__`` instead of ``__call__`` for regular callables.
+
+It is a ``SyntaxError`` to use ``await`` outside of a coroutine.
+
+
+Asynchronous Context Managers and "async with"
+----------------------------------------------
+
+An *asynchronous context manager* is a context manager that is able to suspend
+execution in its *enter* and *exit* methods.
+
+To make this possible, a new protocol for asynchronous context managers is
+proposed. Two new magic methods are added: ``__aenter__`` and ``__aexit__``.
+Both must return an *awaitable*.
+
+An example of an asynchronous context manager::
+
+ class AsyncContextManager:
+ async def __aenter__(self):
+ await log('entering context')
+
+ async def __aexit__(self, exc_type, exc, tb):
+ await log('exiting context')
+
+
+New Syntax
+''''''''''
+
+A new statement for asynchronous context managers is proposed::
+
+ async with EXPR as VAR:
+ BLOCK
+
+
+which is semantically equivalent to::
+
+ mgr = (EXPR)
+ aexit = type(mgr).__aexit__
+ aenter = type(mgr).__aenter__(mgr)
+ exc = True
+
+ try:
+ try:
+ VAR = await aenter
+ BLOCK
+ except:
+ exc = False
+ exit_res = await aexit(mgr, *sys.exc_info())
+ if not exit_res:
+ raise
+
+ finally:
+ if exc:
+ await aexit(mgr, None, None, None)
+
+
+As with regular ``with`` statements, it is possible to specify multiple context
+managers in a single ``async with`` statement.
+
+It is an error to pass a regular context manager without ``__aenter__`` and
+``__aexit__`` methods to ``async with``. It is a ``SyntaxError`` to use
+``async with`` outside of a coroutine.
+
+
+Example
+'''''''
+
+With asynchronous context managers it is easy to implement proper database
+transaction managers for coroutines::
+
+ async def commit(session, data):
+ ...
+
+ async with session.transaction():
+ ...
+ await session.update(data)
+ ...
+
+Code that needs locking also looks lighter::
+
+ async with lock:
+ ...
+
+instead of::
+
+ with (yield from lock):
+ ...
+
+
+Asynchronous Iterators and "async for"
+--------------------------------------
+
+An *asynchronous iterable* is able to call asynchronous code in its *iter*
+implementation, and *asynchronous iterator* can call asynchronous code in its
+*next* method. To support asynchronous iteration:
+
+1. An object must implement an ``__aiter__`` method returning an *awaitable*
+ resulting in an *asynchronous iterator object*.
+
+2. An *asynchronous iterator object* must implement an ``__anext__`` method
+ returning an *awaitable*.
+
+3. To stop iteration```__anext__`` must raise a ``StopAsyncIteration``
+ exception.
+
+An example of asynchronous iterable::
+
+ class AsyncIterable:
+ async def __aiter__(self):
+ return self
+
+ async def __anext__(self):
+ data = await self.fetch_data()
+ if data:
+ return data
+ else:
+ raise StopAsyncIteration
+
+ async def fetch_data(self):
+ ...
+
+
+New Syntax
+''''''''''
+
+A new statement for iterating through asynchronous iterators is proposed::
+
+ async for TARGET in ITER:
+ BLOCK
+ else:
+ BLOCK2
+
+which is semantically equivalent to::
+
+ iter = (ITER)
+ iter = await type(iter).__aiter__(iter)
+ running = True
+ while running:
+ try:
+ TARGET = await type(iter).__anext__(iter)
+ except StopAsyncIteration:
+ running = False
+ else:
+ BLOCK
+ else:
+ BLOCK2
+
+
+It is an error to pass a regular iterable without ``__aiter__`` method to
+``async for``. It is a ``SyntaxError`` to use ``async for`` outside of a
+coroutine.
+
+As for with regular ``for`` statement, ``async for`` has an optional ``else``
+clause.
+
+
+Example 1
+'''''''''
+
+With asynchronous iteration protocol it is possible to asynchronously buffer
+data during iteration::
+
+ async for data in cursor:
+ ...
+
+Where ``cursor`` is an asynchronous iterator that prefetches ``N`` rows
+of data from a database after every ``N`` iterations.
+
+The following code illustrates new asynchronous iteration protocol::
+
+ class Cursor:
+ def __init__(self):
+ self.buffer = collections.deque()
+
+ def _prefetch(self):
+ ...
+
+ async def __aiter__(self):
+ return self
+
+ async def __anext__(self):
+ if not self.buffer:
+ self.buffer = await self._prefetch()
+ if not self.buffer:
+ raise StopAsyncIteration
+ return self.buffer.popleft()
+
+then the ``Cursor`` class can be used as follows::
+
+ async for row in Cursor():
+ print(row)
+
+which would be equivalent to the following code::
+
+ i = await Cursor().__aiter__()
+ while True:
+ try:
+ row = await i.__anext__()
+ except StopAsyncIteration:
+ break
+ else:
+ print(row)
+
+
+Example 2
+'''''''''
+
+The following is a utility class that transforms a regular iterable to an
+asynchronous one. While this is not a very useful thing to do, the code
+illustrates the relationship between regular and asynchronous iterators.
+
+::
+
+ class AsyncIteratorWrapper:
+ def __init__(self, obj):
+ self._it = iter(obj)
+
+ async def __aiter__(self):
+ return self
+
+ async def __anext__(self):
+ try:
+ value = next(self._it)
+ except StopIteration:
+ raise StopAsyncIteration
+ return value
+
+ data = "abc"
+ it = AsyncIteratorWrapper("abc")
+ async for item in it:
+ print(it)
+
+
+Why StopAsyncIteration?
+'''''''''''''''''''''''
+
+Coroutines are still based on generators internally. So, before PEP 479, there
+was no fundamental difference between
+
+::
+
+ def g1():
+ yield from fut
+ return 'spam'
+
+and
+
+::
+
+ def g2():
+ yield from fut
+ raise StopIteration('spam')
+
+And since PEP 479 is accepted and enabled by default for coroutines, the
+following example will have its ``StopIteration`` wrapped into a
+``RuntimeError``
+
+::
+
+ async def a1():
+ await fut
+ raise StopIteration('spam')
+
+The only way to tell the outside code that the iteration has ended is to raise
+something other than ``StopIteration``. Therefore, a new built-in exception
+class ``StopAsyncIteration`` was added.
+
+Moreover, with semantics from PEP 479, all ``StopIteration`` exceptions raised
+in coroutines are wrapped in ``RuntimeError``.
+
+
+Debugging Features
+------------------
+
+One of the most frequent mistakes that people make when using generators as
+coroutines is forgetting to use ``yield from``::
+
+ @asyncio.coroutine
+ def useful():
+ asyncio.sleep(1) # this will do noting without 'yield from'
+
+For debugging this kind of mistakes there is a special debug mode in asyncio,
+in which ``@coroutine`` decorator wraps all functions with a special object
+with a destructor logging a warning. Whenever a wrapped generator gets garbage
+collected, a detailed logging message is generated with information about where
+exactly the decorator function was defined, stack trace of where it was
+collected, etc. Wrapper object also provides a convenient ``__repr__``
+function with detailed information about the generator.
+
+The only problem is how to enable these debug capabilities. Since debug
+facilities should be a no-op in production mode, ``@coroutine`` decorator makes
+the decision of whether to wrap or not to wrap based on an OS environment
+variable ``PYTHONASYNCIODEBUG``. This way it is possible to run asyncio
+programs with asyncio's own functions instrumented. ``EventLoop.set_debug``, a
+different debug facility, has no impact on ``@coroutine`` decorator's behavior.
+
+With this proposal, coroutines is a native, distinct from generators,
+concept. A new method ``set_async_wrapper`` is added to the ``sys`` module,
+with which frameworks can provide advanced debugging facilities.
+
+It is also important to make coroutines as fast and efficient as possible,
+therefore there are no debug features enabled by default.
+
+Example::
+
+ async def debug_me():
+ await asyncio.sleep(1)
+
+ def async_debug_wrap(generator):
+ return asyncio.AsyncDebugWrapper(generator)
+
+ sys.set_async_wrapper(async_debug_wrap)
+
+ debug_me() # <- this line will likely GC the coroutine object and
+ # trigger AsyncDebugWrapper's code.
+
+ assert isinstance(debug_me(), AsyncDebugWrapper)
+
+ sys.set_async_wrapper(None) # <- this unsets any previously set wrapper
+ assert not isinstance(debug_me(), AsyncDebugWrapper)
+
+If ``sys.set_async_wrapper()`` is called twice, the new wrapper replaces the
+previous wrapper. ``sys.set_async_wrapper(None)`` unsets the wrapper.
+
+
+Glossary
+========
+
+:Coroutine:
+ A coroutine function, or just "coroutine", is declared with ``async def``.
+ It uses ``await`` and ``return value``; see `New Coroutine Declaration
+ Syntax`_ for details.
+
+:Coroutine object:
+ Returned from a coroutine function. See `Await Expression`_ for details.
+
+:Future-like object:
+ An object with an ``__await__`` method. It is consumed by ``await`` in a
+ coroutine. A coroutine waiting for a Future-like object is suspended until
+ the Future-like object's ``__await__`` completes. ``await`` returns the
+ result of the Future-like object. See `Await Expression`_ for details.
+
+:Awaitable:
+ A *future-like* object or a *coroutine object*. See `Await Expression`_
+ for details.
+
+:Generator-based coroutine:
+ Coroutines based in generator syntax. Most common example is
+ ``@asyncio.coroutine``.
+
+:Asynchronous context manager:
+ An asynchronous context manager has ``__aenter__`` and ``__aexit__`` methods
+ and can be used with ``async with``. See
+ `Asynchronous Context Managers and "async with"`_ for details.
+
+:Asynchronous iterable:
+ An object with an ``__aiter__`` method, which must return an *asynchronous
+ iterator* object. Can be used with ``async for``. See
+ `Asynchronous Iterators and "async for"`_ for details.
+
+:Asynchronous iterator:
+ An asynchronous iterator has an ``__anext__`` method.See
+ `Asynchronous Iterators and "async for"`_ for details.
+
+
+List of functions and methods
+=============================
+
+================= ======================================= =================
+Method Can contain Can't contain
+================= ======================================= =================
+async def func await, return value yield, yield from
+async def __a*__ await, return value yield, yield from
+def __a*__ return Future-like await
+def __await__ yield, yield from, return iterable await
+generator yield, yield from, return value await
+================= ======================================= =================
+
+Where:
+
+* "async def func": coroutine;
+
+* "async def __a*__": ``__aiter__``, ``__anext__``, ``__aenter__``,
+ ``__aexit__`` defined with the ``async`` keyword;
+
+* "def __a*__": ``__aiter__``, ``__anext__``, ``__aenter__``, ``__aexit__``
+ defined without the ``async`` keyword, must return an *awaitable*;
+
+* "def __await__": ``__await__`` method to implement *Future-like* objects;
+
+* generator: a "regular" generator, function defined with ``def`` and which
+ contains a least one ``yield`` or ``yield from`` expression.
+
+*Future-like* is an object with an ``__await__`` method, see
+`Await Expression`_ section for details.
+
+
+Transition Plan
+===============
+
+To avoid backwards compatibility issues with ``async`` and ``await`` keywords,
+it was decided to modify ``tokenizer.c`` in such a way, that it:
+
+* recognizes ``async def`` name tokens combination (start of a coroutine);
+
+* keeps track of regular functions and coroutines;
+
+* replaces ``'async'`` token with ``ASYNC`` and ``'await'`` token with
+ ``AWAIT`` when in the process of yielding tokens for coroutines.
+
+This approach allows for seamless combination of new syntax features (all of
+them available only in ``async`` functions) with any existing code.
+
+An example of having "async def" and "async" attribute in one piece of code::
+
+ class Spam:
+ async = 42
+
+ async def ham():
+ print(getattr(Spam, 'async'))
+
+ # The coroutine can be executed and will print '42'
+
+
+Backwards Compatibility
+-----------------------
+
+The only backwards incompatible change is an extra argument ``is_async`` to
+``FunctionDef`` AST node. But since it is a documented fact that the structure
+of AST nodes is an implementation detail and subject to change, this should not
+be considered a serious issue.
+
+
+Grammar Updates
+---------------
+
+Grammar changes are also fairly minimal::
+
+ await_expr: AWAIT test
+ await_stmt: await_expr
+
+ decorated: decorators (classdef | funcdef | async_funcdef)
+ async_funcdef: ASYNC funcdef
+
+ async_stmt: ASYNC (funcdef | with_stmt) # will add for_stmt later
+
+ compound_stmt: (if_stmt | while_stmt | for_stmt | try_stmt | with_stmt
+ | funcdef | classdef | decorated | async_stmt)
+
+ atom: ('(' [yield_expr|await_expr|testlist_comp] ')' |
+ '[' [testlist_comp] ']' |
+ '{' [dictorsetmaker] '}' |
+ NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False’)
+
+ expr_stmt: testlist_star_expr (augassign (yield_expr|await_expr|testlist) |
+ ('=' (yield_expr|await_expr|testlist_star_expr))*)
+
+
+Transition Period Shortcomings
+------------------------------
+
+There is just one.
+
+Until ``async`` and ``await`` are not proper keywords, it is not possible (or
+at least very hard) to fix ``tokenizer.c`` to recognize them on the **same
+line** with ``def`` keyword::
+
+ # async and await will always be parsed as variables
+
+ async def outer(): # 1
+ def nested(a=(await fut)):
+ pass
+
+ async def foo(): return (await fut) # 2
+
+Since ``await`` and ``async`` in such cases are parsed as ``NAME`` tokens, a
+``SyntaxError`` will be raised.
+
+To workaround these issues, the above examples can be easily rewritten to a
+more readable form::
+
+ async def outer(): # 1
+ a_default = await fut
+ def nested(a=a_default):
+ pass
+
+ async def foo(): # 2
+ return (await fut)
+
+This limitation will go away as soon as ``async`` and ``await`` ate proper
+keywords. Or if it's decided to use a future import for this PEP.
+
+
+Deprecation Plans
+-----------------
+
+``async`` and ``await`` names will be softly deprecated in CPython 3.5 and 3.6.
+In 3.7 we will transform them to proper keywords. Making ``async`` and
+``await`` proper keywords before 3.7 might make it harder for people to port
+their code to Python 3.
+
+
+asyncio
+-------
+
+``asyncio`` module was adapted and tested to work with coroutines and new
+statements. Backwards compatibility is 100% preserved.
+
+The required changes are mainly:
+
+1. Modify ``@asyncio.coroutine`` decorator to use new ``types.async_def()``
+ function.
+
+2. Add ``__await__ = __iter__`` line to ``asyncio.Future`` class.
+
+3. Add ``ensure_task()`` as an alias for ``async()`` function. Deprecate
+ ``async()`` function.
+
+
+Design Considerations
+=====================
+
+No implicit wrapping in Futures
+-------------------------------
+
+There is a proposal to add similar mechanism to ECMAScript 7 [2]_. A key
+difference is that JavaScript "async functions" always return a Promise. While
+this approach has some advantages, it also implies that a new Promise object is
+created on each "async function" invocation.
+
+We could implement a similar functionality in Python, by wrapping all
+coroutines in a Future object, but this has the following disadvantages:
+
+1. Performance. A new Future object would be instantiated on each coroutine
+ call. Moreover, this makes implementation of ``await`` expressions slower
+ (disabling optimizations of ``yield from``).
+
+2. A new built-in ``Future`` object would need to be added.
+
+3. Coming up with a generic ``Future`` interface that is usable for any use
+ case in any framework is a very hard to solve problem.
+
+4. It is not a feature that is used frequently, when most of the code is
+ coroutines.
+
+
+Why "async" and "await" keywords
+--------------------------------
+
+async/await is not a new concept in programming languages:
+
+* C# has it since long time ago [5]_;
+
+* proposal to add async/await in ECMAScript 7 [2]_;
+ see also Traceur project [9]_;
+
+* Facebook's Hack/HHVM [6]_;
+
+* Google's Dart language [7]_;
+
+* Scala [8]_;
+
+* proposal to add async/await to C++ [10]_;
+
+* and many other less popular languages.
+
+This is a huge benefit, as some users already have experience with async/await,
+and because it makes working with many languages in one project easier (Python
+with ECMAScript 7 for instance).
+
+
+Why "__aiter__" is a coroutine
+------------------------------
+
+In principle, ``__aiter__`` could be a regular function. There are several
+good reasons to make it a coroutine:
+
+* as most of the ``__anext__``, ``__aenter__``, and ``__aexit__`` methods are
+ coroutines, users would often make a mistake defining it as ``async``
+ anyways;
+
+* there might be a need to run some asynchronous operations in ``__aiter__``,
+ for instance to prepare DB queries or do some file operation.
+
+
+Importance of "async" keyword
+-----------------------------
+
+While it is possible to just implement ``await`` expression and treat all
+functions with at least one ``await`` as coroutines, this approach makes
+APIs design, code refactoring and its long time support harder.
+
+Let's pretend that Python only has ``await`` keyword::
+
+ def useful():
+ ...
+ await log(...)
+ ...
+
+ def important():
+ await useful()
+
+If ``useful()`` function is refactored and someone removes all ``await``
+expressions from it, it would become a regular python function, and all code
+that depends on it, including ``important()`` would be broken. To mitigate this
+issue a decorator similar to ``@asyncio.coroutine`` has to be introduced.
+
+
+Why "async def"
+---------------
+
+For some people bare ``async name(): pass`` syntax might look more appealing
+than ``async def name(): pass``. It is certainly easier to type. But on the
+other hand, it breaks the symmetry between ``async def``, ``async with`` and
+``async for``, where ``async`` is a modifier, stating that the statement is
+asynchronous. It is also more consistent with the existing grammar.
+
+
+Why not a __future__ import
+---------------------------
+
+``__future__`` imports are inconvenient and easy to forget to add. Also, they
+are enabled for the whole source file. Consider that there is a big project
+with a popular module named "async.py". With future imports it is required to
+either import it using ``__import__()`` or ``importlib.import_module()`` calls,
+or to rename the module. The proposed approach makes it possible to continue
+using old code and modules without a hassle, while coming up with a migration
+plan for future python versions.
+
+
+Why magic methods start with "a"
+--------------------------------
+
+New asynchronous magic methods ``__aiter__``, ``__anext__``, ``__aenter__``,
+and ``__aexit__`` all start with the same prefix "a". An alternative proposal
+is to use "async" prefix, so that ``__aiter__`` becomes ``__async_iter__``.
+However, to align new magic methods with the existing ones, such as
+``__radd__`` and ``__iadd__`` it was decided to use a shorter version.
+
+
+Why not reuse existing magic names
+----------------------------------
+
+An alternative idea about new asynchronous iterators and context managers was
+to reuse existing magic methods, by adding an ``async`` keyword to their
+declarations::
+
+ class CM:
+ async def __enter__(self): # instead of __aenter__
+ ...
+
+This approach has the following downsides:
+
+* it would not be possible to create an object that works in both ``with`` and
+ ``async with`` statements;
+
+* it would look confusing and would require some implicit magic behind the
+ scenes in the interpreter;
+
+* one of the main points of this proposal is to make coroutines as simple
+ and foolproof as possible.
+
+
+Comprehensions
+--------------
+
+For the sake of restricting the broadness of this PEP there is no new syntax
+for asynchronous comprehensions. This should be considered in a separate PEP,
+if there is a strong demand for this feature.
+
+
+Performance
+===========
+
+Overall Impact
+--------------
+
+This proposal introduces no observable performance impact. Here is an output
+of python's official set of benchmarks [4]_:
+
+::
+
+ python perf.py -r -b default ../cpython/python.exe ../cpython-aw/python.exe
+
+ [skipped]
+
+ Report on Darwin ysmac 14.3.0 Darwin Kernel Version 14.3.0:
+ Mon Mar 23 11:59:05 PDT 2015; root:xnu-2782.20.48~5/RELEASE_X86_64
+ x86_64 i386
+
+ Total CPU cores: 8
+
+ ### etree_iterparse ###
+ Min: 0.365359 -> 0.349168: 1.05x faster
+ Avg: 0.396924 -> 0.379735: 1.05x faster
+ Significant (t=9.71)
+ Stddev: 0.01225 -> 0.01277: 1.0423x larger
+
+ The following not significant results are hidden, use -v to show them:
+ django_v2, 2to3, etree_generate, etree_parse, etree_process, fastpickle,
+ fastunpickle, json_dump_v2, json_load, nbody, regex_v8, tornado_http.
+
+
+Tokenizer modifications
+-----------------------
+
+There is no observable slowdown of parsing python files with the modified
+tokenizer: parsing of one 12Mb file (``Lib/test/test_binop.py`` repeated 1000
+times) takes the same amount of time.
+
+
+async/await
+-----------
+
+The following micro-benchmark was used to determine performance difference
+between "async" functions and generators::
+
+ import sys
+ import time
+
+ def binary(n):
+ if n <= 0:
+ return 1
+ l = yield from binary(n - 1)
+ r = yield from binary(n - 1)
+ return l + 1 + r
+
+ async def abinary(n):
+ if n <= 0:
+ return 1
+ l = await abinary(n - 1)
+ r = await abinary(n - 1)
+ return l + 1 + r
+
+ def timeit(gen, depth, repeat):
+ t0 = time.time()
+ for _ in range(repeat):
+ list(gen(depth))
+ t1 = time.time()
+ print('{}({}) * {}: total {:.3f}s'.format(
+ gen.__name__, depth, repeat, t1-t0))
+
+The result is that there is no observable performance difference. Minimum
+timing of 3 runs
+
+::
+
+ abinary(19) * 30: total 12.985s
+ binary(19) * 30: total 12.953s
+
+Note that depth of 19 means 1,048,575 calls.
+
+
+Reference Implementation
+========================
+
+The reference implementation can be found here: [3]_.
+
+List of high-level changes and new protocols
+--------------------------------------------
+
+1. New syntax for defining coroutines: ``async def`` and new ``await``
+ keyword.
+
+2. New ``__await__`` method for Future-like objects.
+
+3. New syntax for asynchronous context managers: ``async with``. And
+ associated protocol with ``__aenter__`` and ``__aexit__`` methods.
+
+4. New syntax for asynchronous iteration: ``async for``. And associated
+ protocol with ``__aiter__``, ``__aexit__`` and new built-in exception
+ ``StopAsyncIteration``.
+
+5. New AST nodes: ``AsyncFor``, ``AsyncWith``, ``Await``; ``FunctionDef`` AST
+ node got a new argument ``is_async``.
+
+6. New functions: ``sys.set_async_wrapper(callback)`` and
+ ``types.async_def(gen)``.
+
+7. New ``CO_ASYNC`` bit flag for code objects.
+
+While the list of changes and new things is not short, it is important to
+understand, that most users will not use these features directly. It is
+intended to be used in frameworks and libraries to provide users with
+convenient to use and unambiguous APIs with ``async def``, ``await``, ``async
+for`` and ``async with`` syntax.
+
+
+Working example
+---------------
+
+All concepts proposed in this PEP are implemented [3]_ and can be tested.
+
+::
+
+ import asyncio
+
+
+ async def echo_server():
+ print('Serving on localhost:8000')
+ await asyncio.start_server(handle_connection, 'localhost', 8000)
+
+
+ async def handle_connection(reader, writer):
+ print('New connection...')
+
+ while True:
+ data = await reader.read(8192)
+
+ if not data:
+ break
+
+ print('Sending {:.10}... back'.format(repr(data)))
+ writer.write(data)
+
+
+ loop = asyncio.get_event_loop()
+ loop.run_until_complete(echo_server())
+ try:
+ loop.run_forever()
+ finally:
+ loop.close()
+
+
+References
+==========
+
+.. [1] https://docs.python.org/3/library/asyncio-task.html#asyncio.coroutine
+
+.. [2] http://wiki.ecmascript.org/doku.php?id=strawman:async_functions
+
+.. [3] https://github.com/1st1/cpython/tree/await
+
+.. [4] https://hg.python.org/benchmarks
+
+.. [5] https://msdn.microsoft.com/en-us/library/hh191443.aspx
+
+.. [6] http://docs.hhvm.com/manual/en/hack.async.php
+
+.. [7] https://www.dartlang.org/articles/await-async/
+
+.. [8] http://docs.scala-lang.org/sips/pending/async.html
+
+.. [9] https://github.com/google/traceur-compiler/wiki/LanguageFeatures#async-functions-experimental
+
+.. [10] http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3722.pdf (PDF)
+
+
+Acknowledgments
+===============
+
+I thank Guido van Rossum, Victor Stinner, Elvis Pranskevichus, Andrew Svetlov,
+and Łukasz Langa for their initial feedback.
+
+
+Copyright
+=========
+
+This document has been placed in the public domain.
+
+..
+ Local Variables:
+ mode: indented-text
+ indent-tabs-mode: nil
+ sentence-end-double-space: t
+ fill-column: 70
+ coding: utf-8
+ End:
--
Repository URL: https://hg.python.org/peps
More information about the Python-checkins
mailing list