[Python-ideas] async/await in Python
Andrew Barnert
abarnert at yahoo.com
Sat Apr 18 00:15:28 CEST 2015
Could you show the best yield from parallel for the example at the end, so it's easier to see the readability benefits?
Or, even better, provide a simple example that uses async for or async with and the yield from parallel, so it's easier to see the (I'm assuming even bigger, by a long shot) readability benefits?
Sent from my iPhone
> On Apr 17, 2015, at 11:58, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
>
> Hello python-ideas,
>
> Here's my proposal to add async/await in Python.
>
> I believe that PEPs 380 and 3156 were a major breakthrough for Python 3,
> and I hope that this PEP can be another big step. Who knows, maybe it
> will be one of the reasons to drive people's interest towards Python 3.
>
>
> PEP: XXX
> Title: Coroutines with async and await syntax
> Version: $Revision$
> Last-Modified: $Date$
> Author: Yury Selivanov <yselivanov at sprymix.com>
> Discussions-To: Python-Dev <python-dev at python.org>
> Python-Version: 3.5
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 09-Apr-2015
> Post-History:
> Resolution:
>
>
> Abstract
> ========
>
> This PEP introduces new syntax for coroutines, asynchronous ``with``
> statements and ``for`` loops. The main motivation behind this proposal is to
> streamline writing and maintaining asynchronous code, as well as to simplify
> previously hard to implement code patterns.
>
>
> Rationale and Goals
> ===================
>
> Current Python supports implementing coroutines via generators (PEP 342),
> further enhanced by the ``yield from`` syntax introduced in PEP 380.
> This approach has a number of shortcomings:
>
> * it is easy to confuse coroutines with regular generators, since they share
> the same syntax; async libraries often attempt to alleviate this by using
> decorators (e.g. ``@asyncio.coroutine`` [1]_);
>
> * it is not possible to natively define a coroutine which has no ``yield``
> or ``yield from`` statements, again requiring the use of decorators to
> fix potential refactoring issues;
>
> * support for asynchronous calls is limited to expressions where ``yield`` is
> allowed syntactically, limiting the usefulness of syntactic features, such
> as ``with`` and ``for`` statements.
>
> This proposal makes coroutines a native Python language feature, and clearly
> separates them from generators. This removes generator/coroutine ambiguity,
> and makes it possible to reliably define coroutines without reliance on a
> specific library. This also enables linters and IDEs to improve static code
> analysis and refactoring.
>
> Native coroutines and the associated new syntax features make it possible
> to define context manager and iteration protocols in asynchronous terms.
> As shown later in this proposal, the new ``async with`` statement lets Python
> programs perform asynchronous calls when entering and exiting a runtime
> context, and the new ``async for`` statement makes it possible to perform
> asynchronous calls in iterators.
>
>
> Specification
> =============
>
> This proposal introduces new syntax and semantics to enhance coroutine support
> in Python, it does not change the internal implementation of coroutines, which
> are still based on generators.
>
> It is strongly suggested that the reader understands how coroutines are
> implemented in Python (PEP 342 and PEP 380). It is also recommended to read
> PEP 3156 (asyncio framework).
>
> From this point in this document we use the word *coroutine* to refer to
> functions declared using the new syntax. *generator-based coroutine* is used
> where necessary to refer to coroutines that are based on generator syntax.
>
>
> New Coroutine Declaration Syntax
> --------------------------------
>
> The following new syntax is used to declare a coroutine::
>
> async def read_data(db):
> pass
>
> Key properties of coroutines:
>
> * Coroutines are always generators, even if they do not contain ``await``
> expressions.
>
> * It is a ``SyntaxError`` to have ``yield`` or ``yield from`` expressions in
> an ``async`` function.
>
> * Internally, a new code object flag - ``CO_ASYNC`` - is introduced to enable
> runtime detection of coroutines (and migrating existing code).
> All coroutines have both ``CO_ASYNC`` and ``CO_GENERATOR`` flags set.
>
> * Regular generators, when called, return a *generator object*; similarly,
> coroutines return a *coroutine object*.
>
> * ``StopIteration`` exceptions are not propagated out of coroutines, and are
> replaced with a ``RuntimeError``. For regular generators such behavior
> requires a future import (see PEP 479).
>
>
> types.async_def()
> -----------------
>
> A new function ``async_def(gen)`` is added to the ``types`` module. It
> applies ``CO_ASYNC`` flag to the passed generator's code object, so that it
> returns a *coroutine object* when called.
>
> This feature enables an easy upgrade path for existing libraries.
>
>
> Await Expression
> ----------------
>
> The following new ``await`` expression is used to obtain a result of coroutine
> execution::
>
> async def read_data(db):
> data = await db.fetch('SELECT ...')
> ...
>
> ``await``, similarly to ``yield from``, suspends execution of ``read_data``
> coroutine until ``db.fetch`` *awaitable* completes and returns the result
> data.
>
> It uses the ``yield from`` implementation with an extra step of validating its
> argument. ``await`` only accepts an *awaitable*, which can be one of:
>
> * A *coroutine object* returned from a coroutine or a generator decorated with
> ``types.async_def()``.
>
> * An object with an ``__await__`` method returning an iterator.
>
> Any ``yield from`` chain of calls ends with a ``yield``. This is a
> fundamental mechanism of how *Futures* are implemented. Since, internally,
> coroutines are a special kind of generators, every ``await`` is suspended by
> a ``yield`` somewhere down the chain of ``await`` calls (please refer to PEP
> 3156 for a detailed explanation.)
>
> To enable this behavior for coroutines, a new magic method called
> ``__await__`` is added. In asyncio, for instance, to enable Future objects
> in ``await`` statements, the only change is to add ``__await__ = __iter__``
> line to ``asyncio.Future`` class.
>
> Objects with ``__await__`` method are called *Future-like* objects in the
> rest of this PEP.
>
> Also, please note that ``__aiter__`` method (see its definition below) cannot
> be used for this purpose. It is a different protocol, and would be like
> using ``__iter__`` instead of ``__call__`` for regular callables.
>
> It is a ``SyntaxError`` to use ``await`` outside of a coroutine.
>
>
> Asynchronous Context Managers and "async with"
> ----------------------------------------------
>
> An *asynchronous context manager* is a context manager that is able to suspend
> execution in its *enter* and *exit* methods.
>
> To make this possible, a new protocol for asynchronous context managers is
> proposed. Two new magic methods are added: ``__aenter__`` and ``__aexit__``.
> Both must return an *awaitable*.
>
> An example of an asynchronous context manager::
>
> class AsyncContextManager:
> async def __aenter__(self):
> await log('entering context')
>
> async def __aexit__(self, exc_type, exc, tb):
> await log('exiting context')
>
>
> New Syntax
> ''''''''''
>
> A new statement for asynchronous context managers is proposed::
>
> async with EXPR as VAR:
> BLOCK
>
>
> which is semantically equivalent to::
>
> mgr = (EXPR)
> aexit = type(mgr).__aexit__
> aenter = type(mgr).__aenter__(mgr)
> exc = True
>
> try:
> try:
> VAR = await aenter
> BLOCK
> except:
> exc = False
> exit_res = await aexit(mgr, *sys.exc_info())
> if not exit_res:
> raise
>
> finally:
> if exc:
> await aexit(mgr, None, None, None)
>
>
> As with regular ``with`` statements, it is possible to specify multiple context
> managers in a single ``async with`` statement.
>
> It is an error to pass a regular context manager without ``__aenter__`` and
> ``__aexit__`` methods to ``async with``. It is a ``SyntaxError`` to use
> ``async with`` outside of a coroutine.
>
>
> Example
> '''''''
>
> With asynchronous context managers it is easy to implement proper database
> transaction managers for coroutines::
>
> async def commit(session, data):
> ...
>
> async with session.transaction():
> ...
> await session.update(data)
> ...
>
> Code that needs locking also looks lighter::
>
> async with lock:
> ...
>
> instead of::
>
> with (yield from lock):
> ...
>
>
> Asynchronous Iterators and "async for"
> --------------------------------------
>
> An *asynchronous iterable* is able to call asynchronous code in its *iter*
> implementation, and *asynchronous iterator* can call asynchronous code in its
> *next* method. To support asynchronous iteration:
>
> 1. An object must implement an ``__aiter__`` method returning an *awaitable*
> resulting in an *asynchronous iterator object*.
>
> 2. An *asynchronous iterator object* must implement an ``__anext__`` method
> returning an *awaitable*.
>
> 3. To stop iteration```__anext__`` must raise a ``StopAsyncIteration``
> exception.
>
> An example of asynchronous iterable::
>
> class AsyncIterable:
> async def __aiter__(self):
> return self
>
> async def __anext__(self):
> data = await self.fetch_data()
> if data:
> return data
> else:
> raise StopAsyncIteration
>
> async def fetch_data(self):
> ...
>
>
> New Syntax
> ''''''''''
>
> A new statement for iterating through asynchronous iterators is proposed::
>
> async for TARGET in ITER:
> BLOCK
> else:
> BLOCK2
>
> which is semantically equivalent to::
>
> iter = (ITER)
> iter = await type(iter).__aiter__(iter)
> running = True
> while running:
> try:
> TARGET = await type(iter).__anext__(iter)
> except StopAsyncIteration:
> running = False
> else:
> BLOCK
> else:
> BLOCK2
>
>
> It is an error to pass a regular iterable without ``__aiter__`` method to
> ``async for``. It is a ``SyntaxError`` to use ``async for`` outside of a
> coroutine.
>
> As for with regular ``for`` statement, ``async for`` has an optional ``else``
> clause.
>
>
> Example 1
> '''''''''
>
> With asynchronous iteration protocol it is possible to asynchronously buffer
> data during iteration::
>
> async for data in cursor:
> ...
>
> Where ``cursor`` is an asynchronous iterator that prefetches ``N`` rows
> of data from a database after every ``N`` iterations.
>
> The following code illustrates new asynchronous iteration protocol::
>
> class Cursor:
> def __init__(self):
> self.buffer = collections.deque()
>
> def _prefetch(self):
> ...
>
> async def __aiter__(self):
> return self
>
> async def __anext__(self):
> if not self.buffer:
> self.buffer = await self._prefetch()
> if not self.buffer:
> raise StopAsyncIteration
> return self.buffer.popleft()
>
> then the ``Cursor`` class can be used as follows::
>
> async for row in Cursor():
> print(row)
>
> which would be equivalent to the following code::
>
> i = await Cursor().__aiter__()
> while True:
> try:
> row = await i.__anext__()
> except StopAsyncIteration:
> break
> else:
> print(row)
>
>
> Example 2
> '''''''''
>
> The following is a utility class that transforms a regular iterable to an
> asynchronous one. While this is not a very useful thing to do, the code
> illustrates the relationship between regular and asynchronous iterators.
>
> ::
>
> class AsyncIteratorWrapper:
> def __init__(self, obj):
> self._it = iter(obj)
>
> async def __aiter__(self):
> return self
>
> async def __anext__(self):
> try:
> value = next(self._it)
> except StopIteration:
> raise StopAsyncIteration
> return value
>
> data = "abc"
> it = AsyncIteratorWrapper("abc")
> async for item in it:
> print(it)
>
>
> Why StopAsyncIteration?
> '''''''''''''''''''''''
>
> Coroutines are still based on generators internally. So, before PEP 479, there
> was no fundamental difference between
>
> ::
>
> def g1():
> yield from fut
> return 'spam'
>
> and
>
> ::
>
> def g2():
> yield from fut
> raise StopIteration('spam')
>
> And since PEP 479 is accepted and enabled by default for coroutines, the
> following example will have its ``StopIteration`` wrapped into a
> ``RuntimeError``
>
> ::
>
> async def a1():
> await fut
> raise StopIteration('spam')
>
> The only way to tell the outside code that the iteration has ended is to raise
> something other than ``StopIteration``. Therefore, a new built-in exception
> class ``StopAsyncIteration`` was added.
>
> Moreover, with semantics from PEP 479, all ``StopIteration`` exceptions raised
> in coroutines are wrapped in ``RuntimeError``.
>
>
> Debugging Features
> ------------------
>
> One of the most frequent mistakes that people make when using generators as
> coroutines is forgetting to use ``yield from``::
>
> @asyncio.coroutine
> def useful():
> asyncio.sleep(1) # this will do noting without 'yield from'
>
> For debugging this kind of mistakes there is a special debug mode in asyncio,
> in which ``@coroutine`` decorator wraps all functions with a special object
> with a destructor logging a warning. Whenever a wrapped generator gets garbage
> collected, a detailed logging message is generated with information about where
> exactly the decorator function was defined, stack trace of where it was
> collected, etc. Wrapper object also provides a convenient ``__repr__``
> function with detailed information about the generator.
>
> The only problem is how to enable these debug capabilities. Since debug
> facilities should be a no-op in production mode, ``@coroutine`` decorator makes
> the decision of whether to wrap or not to wrap based on an OS environment
> variable ``PYTHONASYNCIODEBUG``. This way it is possible to run asyncio
> programs with asyncio's own functions instrumented. ``EventLoop.set_debug``, a
> different debug facility, has no impact on ``@coroutine`` decorator's behavior.
>
> With this proposal, coroutines is a native, distinct from generators,
> concept. A new method ``set_async_wrapper`` is added to the ``sys`` module,
> with which frameworks can provide advanced debugging facilities.
>
> It is also important to make coroutines as fast and efficient as possible,
> therefore there are no debug features enabled by default.
>
> Example::
>
> async def debug_me():
> await asyncio.sleep(1)
>
> def async_debug_wrap(generator):
> return asyncio.AsyncDebugWrapper(generator)
>
> sys.set_async_wrapper(async_debug_wrap)
>
> debug_me() # <- this line will likely GC the coroutine object and
> # trigger AsyncDebugWrapper's code.
>
> assert isinstance(debug_me(), AsyncDebugWrapper)
>
> sys.set_async_wrapper(None) # <- this unsets any previously set wrapper
> assert not isinstance(debug_me(), AsyncDebugWrapper)
>
> If ``sys.set_async_wrapper()`` is called twice, the new wrapper replaces the
> previous wrapper. ``sys.set_async_wrapper(None)`` unsets the wrapper.
>
>
> Glossary
> ========
>
> :Coroutine:
> A coroutine function, or just "coroutine", is declared with ``async def``.
> It uses ``await`` and ``return value``; see `New Coroutine Declaration
> Syntax`_ for details.
>
> :Coroutine object:
> Returned from a coroutine function. See `Await Expression`_ for details.
>
> :Future-like object:
> An object with an ``__await__`` method. It is consumed by ``await`` in a
> coroutine. A coroutine waiting for a Future-like object is suspended until
> the Future-like object's ``__await__`` completes. ``await`` returns the
> result of the Future-like object. See `Await Expression`_ for details.
>
> :Awaitable:
> A *future-like* object or a *coroutine object*. See `Await Expression`_
> for details.
>
> :Generator-based coroutine:
> Coroutines based in generator syntax. Most common example is
> ``@asyncio.coroutine``.
>
> :Asynchronous context manager:
> An asynchronous context manager has ``__aenter__`` and ``__aexit__`` methods
> and can be used with ``async with``. See
> `Asynchronous Context Managers and "async with"`_ for details.
>
> :Asynchronous iterable:
> An object with an ``__aiter__`` method, which must return an *asynchronous
> iterator* object. Can be used with ``async for``. See
> `Asynchronous Iterators and "async for"`_ for details.
>
> :Asynchronous iterator:
> An asynchronous iterator has an ``__anext__`` method.See
> `Asynchronous Iterators and "async for"`_ for details.
>
>
> List of functions and methods
> =============================
>
> ================= ======================================= =================
> Method Can contain Can't contain
> ================= ======================================= =================
> async def func await, return value yield, yield from
> async def __a*__ await, return value yield, yield from
> def __a*__ return Future-like await
> def __await__ yield, yield from, return iterable await
> generator yield, yield from, return value await
> ================= ======================================= =================
>
> Where:
>
> * ""async def func": coroutine;
>
> * "async def __a*__": ``__aiter__``, ``__anext__``, ``__aenter__``,
> ``__aexit__`` defined with the ``async`` keyword;
>
> * "def __a*__": ``__aiter__``, ``__anext__``, ``__aenter__``, ``__aexit__``
> defined without the ``async`` keyword, must return an *awaitable*;
>
> * "def __await__": ``__await__`` method to implement *Future-like* objects;
>
> * generator: a "regular" generator, function defined with ``def`` and which
> contains a least one ``yield`` or ``yield from`` expression.
>
> *Future-like* is an object with an ``__await__`` method, see
> `Await Expression`_ section for details.
>
>
> Transition Plan
> ===============
>
> To avoid backwards compatibility issues with ``async`` and ``await`` keywords,
> it was decided to modify ``tokenizer.c`` in such a way, that it:
>
> * recognizes ``async def`` name tokens combination (start of a coroutine);
>
> * keeps track of regular functions and coroutines;
>
> * replaces ``'async'`` token with ``ASYNC`` and ``'await'`` token with
> ``AWAIT`` when in the process of yielding tokens for coroutines.
>
> This approach allows for seamless combination of new syntax features (all of
> them available only in ``async`` functions) with any existing code.
>
> An example of having "async def" and "async" attribute in one piece of code::
>
> class Spam:
> async = 42
>
> async def ham():
> print(getattr(Spam, 'async'))
>
> # The coroutine can be executed and will print '42'
>
>
> Backwards Compatibility
> -----------------------
>
> The only backwards incompatible change is an extra argument ``is_async`` to
> ``FunctionDef`` AST node. But since it is a documented fact that the structure
> of AST nodes is an implementation detail and subject to change, this should not
> be considered a serious issue.
>
>
> Grammar Updates
> ---------------
>
> Grammar changes are also fairly minimal::
>
> await_expr: AWAIT test
> await_stmt: await_expr
>
> decorated: decorators (classdef | funcdef | async_funcdef)
> async_funcdef: ASYNC funcdef
>
> async_stmt: ASYNC (funcdef | with_stmt) # will add for_stmt later
>
> compound_stmt: (if_stmt | while_stmt | for_stmt | try_stmt | with_stmt
> | funcdef | classdef | decorated | async_stmt)
>
> atom: ('(' [yield_expr|await_expr|testlist_comp] ')' |
> '[' [testlist_comp] ']' |
> '{' [dictorsetmaker] '}' |
> NAME | NUMBER | STRING+ | '...' | 'None' | 'True' | 'False’)
>
> expr_stmt: testlist_star_expr (augassign (yield_expr|await_expr|testlist) |
> ('=' (yield_expr|await_expr|testlist_star_expr))*)
>
>
> Transition Period Shortcomings
> ------------------------------
>
> There is just one.
>
> Until ``async`` and ``await`` are not proper keywords, it is not possible (or
> at least very hard) to fix ``tokenizer.c`` to recognize them on the **same
> line** with ``def`` keyword::
>
> # async and await will always be parsed as variables
>
> async def outer(): # 1
> def nested(a=(await fut)):
> pass
>
> async def foo(): return (await fut) # 2
>
> Since ``await`` and ``async`` in such cases are parsed as ``NAME`` tokens, a
> ``SyntaxError`` will be raised.
>
> To workaround these issues, the above examples can be easily rewritten to a
> more readable form::
>
> async def outer(): # 1
> a_default = await fut
> def nested(a=a_default):
> pass
>
> async def foo(): # 2
> return (await fut)
>
> This limitation will go away as soon as ``async`` and ``await`` ate proper
> keywords. Or if it's decided to use a future import for this PEP.
>
>
> Deprecation Plans
> -----------------
>
> ``async`` and ``await`` names will be softly deprecated in CPython 3.5 and 3.6.
> In 3.7 we will transform them to proper keywords. Making ``async`` and
> ``await`` proper keywords before 3.7 might make it harder for people to port
> their code to Python 3.
>
>
> asyncio
> -------
>
> ``asyncio`` module was adapted and tested to work with coroutines and new
> statements. Backwards compatibility is 100% preserved.
>
> The required changes are mainly:
>
> 1. Modify ``@asyncio.coroutine`` decorator to use new ``types.async_def()``
> function.
>
> 2. Add ``__await__ = __iter__`` line to ``asyncio.Future`` class.
>
> 3. Add ``ensure_task()`` as an alias for ``async()`` function. Deprecate
> ``async()`` function.
>
>
> Design Considerations
> =====================
>
> No implicit wrapping in Futures
> -------------------------------
>
> There is a proposal to add similar mechanism to ECMAScript 7 [2]_. A key
> difference is that JavaScript "async functions" always return a Promise. While
> this approach has some advantages, it also implies that a new Promise object is
> created on each "async function" invocation.
>
> We could implement a similar functionality in Python, by wrapping all
> coroutines in a Future object, but this has the following disadvantages:
>
> 1. Performance. A new Future object would be instantiated on each coroutine
> call. Moreover, this makes implementation of ``await`` expressions slower
> (disabling optimizations of ``yield from``).
>
> 2. A new built-in ``Future`` object would need to be added.
>
> 3. Coming up with a generic ``Future`` interface that is usable for any use
> case in any framework is a very hard to solve problem.
>
> 4. It is not a feature that is used frequently, when most of the code is
> coroutines.
>
>
> Why "async" and "await" keywords
> --------------------------------
>
> async/await is not a new concept in programming languages:
>
> * C# has it since long time ago [5]_;
>
> * proposal to add async/await in ECMAScript 7 [2]_;
> see also Traceur project [9]_;
>
> * Facebook's Hack/HHVM [6]_;
>
> * Google's Dart language [7]_;
>
> * Scala [8]_;
>
> * proposal to add async/await to C++ [10]_;
>
> * and many other less popular languages.
>
> This is a huge benefit, as some users already have experience with async/await,
> and because it makes working with many languages in one project easier (Python
> with ECMAScript 7 for instance).
>
>
> Why "__aiter__" is a coroutine
> ------------------------------
>
> In principle, ``__aiter__`` could be a regular function. There are several
> good reasons to make it a coroutine:
>
> * as most of the ``__anext__``, ``__aenter__``, and ``__aexit__`` methods are
> coroutines, users would often make a mistake defining it as ``async``
> anyways;
>
> * there might be a need to run some asynchronous operations in ``__aiter__``,
> for instance to prepare DB queries or do some file operation.
>
>
> Importance of "async" keyword
> -----------------------------
>
> While it is possible to just implement ``await`` expression and treat all
> functions with at least one ``await`` as coroutines, this approach makes
> APIs design, code refactoring and its long time support harder.
>
> Let's pretend that Python only has ``await`` keyword::
>
> def useful():
> ...
> await log(...)
> ...
>
> def important():
> await useful()
>
> If ``useful()`` function is refactored and someone removes all ``await``
> expressions from it, it would become a regular python function, and all code
> that depends on it, including ``important()`` would be broken. To mitigate this
> issue a decorator similar to ``@asyncio.coroutine`` has to be introduced.
>
>
> Why "async def"
> ---------------
>
> For some people bare ``async name(): pass`` syntax might look more appealing
> than ``async def name(): pass``. It is certainly easier to type. But on the
> other hand, it breaks the symmetry between ``async def``, ``async with`` and
> ``async for``, where ``async`` is a modifier, stating that the statement is
> asynchronous. It is also more consistent with the existing grammar.
>
>
> Why not a __future__ import
> ---------------------------
>
> ``__future__`` imports are inconvenient and easy to forget to add. Also, they
> are enabled for the whole source file. Consider that there is a big project
> with a popular module named "async.py". With future imports it is required to
> either import it using ``__import__()`` or ``importlib.import_module()`` calls,
> or to rename the module. The proposed approach makes it possible to continue
> using old code and modules without a hassle, while coming up with a migration
> plan for future python versions.
>
>
> Why magic methods start with "a"
> --------------------------------
>
> New asynchronous magic methods ``__aiter__``, ``__anext__``, ``__aenter__``,
> and ``__aexit__`` all start with the same prefix "a". An alternative proposal
> is to use "async" prefix, so that ``__aiter__`` becomes ``__async_iter__``.
> However, to align new magic methods with the existing ones, such as
> ``__radd__`` and ``__iadd__`` it was decided to use a shorter version.
>
>
> Why not reuse existing magic names
> ----------------------------------
>
> An alternative idea about new asynchronous iterators and context managers was
> to reuse existing magic methods, by adding an ``async`` keyword to their
> declarations::
>
> class CM:
> async def __enter__(self): # instead of __aenter__
> ...
>
> This approach has the following downsides:
>
> * it would not be possible to create an object that works in both ``with`` and
> ``async with`` statements;
>
> * it would look confusing and would require some implicit magic behind the
> scenes in the interpreter;
>
> * one of the main points of this proposal is to make coroutines as simple
> and foolproof as possible.
>
>
> Comprehensions
> --------------
>
> For the sake of restricting the broadness of this PEP there is no new syntax
> for asynchronous comprehensions. This should be considered in a separate PEP,
> if there is a strong demand for this feature.
>
>
> Performance
> ===========
>
> Overall Impact
> --------------
>
> This proposal introduces no observable performance impact. Here is an output
> of python's official set of benchmarks [4]_:
>
> ::
>
> python perf.py -r -b default ../cpython/python.exe ../cpython-aw/python.exe
>
> [skipped]
>
> Report on Darwin ysmac 14.3.0 Darwin Kernel Version 14.3.0:
> Mon Mar 23 11:59:05 PDT 2015; root:xnu-2782.20.48~5/RELEASE_X86_64
> x86_64 i386
>
> Total CPU cores: 8
>
> ### etree_iterparse ###
> Min: 0.365359 -> 0.349168: 1.05x faster
> Avg: 0.396924 -> 0.379735: 1.05x faster
> Significant (t=9.71)
> Stddev: 0.01225 -> 0.01277: 1.0423x larger
>
> The following not significant results are hidden, use -v to show them:
> django_v2, 2to3, etree_generate, etree_parse, etree_process, fastpickle,
> fastunpickle, json_dump_v2, json_load, nbody, regex_v8, tornado_http.
>
>
> Tokenizer modifications
> -----------------------
>
> There is no observable slowdown of parsing python files with the modified
> tokenizer: parsing of one 12Mb file (``Lib/test/test_binop.py`` repeated 1000
> times) takes the same amount of time.
>
>
> async/await
> -----------
>
> The following micro-benchmark was used to determine performance difference
> between "async" functions and generators::
>
> import sys
> import time
>
> def binary(n):
> if n <= 0:
> return 1
> l = yield from binary(n - 1)
> r = yield from binary(n - 1)
> return l + 1 + r
>
> async def abinary(n):
> if n <= 0:
> return 1
> l = await abinary(n - 1)
> r = await abinary(n - 1)
> return l + 1 + r
>
> def timeit(gen, depth, repeat):
> t0 = time.time()
> for _ in range(repeat):
> list(gen(depth))
> t1 = time.time()
> print('{}({}) * {}: total {:.3f}s'.format(
> gen.__name__, depth, repeat, t1-t0))
>
> The result is that there is no observable performance difference. Minimum
> timing of 3 runs
>
> ::
>
> abinary(19) * 30: total 12.985s
> binary(19) * 30: total 12.953s
>
> Note that depth of 19 means 1,048,575 calls.
>
>
> Reference Implementation
> ========================
>
> The reference implementation can be found here: [3]_.
>
> List of high-level changes and new protocols
> --------------------------------------------
>
> 1. New syntax for defining coroutines: ``async def`` and new ``await``
> keyword.
>
> 2. New ``__await__`` method for Future-like objects.
>
> 3. New syntax for asynchronous context managers: ``async with``. And
> associated protocol with ``__aenter__`` and ``__aexit__`` methods.
>
> 4. New syntax for asynchronous iteration: ``async for``. And associated
> protocol with ``__aiter__``, ``__aexit__`` and new built-in exception
> ``StopAsyncIteration``.
>
> 5. New AST nodes: ``AsyncFor``, ``AsyncWith``, ``Await``; ``FunctionDef`` AST
> node got a new argument ``is_async``.
>
> 6. New functions: ``sys.set_async_wrapper(callback)`` and
> ``types.async_def(gen)``.
>
> 7. New ``CO_ASYNC`` bit flag for code objects.
>
> While the list of changes and new things is not short, it is important to
> understand, that most users will not use these features directly. It is
> intended to be used in frameworks and libraries to provide users with
> convenient to use and unambiguous APIs with ``async def``, ``await``, ``async
> for`` and ``async with`` syntax.
>
>
> Working example
> ---------------
>
> All concepts proposed in this PEP are implemented [3]_ and can be tested.
>
> ::
>
> import asyncio
>
>
> async def echo_server():
> print('Serving on localhost:8000')
> await asyncio.start_server(handle_connection, 'localhost', 8000)
>
>
> async def handle_connection(reader, writer):
> print('New connection...')
>
> while True:
> data = await reader.read(8192)
>
> if not data:
> break
>
> print('Sending {:.10}... back'.format(repr(data)))
> writer.write(data)
>
>
> loop = asyncio.get_event_loop()
> loop.run_until_complete(echo_server())
> try:
> loop.run_forever()
> finally:
> loop.close()
>
>
> References
> ==========
>
> .. [1] https://docs.python.org/3/library/asyncio-task.html#asyncio.coroutine
>
> .. [2] http://wiki.ecmascript.org/doku.php?id=strawman:async_functions
>
> .. [3] https://github.com/1st1/cpython/tree/await
>
> .. [4] https://hg.python.org/benchmarks
>
> .. [5] https://msdn.microsoft.com/en-us/library/hh191443.aspx
>
> .. [6] http://docs.hhvm.com/manual/en/hack.async.php
>
> .. [7] https://www.dartlang.org/articles/await-async/
>
> .. [8] http://docs.scala-lang.org/sips/pending/async.html
>
> .. [9] https://github.com/google/traceur-compiler/wiki/LanguageFeatures#async-functions-experimental
>
> .. [10] http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3722.pdf (PDF)
>
>
> Acknowledgments
> ===============
>
> I thank Guido van Rossum, Victor Stinner, Elvis Pranskevichus, Andrew Svetlov,
> and Łukasz Langa for their initial feedback.
>
>
> Copyright
> =========
>
> This document has been placed in the public domain.
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
More information about the Python-ideas
mailing list