[Python-checkins] gh-90908: Document asyncio.Task.cancelling() and asyncio.Task.uncancel() (#95253)
gvanrossum
webhook-mailer at python.org
Sat Oct 1 13:42:45 EDT 2022
https://github.com/python/cpython/commit/f00645d5dbf4cfa0b8f382c8977724578dff191d
commit: f00645d5dbf4cfa0b8f382c8977724578dff191d
branch: main
author: Łukasz Langa <lukasz at langa.pl>
committer: gvanrossum <gvanrossum at gmail.com>
date: 2022-10-01T10:42:36-07:00
summary:
gh-90908: Document asyncio.Task.cancelling() and asyncio.Task.uncancel() (#95253)
Co-authored-by: Thomas Grainger <tagrain at gmail.com>
files:
M Doc/library/asyncio-task.rst
M Lib/asyncio/tasks.py
M Lib/test/test_asyncio/test_tasks.py
diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst
index 221197ea40ff..ade969220ea7 100644
--- a/Doc/library/asyncio-task.rst
+++ b/Doc/library/asyncio-task.rst
@@ -294,11 +294,13 @@ perform clean-up logic. In case :exc:`asyncio.CancelledError`
is explicitly caught, it should generally be propagated when
clean-up is complete. Most code can safely ignore :exc:`asyncio.CancelledError`.
-Important asyncio components, like :class:`asyncio.TaskGroup` and the
-:func:`asyncio.timeout` context manager, are implemented using cancellation
-internally and might misbehave if a coroutine swallows
-:exc:`asyncio.CancelledError`.
+The asyncio components that enable structured concurrency, like
+:class:`asyncio.TaskGroup` and :func:`asyncio.timeout`,
+are implemented using cancellation internally and might misbehave if
+a coroutine swallows :exc:`asyncio.CancelledError`. Similarly, user code
+should not call :meth:`uncancel <asyncio.Task.uncancel>`.
+.. _taskgroups:
Task Groups
===========
@@ -1003,76 +1005,6 @@ Task Object
Deprecation warning is emitted if *loop* is not specified
and there is no running event loop.
- .. method:: cancel(msg=None)
-
- Request the Task to be cancelled.
-
- This arranges for a :exc:`CancelledError` exception to be thrown
- into the wrapped coroutine on the next cycle of the event loop.
-
- The coroutine then has a chance to clean up or even deny the
- request by suppressing the exception with a :keyword:`try` ...
- ... ``except CancelledError`` ... :keyword:`finally` block.
- Therefore, unlike :meth:`Future.cancel`, :meth:`Task.cancel` does
- not guarantee that the Task will be cancelled, although
- suppressing cancellation completely is not common and is actively
- discouraged.
-
- .. versionchanged:: 3.9
- Added the *msg* parameter.
-
- .. deprecated-removed:: 3.11 3.14
- *msg* parameter is ambiguous when multiple :meth:`cancel`
- are called with different cancellation messages.
- The argument will be removed.
-
- .. _asyncio_example_task_cancel:
-
- The following example illustrates how coroutines can intercept
- the cancellation request::
-
- async def cancel_me():
- print('cancel_me(): before sleep')
-
- try:
- # Wait for 1 hour
- await asyncio.sleep(3600)
- except asyncio.CancelledError:
- print('cancel_me(): cancel sleep')
- raise
- finally:
- print('cancel_me(): after sleep')
-
- async def main():
- # Create a "cancel_me" Task
- task = asyncio.create_task(cancel_me())
-
- # Wait for 1 second
- await asyncio.sleep(1)
-
- task.cancel()
- try:
- await task
- except asyncio.CancelledError:
- print("main(): cancel_me is cancelled now")
-
- asyncio.run(main())
-
- # Expected output:
- #
- # cancel_me(): before sleep
- # cancel_me(): cancel sleep
- # cancel_me(): after sleep
- # main(): cancel_me is cancelled now
-
- .. method:: cancelled()
-
- Return ``True`` if the Task is *cancelled*.
-
- The Task is *cancelled* when the cancellation was requested with
- :meth:`cancel` and the wrapped coroutine propagated the
- :exc:`CancelledError` exception thrown into it.
-
.. method:: done()
Return ``True`` if the Task is *done*.
@@ -1186,3 +1118,125 @@ Task Object
in the :func:`repr` output of a task object.
.. versionadded:: 3.8
+
+ .. method:: cancel(msg=None)
+
+ Request the Task to be cancelled.
+
+ This arranges for a :exc:`CancelledError` exception to be thrown
+ into the wrapped coroutine on the next cycle of the event loop.
+
+ The coroutine then has a chance to clean up or even deny the
+ request by suppressing the exception with a :keyword:`try` ...
+ ... ``except CancelledError`` ... :keyword:`finally` block.
+ Therefore, unlike :meth:`Future.cancel`, :meth:`Task.cancel` does
+ not guarantee that the Task will be cancelled, although
+ suppressing cancellation completely is not common and is actively
+ discouraged.
+
+ .. versionchanged:: 3.9
+ Added the *msg* parameter.
+
+ .. deprecated-removed:: 3.11 3.14
+ *msg* parameter is ambiguous when multiple :meth:`cancel`
+ are called with different cancellation messages.
+ The argument will be removed.
+
+ .. _asyncio_example_task_cancel:
+
+ The following example illustrates how coroutines can intercept
+ the cancellation request::
+
+ async def cancel_me():
+ print('cancel_me(): before sleep')
+
+ try:
+ # Wait for 1 hour
+ await asyncio.sleep(3600)
+ except asyncio.CancelledError:
+ print('cancel_me(): cancel sleep')
+ raise
+ finally:
+ print('cancel_me(): after sleep')
+
+ async def main():
+ # Create a "cancel_me" Task
+ task = asyncio.create_task(cancel_me())
+
+ # Wait for 1 second
+ await asyncio.sleep(1)
+
+ task.cancel()
+ try:
+ await task
+ except asyncio.CancelledError:
+ print("main(): cancel_me is cancelled now")
+
+ asyncio.run(main())
+
+ # Expected output:
+ #
+ # cancel_me(): before sleep
+ # cancel_me(): cancel sleep
+ # cancel_me(): after sleep
+ # main(): cancel_me is cancelled now
+
+ .. method:: cancelled()
+
+ Return ``True`` if the Task is *cancelled*.
+
+ The Task is *cancelled* when the cancellation was requested with
+ :meth:`cancel` and the wrapped coroutine propagated the
+ :exc:`CancelledError` exception thrown into it.
+
+ .. method:: uncancel()
+
+ Decrement the count of cancellation requests to this Task.
+
+ Returns the remaining number of cancellation requests.
+
+ Note that once execution of a cancelled task completed, further
+ calls to :meth:`uncancel` are ineffective.
+
+ .. versionadded:: 3.11
+
+ This method is used by asyncio's internals and isn't expected to be
+ used by end-user code. In particular, if a Task gets successfully
+ uncancelled, this allows for elements of structured concurrency like
+ :ref:`taskgroups` and :func:`asyncio.timeout` to continue running,
+ isolating cancellation to the respective structured block.
+ For example::
+
+ async def make_request_with_timeout():
+ try:
+ async with asyncio.timeout(1):
+ # Structured block affected by the timeout:
+ await make_request()
+ await make_another_request()
+ except TimeoutError:
+ log("There was a timeout")
+ # Outer code not affected by the timeout:
+ await unrelated_code()
+
+ While the block with ``make_request()`` and ``make_another_request()``
+ might get cancelled due to the timeout, ``unrelated_code()`` should
+ continue running even in case of the timeout. This is implemented
+ with :meth:`uncancel`. :class:`TaskGroup` context managers use
+ :func:`uncancel` in a similar fashion.
+
+ .. method:: cancelling()
+
+ Return the number of pending cancellation requests to this Task, i.e.,
+ the number of calls to :meth:`cancel` less the number of
+ :meth:`uncancel` calls.
+
+ Note that if this number is greater than zero but the Task is
+ still executing, :meth:`cancelled` will still return ``False``.
+ This is because this number can be lowered by calling :meth:`uncancel`,
+ which can lead to the task not being cancelled after all if the
+ cancellation requests go down to zero.
+
+ This method is used by asyncio's internals and isn't expected to be
+ used by end-user code. See :meth:`uncancel` for more details.
+
+ .. versionadded:: 3.11
diff --git a/Lib/asyncio/tasks.py b/Lib/asyncio/tasks.py
index 56a355cbdc70..e48da0f20088 100644
--- a/Lib/asyncio/tasks.py
+++ b/Lib/asyncio/tasks.py
@@ -243,8 +243,8 @@ def cancelling(self):
def uncancel(self):
"""Decrement the task's count of cancellation requests.
- This should be used by tasks that catch CancelledError
- and wish to continue indefinitely until they are cancelled again.
+ This should be called by the party that called `cancel()` on the task
+ beforehand.
Returns the remaining number of cancellation requests.
"""
diff --git a/Lib/test/test_asyncio/test_tasks.py b/Lib/test/test_asyncio/test_tasks.py
index de735ba77aae..04bdf6483131 100644
--- a/Lib/test/test_asyncio/test_tasks.py
+++ b/Lib/test/test_asyncio/test_tasks.py
@@ -521,7 +521,7 @@ async def task():
finally:
loop.close()
- def test_uncancel(self):
+ def test_uncancel_basic(self):
loop = asyncio.new_event_loop()
async def task():
@@ -534,17 +534,137 @@ async def task():
try:
t = self.new_task(loop, task())
loop.run_until_complete(asyncio.sleep(0.01))
- self.assertTrue(t.cancel()) # Cancel first sleep
+
+ # Cancel first sleep
+ self.assertTrue(t.cancel())
self.assertIn(" cancelling ", repr(t))
+ self.assertEqual(t.cancelling(), 1)
+ self.assertFalse(t.cancelled()) # Task is still not complete
loop.run_until_complete(asyncio.sleep(0.01))
- self.assertNotIn(" cancelling ", repr(t)) # after .uncancel()
- self.assertTrue(t.cancel()) # Cancel second sleep
+ # after .uncancel()
+ self.assertNotIn(" cancelling ", repr(t))
+ self.assertEqual(t.cancelling(), 0)
+ self.assertFalse(t.cancelled()) # Task is still not complete
+
+ # Cancel second sleep
+ self.assertTrue(t.cancel())
+ self.assertEqual(t.cancelling(), 1)
+ self.assertFalse(t.cancelled()) # Task is still not complete
with self.assertRaises(asyncio.CancelledError):
loop.run_until_complete(t)
+ self.assertTrue(t.cancelled()) # Finally, task complete
+ self.assertTrue(t.done())
+
+ # uncancel is no longer effective after the task is complete
+ t.uncancel()
+ self.assertTrue(t.cancelled())
+ self.assertTrue(t.done())
finally:
loop.close()
+ def test_uncancel_structured_blocks(self):
+ # This test recreates the following high-level structure using uncancel()::
+ #
+ # async def make_request_with_timeout():
+ # try:
+ # async with asyncio.timeout(1):
+ # # Structured block affected by the timeout:
+ # await make_request()
+ # await make_another_request()
+ # except TimeoutError:
+ # pass # There was a timeout
+ # # Outer code not affected by the timeout:
+ # await unrelated_code()
+
+ loop = asyncio.new_event_loop()
+
+ async def make_request_with_timeout(*, sleep: float, timeout: float):
+ task = asyncio.current_task()
+ loop = task.get_loop()
+
+ timed_out = False
+ structured_block_finished = False
+ outer_code_reached = False
+
+ def on_timeout():
+ nonlocal timed_out
+ timed_out = True
+ task.cancel()
+
+ timeout_handle = loop.call_later(timeout, on_timeout)
+ try:
+ try:
+ # Structured block affected by the timeout
+ await asyncio.sleep(sleep)
+ structured_block_finished = True
+ finally:
+ timeout_handle.cancel()
+ if (
+ timed_out
+ and task.uncancel() == 0
+ and sys.exc_info()[0] is asyncio.CancelledError
+ ):
+ # Note the five rules that are needed here to satisfy proper
+ # uncancellation:
+ #
+ # 1. handle uncancellation in a `finally:` block to allow for
+ # plain returns;
+ # 2. our `timed_out` flag is set, meaning that it was our event
+ # that triggered the need to uncancel the task, regardless of
+ # what exception is raised;
+ # 3. we can call `uncancel()` because *we* called `cancel()`
+ # before;
+ # 4. we call `uncancel()` but we only continue converting the
+ # CancelledError to TimeoutError if `uncancel()` caused the
+ # cancellation request count go down to 0. We need to look
+ # at the counter vs having a simple boolean flag because our
+ # code might have been nested (think multiple timeouts). See
+ # commit 7fce1063b6e5a366f8504e039a8ccdd6944625cd for
+ # details.
+ # 5. we only convert CancelledError to TimeoutError; for other
+ # exceptions raised due to the cancellation (like
+ # a ConnectionLostError from a database client), simply
+ # propagate them.
+ #
+ # Those checks need to take place in this exact order to make
+ # sure the `cancelling()` counter always stays in sync.
+ #
+ # Additionally, the original stimulus to `cancel()` the task
+ # needs to be unscheduled to avoid re-cancelling the task later.
+ # Here we do it by cancelling `timeout_handle` in the `finally:`
+ # block.
+ raise TimeoutError
+ except TimeoutError:
+ self.assertTrue(timed_out)
+
+ # Outer code not affected by the timeout:
+ outer_code_reached = True
+ await asyncio.sleep(0)
+ return timed_out, structured_block_finished, outer_code_reached
+
+ # Test which timed out.
+ t1 = self.new_task(loop, make_request_with_timeout(sleep=10.0, timeout=0.1))
+ timed_out, structured_block_finished, outer_code_reached = (
+ loop.run_until_complete(t1)
+ )
+ self.assertTrue(timed_out)
+ self.assertFalse(structured_block_finished) # it was cancelled
+ self.assertTrue(outer_code_reached) # task got uncancelled after leaving
+ # the structured block and continued until
+ # completion
+ self.assertEqual(t1.cancelling(), 0) # no pending cancellation of the outer task
+
+ # Test which did not time out.
+ t2 = self.new_task(loop, make_request_with_timeout(sleep=0, timeout=10.0))
+ timed_out, structured_block_finished, outer_code_reached = (
+ loop.run_until_complete(t2)
+ )
+ self.assertFalse(timed_out)
+ self.assertTrue(structured_block_finished)
+ self.assertTrue(outer_code_reached)
+ self.assertEqual(t2.cancelling(), 0)
+
def test_cancel(self):
def gen():
More information about the Python-checkins
mailing list