[Python-checkins] bpo-31183: `dis` now handles coroutines & async generators (GH-3077)
Nick Coghlan
webhook-mailer at python.org
Thu Aug 17 22:29:23 EDT 2017
https://github.com/python/cpython/commit/fe2b56ab9212c1cf19c48b848fa60f7f201c366f
commit: fe2b56ab9212c1cf19c48b848fa60f7f201c366f
branch: master
author: syncosmic <gc at syncosmic.io>
committer: Nick Coghlan <ncoghlan at gmail.com>
date: 2017-08-18T12:29:21+10:00
summary:
bpo-31183: `dis` now handles coroutines & async generators (GH-3077)
Coroutines and async generators use a distinct attribute name for their
code objects, so this updates the `dis` module to correctly disassemble
objects with those attributes.
Due to the increase in the test module length, it also fixes some latent
defects in the tests related to how the displayed source line numbers
are extracted.
https://bugs.python.org/issue31230 is a follow-up issue suggesting we
may want to solve this a different way, by instead giving all these object
types a common `__code__` attribute, avoiding the need for special
casing in the `dis` module.
files:
A Misc/NEWS.d/next/Library/2017-08-13-09-17-01.bpo-31183.-2_YGj.rst
M Doc/library/dis.rst
M Lib/dis.py
M Lib/test/test_dis.py
diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst
index bc32380e976..7b7c84df776 100644
--- a/Doc/library/dis.rst
+++ b/Doc/library/dis.rst
@@ -53,8 +53,9 @@ code.
.. class:: Bytecode(x, *, first_line=None, current_offset=None)
- Analyse the bytecode corresponding to a function, generator, method, string
- of source code, or a code object (as returned by :func:`compile`).
+ Analyse the bytecode corresponding to a function, generator, asynchronous
+ generator, coroutine, method, string of source code, or a code object (as
+ returned by :func:`compile`).
This is a convenience wrapper around many of the functions listed below, most
notably :func:`get_instructions`, as iterating over a :class:`Bytecode`
@@ -92,6 +93,9 @@ code.
Return a formatted multi-line string with detailed information about the
code object, like :func:`code_info`.
+ .. versionchanged:: 3.7
+ This can now handle coroutine and asynchronous generator objects.
+
Example::
>>> bytecode = dis.Bytecode(myfunc)
@@ -114,7 +118,8 @@ operation is being performed, so the intermediate analysis object isn't useful:
.. function:: code_info(x)
Return a formatted multi-line string with detailed code object information
- for the supplied function, generator, method, source code string or code object.
+ for the supplied function, generator, asynchronous generator, coroutine,
+ method, source code string or code object.
Note that the exact contents of code info strings are highly implementation
dependent and they may change arbitrarily across Python VMs or Python
@@ -122,6 +127,9 @@ operation is being performed, so the intermediate analysis object isn't useful:
.. versionadded:: 3.2
+ .. versionchanged:: 3.7
+ This can now handle coroutine and asynchronous generator objects.
+
.. function:: show_code(x, *, file=None)
@@ -141,12 +149,13 @@ operation is being performed, so the intermediate analysis object isn't useful:
.. function:: dis(x=None, *, file=None, depth=None)
Disassemble the *x* object. *x* can denote either a module, a class, a
- method, a function, a generator, a code object, a string of source code or
- a byte sequence of raw bytecode. For a module, it disassembles all functions.
- For a class, it disassembles all methods (including class and static methods).
- For a code object or sequence of raw bytecode, it prints one line per bytecode
- instruction. It also recursively disassembles nested code objects (the code
- of comprehensions, generator expressions and nested functions, and the code
+ method, a function, a generator, an asynchronous generator, a couroutine,
+ a code object, a string of source code or a byte sequence of raw bytecode.
+ For a module, it disassembles all functions. For a class, it disassembles
+ all methods (including class and static methods). For a code object or
+ sequence of raw bytecode, it prints one line per bytecode instruction.
+ It also recursively disassembles nested code objects (the code of
+ comprehensions, generator expressions and nested functions, and the code
used for building nested classes).
Strings are first compiled to code objects with the :func:`compile`
built-in function before being disassembled. If no object is provided, this
@@ -164,6 +173,9 @@ operation is being performed, so the intermediate analysis object isn't useful:
.. versionchanged:: 3.7
Implemented recursive disassembling and added *depth* parameter.
+ .. versionchanged:: 3.7
+ This can now handle coroutine and asynchronous generator objects.
+
.. function:: distb(tb=None, *, file=None)
diff --git a/Lib/dis.py b/Lib/dis.py
index b990839bcbf..90ddf4f3360 100644
--- a/Lib/dis.py
+++ b/Lib/dis.py
@@ -32,20 +32,30 @@ def _try_compile(source, name):
return c
def dis(x=None, *, file=None, depth=None):
- """Disassemble classes, methods, functions, generators, or code.
+ """Disassemble classes, methods, functions, and other compiled objects.
With no argument, disassemble the last traceback.
+ Compiled objects currently include generator objects, async generator
+ objects, and coroutine objects, all of which store their code object
+ in a special attribute.
"""
if x is None:
distb(file=file)
return
- if hasattr(x, '__func__'): # Method
+ # Extract functions from methods.
+ if hasattr(x, '__func__'):
x = x.__func__
- if hasattr(x, '__code__'): # Function
+ # Extract compiled code objects from...
+ if hasattr(x, '__code__'): # ...a function, or
x = x.__code__
- if hasattr(x, 'gi_code'): # Generator
+ elif hasattr(x, 'gi_code'): #...a generator object, or
x = x.gi_code
+ elif hasattr(x, 'ag_code'): #...an asynchronous generator object, or
+ x = x.ag_code
+ elif hasattr(x, 'cr_code'): #...a coroutine.
+ x = x.cr_code
+ # Perform the disassembly.
if hasattr(x, '__dict__'): # Class or module
items = sorted(x.__dict__.items())
for name, x1 in items:
@@ -107,16 +117,24 @@ def pretty_flags(flags):
return ", ".join(names)
def _get_code_object(x):
- """Helper to handle methods, functions, generators, strings and raw code objects"""
- if hasattr(x, '__func__'): # Method
+ """Helper to handle methods, compiled or raw code objects, and strings."""
+ # Extract functions from methods.
+ if hasattr(x, '__func__'):
x = x.__func__
- if hasattr(x, '__code__'): # Function
+ # Extract compiled code objects from...
+ if hasattr(x, '__code__'): # ...a function, or
x = x.__code__
- if hasattr(x, 'gi_code'): # Generator
+ elif hasattr(x, 'gi_code'): #...a generator object, or
x = x.gi_code
- if isinstance(x, str): # Source code
+ elif hasattr(x, 'ag_code'): #...an asynchronous generator object, or
+ x = x.ag_code
+ elif hasattr(x, 'cr_code'): #...a coroutine.
+ x = x.cr_code
+ # Handle source code.
+ if isinstance(x, str):
x = _try_compile(x, "<disassembly>")
- if hasattr(x, 'co_code'): # Code object
+ # By now, if we don't have a code object, we can't disassemble x.
+ if hasattr(x, 'co_code'):
return x
raise TypeError("don't know how to disassemble %s objects" %
type(x).__name__)
@@ -443,8 +461,8 @@ def findlinestarts(code):
class Bytecode:
"""The bytecode operations of a piece of code
- Instantiate this with a function, method, string of code, or a code object
- (as returned by compile()).
+ Instantiate this with a function, method, other compiled object, string of
+ code, or a code object (as returned by compile()).
Iterating over this yields the bytecode operations as Instruction instances.
"""
diff --git a/Lib/test/test_dis.py b/Lib/test/test_dis.py
index 254b317e498..bfbbee2794a 100644
--- a/Lib/test/test_dis.py
+++ b/Lib/test/test_dis.py
@@ -331,6 +331,13 @@ def _fstring(a, b, c, d):
def _g(x):
yield x
+async def _ag(x):
+ yield x
+
+async def _co(x):
+ async for item in _ag(x):
+ pass
+
def _h(y):
def foo(x):
'''funcdoc'''
@@ -390,6 +397,7 @@ def foo(x):
_h.__code__.co_firstlineno + 3,
)
+
class DisTests(unittest.TestCase):
maxDiff = None
@@ -531,10 +539,22 @@ def test_disassemble_class_method(self):
self.do_disassembly_test(_C.cm, dis_c_class_method)
def test_disassemble_generator(self):
- gen_func_disas = self.get_disassembly(_g) # Disassemble generator function
- gen_disas = self.get_disassembly(_g(1)) # Disassemble generator itself
+ gen_func_disas = self.get_disassembly(_g) # Generator function
+ gen_disas = self.get_disassembly(_g(1)) # Generator iterator
self.assertEqual(gen_disas, gen_func_disas)
+ def test_disassemble_async_generator(self):
+ agen_func_disas = self.get_disassembly(_ag) # Async generator function
+ agen_disas = self.get_disassembly(_ag(1)) # Async generator iterator
+ self.assertEqual(agen_disas, agen_func_disas)
+
+ def test_disassemble_coroutine(self):
+ coro_func_disas = self.get_disassembly(_co) # Coroutine function
+ coro = _co(1) # Coroutine object
+ coro.close() # Avoid a RuntimeWarning (never awaited)
+ coro_disas = self.get_disassembly(coro)
+ self.assertEqual(coro_disas, coro_func_disas)
+
def test_disassemble_fstring(self):
self.do_disassembly_test(_fstring, dis_fstring)
@@ -1051,11 +1071,13 @@ def test_explicit_first_line(self):
def test_source_line_in_disassembly(self):
# Use the line in the source code
- actual = dis.Bytecode(simple).dis()[:3]
- expected = "{:>3}".format(simple.__code__.co_firstlineno)
+ actual = dis.Bytecode(simple).dis()
+ actual = actual.strip().partition(" ")[0] # extract the line no
+ expected = str(simple.__code__.co_firstlineno)
self.assertEqual(actual, expected)
# Use an explicit first line number
- actual = dis.Bytecode(simple, first_line=350).dis()[:3]
+ actual = dis.Bytecode(simple, first_line=350).dis()
+ actual = actual.strip().partition(" ")[0] # extract the line no
self.assertEqual(actual, "350")
def test_info(self):
diff --git a/Misc/NEWS.d/next/Library/2017-08-13-09-17-01.bpo-31183.-2_YGj.rst b/Misc/NEWS.d/next/Library/2017-08-13-09-17-01.bpo-31183.-2_YGj.rst
new file mode 100644
index 00000000000..ef7a31a8d01
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2017-08-13-09-17-01.bpo-31183.-2_YGj.rst
@@ -0,0 +1,2 @@
+`dis` now works with asynchronous generator and coroutine objects. Patch by
+George Collins based on diagnosis by Luciano Ramalho.
More information about the Python-checkins
mailing list