[issue44622] async-for loops are traced incorrectly in Python 3.10

[Ned Batchelder]

New submission from Ned Batchelder <ned at nedbatchelder.com>:

In Python 3.10, the traces at the end of an async-for loop are incorrect and different than at the end of a for-loop.

------------------------------
import linecache, sys

def trace(frame, event, arg):
    # The weird globals here is to avoid a NameError on shutdown...
    if frame.f_code.co_filename == globals().get("__file__"):
        lineno = frame.f_lineno
        print("{} {}: {}".format(event[:4], lineno, linecache.getline(__file__, lineno).rstrip()))
    return trace

import asyncio

class AsyncIter:
    def __init__(self, items): self.items = items

    async def __aiter__(self):
        for i in self.items: yield i

async def test1():
    async for i in AsyncIter([1]):
        print(f"test1 {i}")

def test2():
    for i in [1]:
        print(f"test2 {i}")

print(sys.version)
sys.settrace(trace)

asyncio.run(test1())
test2()
------------------------------------

In 3.7, 3.8 and 3.9, the two for loops behave the same: the loop jumps back to the "for" statement, and then returns (the arrowed lines):

  3.9.5 (default, May  5 2021, 06:50:43)
  [Clang 12.0.0 (clang-1200.0.32.29)]
  call 18: async def test1():
  line 19:     async for i in AsyncIter([1]):
  call 13:     def __init__(self, items): self.items = items
  line 13:     def __init__(self, items): self.items = items
  retu 13:     def __init__(self, items): self.items = items
  call 15:     async def __aiter__(self):
  line 16:         for i in self.items: yield i
  retu 16:         for i in self.items: yield i
  exce 19:     async for i in AsyncIter([1]):
  line 20:         print(f"test1 {i}")
  test1 1
  line 19:     async for i in AsyncIter([1]):
  call 16:         for i in self.items: yield i
  line 16:         for i in self.items: yield i
  retu 16:         for i in self.items: yield i
  exce 19:     async for i in AsyncIter([1]):
> retu 19:     async for i in AsyncIter([1]):
  call 22: def test2():
  line 23:     for i in [1]:
  line 24:         print(f"test2 {i}")
  test2 1
  line 23:     for i in [1]:
> retu 23:     for i in [1]:


In 3.10, the for loop behaves the same, but now the async-for traces the body once more when it doesn't execute, and returns from the body of the loop (the starred line):

  3.10.0b4 (default, Jul 11 2021, 13:51:53) [Clang 12.0.0 (clang-1200.0.32.29)]
  call 18: async def test1():
  line 19:     async for i in AsyncIter([1]):
  call 13:     def __init__(self, items): self.items = items
  line 13:     def __init__(self, items): self.items = items
  retu 13:     def __init__(self, items): self.items = items
  call 15:     async def __aiter__(self):
  line 16:         for i in self.items: yield i
  retu 16:         for i in self.items: yield i
  exce 19:     async for i in AsyncIter([1]):
  line 20:         print(f"test1 {i}")
  test1 1
  line 19:     async for i in AsyncIter([1]):
  call 16:         for i in self.items: yield i
  line 16:         for i in self.items: yield i
  retu 16:         for i in self.items: yield i
  exce 19:     async for i in AsyncIter([1]):
* line 20:         print(f"test1 {i}")
  retu 20:         print(f"test1 {i}")
  call 22: def test2():
  line 23:     for i in [1]:
  line 24:         print(f"test2 {i}")
  test2 1
  line 23:     for i in [1]:
> retu 23:     for i in [1]:

----------
components: Interpreter Core
keywords: 3.10regression
messages: 397396
nosy: Mark.Shannon, nedbat
priority: normal
severity: normal
status: open
title: async-for loops are traced incorrectly in Python 3.10
type: behavior
versions: Python 3.10

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue44622>
_______________________________________