[issue36473] Detect all dictionary changes during iteration

[Thomas Perl]

New submission from Thomas Perl <th.perl at gmail.com>:

On top of issue 36452, I noticed some other corner cases that are still not handled. For one, the patch (Github PR 12596) only handles iterating over keys, but not iterating over values or items:

======
a = {0: 0}
it = iter(a.values())
print('Length hint:', it.__length_hint__())
print(next(it))
print('Length hint:', it.__length_hint__())
del a[0]
a[1] = 99
print(next(it))
print('Length hint:', it.__length_hint__())
======

Replace a.values() there with a.items() -- same issue. Note that PR 12596 fixes the a.keys() case (same as iterating over "a" directly).

Applying the "di->len == 0" check in dictiter_iternextvalue() and dictiter_iternextitem() would fix those two cases above, but would still not fix the following case:

======
a = {0: 'a', 1: 'b', 2: 'c'}

it = iter(a)
i = next(it)
print('got first:', i)
del a[1]
a[1] = 'd'
i = next(it)
print('got second:', i)
i = next(it)
print('got third:', i)

try:
    i = next(it)
    raise RuntimeError(f'got fourth: {i}')
except StopIteration:
    print('stop iteration')
======

The reason for this is that the iteration count (3 in this case) isn't modified, but the dict's keys are still changed, and the iteration order is as follows:

======
got first: 0
got second: 2
got third: 1
stop iteration
======

Note that the key 1 there is first deleted and then set.

I'll add a Github PR that tries to solve these corner cases too by tracking dict keys modification.

----------
components: Interpreter Core
messages: 339115
nosy: thomas.perl
priority: normal
severity: normal
status: open
title: Detect all dictionary changes during iteration
type: behavior
versions: Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36473>
_______________________________________