[New-bugs-announce] [issue45809] Race condition in WeakKeyDictionary/WeakKeyDictionary

[Sam Gross]

New submission from Sam Gross <colesbury at gmail.com>:

The issue described issue7105 (and maybe issue7060) still exists due to a race condition in WeakKeyDictionary. This shows up as test failure that looks like:

  test test_weakref failed -- Traceback (most recent call last):
    File "Lib/test/test_weakref.py", line 1960, in test_threaded_weak_value_dict_deepcopy
      self.check_threaded_weak_dict_copy(weakref.WeakValueDictionary, True)
    File "Lib/test/test_weakref.py", line 1940, in check_threaded_weak_dict_copy
      raise exc[0]
    File "Lib/test/test_weakref.py", line 1897, in dict_copy
      _ = copy.deepcopy(d)
    File "Lib/copy.py", line 153, in deepcopy
      y = copier(memo)
    File "Lib/weakref.py", line 189, in __deepcopy__
      for key, wr in self.data.items():
  RuntimeError: dictionary changed size during iteration

The cause is that the check of "self._iterating" and the call to "_atomic_removal" are not performed atomically together. By the time _atomic_removal() is called, an iteration might have already started.

https://github.com/python/cpython/blob/ec382fac0db6d9159c2d3496a70b7a605545957e/Lib/weakref.py#L109-L114

----------
components: Library (Lib)
messages: 406357
nosy: colesbury
priority: normal
severity: normal
status: open
title: Race condition in WeakKeyDictionary/WeakKeyDictionary
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45809>
_______________________________________