[Python-Dev] Weak Dictionary Iteration Behavior in Python 3

[Armin Ronacher]

Hi everybody,

In Python 2.x when iterating over a weak key dictionary for example, the common
idom for doing that was calling dictionary.keys() to ensure that a list of all
objects is returned it was safe to iterate over as a weak reference could stop
existing during dict iteration which of course raises a runtime error by the
dict iterator.

This was documented behavior and worked pretty well, with the small problem that
suddenly all references in the dict wouldn't die until iteration is over because
the list holds references to the object.

This no longer works in Python 3 because .keys() on the weak key dictionary
returns a generator over the key view of the internal dict which of course has
the same problem as iterkeys in Python 2.x.

The following code shows the problem::

    from weakref import WeakKeyDictionary

    f1 = Foo()
    f2 = Foo()
    d = WeakKeyDictionary()
    d[f1] = 42
    d[f2] = 23

    i = iter(d.keys()) # or use d.keyrefs() here which has the same problem
    print(next(i))
    del f2
    print(next(i))

This example essentially dies with "RuntimeError: dictionary changed
size during iteration" as soon as f2 is deleted.

Iterating over weak key dictionaries might not be the most common task but I
know some situations where this is necessary.  Unfortunately I can't see a
way to achieve that in Python 3.

Regards,
Armin