[issue19332] Guard against changing dict during iteration

[Raymond Hettinger]

Raymond Hettinger added the comment:

A few thoughts:

* No existing, working code will benefit from this patch; however, almost all code will pay a price for it -- bigger size for an empty dict and a runtime cost (possibly very small) on the critical path (every time a value is stored in a dict).

* The sole benefit of the patch is provide an earlier warning that someone is doing something weird.  For most people, this will never come up (we have 23 years of Python history indicating that there isn't a real problem to that needs to be solved). 

* The normal rule (not just for Python) is that a data structures have undefined behavior for mutating while iterating, unless there is a specific guarantee (for example, we guarantee that the dicts are allowed to mutate values but not keys during iteration and we guarantee the behavior of list iteration while iterating).

* It is not clear that other implementations such as IronPython and Jython would be able to implement this behavior (Jython wraps the Java ConcurrentHashMap).

* The current patch second guesses a decision that was made long ago to only detect size changes (because it is cheap, doesn't take extra memory, isn't on the critical path, and handles the common case).

* The only case whether we truly need a stronger protection is when it is needed to defend against a segfault.  That is why collections.deque() implement a change counter.  It has a measureable cost that slows down  deque operations (increasing the number of memory accesses per append, pop, or next) but it is needed to prevent the iterator from spilling into freed memory.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue19332>
_______________________________________