[Python-Dev] Issue 19332: Guard against changing dict during iteration

[Victor Stinner]

2013/11/6 R. David Murray <rdmurray at bitdance.com>:
> On Wed, 06 Nov 2013 23:34:22 +1100, Steven D'Aprano <steve at pearwood.info> wrote:
>> On Tue, Nov 05, 2013 at 08:38:09PM -0800, Ethan Furman wrote:
>>
>> > http://bugs.python.org/issue19332
>>
>> Duplicate of this: http://bugs.python.org/issue6017
>>
>> The conclusion on that also was that it is not worth guarding against
>> such an unusual circumstance.
>
> If I remember correctly (and I may not) the size-change guards were
> added because without them there were certain cases that could
> either segfault or result in an infinite loop.
>
> --David

This exception is quite old:
---
changeset:   17597:32e7d0898eab
branch:      legacy-trunk
user:        Guido van Rossum <guido at python.org>
date:        Fri Apr 20 19:13:02 2001 +0000
files:       Include/Python.h Include/abstract.h Include/object.h
Include/opcode.h Include/pyerrors.h Lib/dis.py Makefile.pre.in
Objects/abstra
description:
Iterators phase 1.  This comprises:

new slot tp_iter in type object, plus new flag Py_TPFLAGS_HAVE_ITER
new C API PyObject_GetIter(), calls tp_iter
new builtin iter(), with two forms: iter(obj), and iter(function, sentinel)
new internal object types iterobject and calliterobject
new exception StopIteration
new opcodes for "for" loops, GET_ITER and FOR_ITER (also supported by dis.py)
new magic number for .pyc files
new special method for instances: __iter__() returns an iterator
iteration over dictionaries: "for x in dict" iterates over the keys
iteration over files: "for x in file" iterates over lines

TODO:

documentation
test suite
decide whether to use a different way to spell iter(function, sentinal)
decide whether "for key in dict" is a good idea
use iterators in map/filter/reduce, min/max, and elsewhere (in/not in?)
speed tuning (make next() a slot tp_next???)
---

More recently, I added another exception if a dictionary is modified
during a lookup.

When I proposed a new frozendict type to secure my pysandbox project,
Armin Rigo wrote that CPython segfaults must be fixed. So I fixed a
corner case on dictionaries.
http://bugs.python.org/issue14205

Thread in python-dev:
https://mail.python.org/pipermail/python-dev/2012-March/117290.html

The discussion in http://bugs.python.org/issue14205 is interesting.

Victor