"dictionary changed size during iteration" error in Python 3 but not in Python 2

[Cameron Simpson]

On 23Aug2020 10:00, Chris Green <cl at isbd.net> wrote:
>I have a (fairly) simple little program that removes old mail messages
>from my junk folder.  I have just tried to upgrade it from Python 2 to
>Python 3 and now, when it finds any message[s] to delete it produces
>the error:-
>
>    RuntimeError: dictionary changed size during iteration
>
>I can sort of see why I'm getting the error but have two questions:
>
>1 - Why doesn't it error in Python 2?

The dict internal implementation has changed. I don't know the 
specifics, but it is now faster and maybe smaller and also now preserves 
insert order.

>2 - How do I fix it?

That standard way is to take a copy of what you're iterating over, 
usually just the keys (consumes less memory).

BTW, this approach is standard for _any_ data structure you're iterating 
over and modifying.

So:

    for k, v in d.iteritems():
       ... d.remove(k) ...

might become:

    for k, v in list(d.iteritems()):
       ... d.remove(k) ...

or:

    for k in list(d):
       v = d[k]
       ... d.remove(k) ...

The iterators from iteritems or iterkeys (and plain old items and keys 
in Python 3) walk over the internal structure, which is a hash table. A 
typical dynamic hash table will resize every so often if items are 
inserted or removed, which doesn't just rearrange things, it can reorder 
the keys because the distribution of the keys into buckets can change as 
the number of buckets changes.

Cheers,
Cameron Simpson <cs at cskk.id.au>