Unexpected Behavior Iterating over a Mutating Object

[Bengt Richter]

On Wed, 14 Sep 2005 11:12:14 +0200, bruno modulix <onurb at xiludom.gro> wrote:

>Dave Hansen wrote:
>(snip code snippets and sensible explanations)
>
>> Again, iterating over an item that is mutating seems like a Bad
>> Idea(tm) to me.  
>
>It as *always* been a bad idea to modify a list in place (I mean adding
>or removing items) while iterating over it, whatever the language. If
>you *really* need to do such a thing (for effeciency reasons - and then
>it's usually in low-level C code), you'd better use indexed access, and
>adjust the index as needed - but this results in tricky, hard to
>maintain code.
Do you consider this to be tricky, or likely to be hard to maintain?
(not specifically tested beyond what you see ;-)

 >>> data = [ 'First', 'Second DEL', 'Third', 'Fourth',
 ...          'Fifth DEL', 'DEL Sixth', 'Seventh DEL', 'Eighth DEL',
 ...          'Ninth DEL', 'Tenth', 'Eleventh', 'Twelfth']
 >>> iw = 0
 >>> for item in data:
 ...     if 'DEL' in item: continue
 ...     data[iw] = item
 ...     iw += 1
 ...
 >>> del data[iw:]
 >>> data
 ['First', 'Third', 'Fourth', 'Tenth', 'Eleventh', 'Twelfth']

Or that maybe a utility function like the following would be hard to maintain?

 >>> def filter_in_place(L, keep=bool):
 ...     iw = 0
 ...     for item in L:
 ...         if keep(item):
 ...             L[iw] = item
 ...             iw += 1
 ...     del L[iw:]
 ...
 >>> data = [ 'First', 'Second DEL', 'Third', 'Fourth',
 ...          'Fifth DEL', 'DEL Sixth', 'Seventh DEL', 'Eighth DEL',
 ...          'Ninth DEL', 'Tenth', 'Eleventh', 'Twelfth']
 >>> data
 ['First', 'Second DEL', 'Third', 'Fourth', 'Fifth DEL', 'DEL Sixth', 'Seventh DEL', 'Eighth DEL'
 , 'Ninth DEL', 'Tenth', 'Eleventh', 'Twelfth']
 >>> filter_in_place(data, lambda x: 'DEL' not in x)
 >>> data
 ['First', 'Third', 'Fourth', 'Tenth', 'Eleventh', 'Twelfth']


>
>> But I was curious: is this the intended behavior, or
>> does this fall under what C programmers would call 'undefined
>> behavior.'
>
>Not being a Language Lawyer(tm), I can't tell for sure, but I'd think
>it's the expected behavior. Anyway it's not a behavior I'd relie upon,
>since it would be too much of a dirty trick anyway.
>
I think it depends on the kind of mutation involved. A mutation of the _structure_
of a container being iterated over effectively modifies the initial parameters of
the iteration control, so there you need either to have intimate knowledge of how
the iterator does its thing, or you have to insulate yourself from that need.
OTOH, a mutation of the content can be safe, if it doesn't modify the content
yet to be accessed during the iteration.

In the case of a list, the order can be depended on, so it's not that hard.
Other iterable containers are not so clear, and of course even in a list you
could have cases of access to early items having side effects on upcoming items,
e.g., if the "keep" evaluation had side effects on deep data shared between early
and later list items. But that would be nasty any which way ;-)

Regards,
Bengt Richter