Unexpected Behavior Iterating over a Mutating Object

[Larry Bates]

You are correct if you remove items from a list that you
are iterating over you get the results you see.  You either
need to iterate over the list in reverse order or better yet
create a new list from the old one with the items removed.

In Python 2.4 you can do:

data = [ 'First', 'Second DEL', 'Third', 'Fourth',
         'Fifth DEL', 'DEL Sixth', 'Seventh DEL', 'Eighth DEL',
         'Ninth DEL', 'Tenth', 'Eleventh', 'Twelfth']

bfr=[x for x in data if 'DEL' not in x]

Note: I'm not sure why 'DEL' is at the end of all the strings
EXCEPT the Sixth one where it is at the beginning, but if the
"DEL" was consistently at the end of the strings I would do
this instead (which would also work in Python 2.2+):

bfr=[x for x in data if not x.endswith('DEL')]

This also is much easier to read (IMHO) than the loops, etc.
of the original code.  In Python 2.4 list comprehensions are
also quite efficient.

Larry Bates

Dave Hansen wrote:
> OK, first, I don't often have the time to read this group, so
> apologies if this is a FAQ, though I couldn't find anything at
> python.org.
> 
> Second, this isn't my code.  I wouldn't do this.  But a colleague did,
> got an unexpected result, and asked me why.  I think I can infer what
> is occurring, and I was able to find a simple work-around.  But I
> thought I'd ask about it anyway.
> 
> I've been pushing Python at work for use as a scripting language to
> get simple tasks done.  My colleague is modifying a parts library for
> a schematic capture program with a Python script.  The library is
> entirely in ASCII text.  
> 
> He's trying to delete a group of parts from the library.  The simple
> code below shows what is occurring (taken from an IDLE session, Python
> 2.4.1#65 for Windoze).  The "parts" to be deleted in the example
> contain the string 'DEL':
> 
> ---begin included file---
> 
>>>>data = [ 'First', 'Second DEL', 'Third', 'Fourth',
> 
> 	     'Fifth DEL', 'DEL Sixth', 'Seventh DEL', 'Eighth DEL',
> 	     'Ninth DEL', 'Tenth', 'Eleventh', 'Twelfth']
> 
> 
>>>>bfr = []
>>>>for item in data:
> 
> 	if item.find('DEL') >= 0:
> 		bfr.append(item)
> 		data.remove(item)
> 
> 		
> 
>>>>data
> 
> ['First', 'Third', 'Fourth', 'DEL Sixth', 'Eighth DEL', 'Tenth',
> 'Eleventh', 'Twelfth']
> 
>>>>bfr
> 
> ['Second DEL', 'Fifth DEL', 'Seventh DEL', 'Ninth DEL']
> ---end included file---
> 
> It seems when an item is 'remove'd from data, the rest of the list
> 'shifts' over one, so what was 'next' now occupies the slot of the
> 'remove'd item.  When the next 'item' is selected from 'data', the
> _desired_ 'next' item is skipped.  So when 'data' has several
> consecutive items to be deleted, only every other one is 'remove'd.
> 
> The workaround is very simple:
> 
> ---begin included file---
> 
>>>>for item in data:
> 
> 	if item.find('DEL') >= 0:
> 		bfr.append(item)
> 
> 		
> 
>>>>for item in bfr:
> 
> 	data.remove(item)
> 
> 	
> 
>>>>data
> 
> ['First', 'Third', 'Fourth', 'Tenth', 'Eleventh', 'Twelfth']
> 
>>>>bfr
> 
> ['Second DEL', 'Fifth DEL', 'DEL Sixth', 'Seventh DEL', 'Eighth DEL',
> 'Ninth DEL']
> ---end included file---
> 
> Again, iterating over an item that is mutating seems like a Bad
> Idea(tm) to me.  But I was curious: is this the intended behavior, or
> does this fall under what C programmers would call 'undefined
> behavior.'
> 
> Thanks,
> 
>                                -=Dave