Dangerous behavior of list(generator)

[Steven D'Aprano]

On Sun, 13 Dec 2009 22:45:58 +0000, exarkun wrote:

> On 08:18 pm, steve at remove-this-cybersource.com.au wrote:
>>On Sun, 13 Dec 2009 14:35:21 +0000, exarkun wrote:
>>>>StopIteration is intended to be used only within the .__next__ method
>>>>of
>>>>iterators. The devs know that other 'off-label' use results in the
>>>>inconsistency you noted, but their and my view is 'don't do that'.
>>>
>>>Which is unfortunate, because it's not that hard to get StopIteration
>>>without explicitly raising it yourself and this behavior makes it
>>>difficult to debug such situations.
>>
>>I can't think of any way to get StopIteration without explicitly raising
>>it yourself. It's not like built-ins or common data structures routinely
>>raise StopIteration. I don't think I've *ever* seen a StopIteration that
>>I didn't raise myself.
> 
> Call next on an iterator.  For example:  iter(()).next()

Or in more recent versions of Python, next(iter(())).

Good example. But next() is a special case, and since next() is 
documented as raising StopIteration if you call it and it raises 
StopIteration, you have raised it yourself. Just not explicitly.



>>>What's with this view, exactly?  Is it just that it's hard to implement
>>>the more desirable behavior?
>>
>>What is that "more desirable behaviour"? That StopIteration is used to
>>signal that Python should stop iterating except when you want it to be
>>ignored? Unfortunately, yes, it's quite hard to implement "do what the
>>caller actually wants, not what he asked for" behaviour -- and even if
>>it were possible, it goes against the grain of the Zen of Python.
>>
>>If you've ever had to debug faulty "Do What I Mean" software, you'd see
>>this as a good thing.
> 
> I have plenty of experience developing and debugging software, Steven.
> Your argument is specious, as it presupposes that only two possibilities
> exist: the current behavior of some kind of magical faerie land
> behavior.
> 
> I'm surprised to hear you say that the magical faerie land behavior
> isn't desirable either, though.  I'd love a tool that did what I wanted,
> not what I asked.  The only serious argument against this, I think, is
> that it is beyond our current ability to create (and so anyone claiming
> to be able to do it is probably mistaken).

I'd argue that tools that do what you want rather than what you ask for 
are not just currently impossible, but always will be -- no matter how 
good the state of the art of artificial intelligent mind-reading software 
becomes.



> You chopped out all the sections of this thread which discussed the more
> desirable behavior.  You can go back and read them in earlier messages
> if you need to be reminded.  I'm not talking about anything beyond
> what's already been raised.

I'm glad for you. But would you mind explaining for those of us aren't 
mind-readers what YOU consider the "more desirable behaviour"?

If you're talking the list constructor and list comprehensions treating 
StopIteration the same, then I don't think it is at all self-evident that 
the current behaviour is a bad thing, nor that the only reason for it is 
that to do otherwise would be hard.

(I don't think it would be hard to have list comps swallow a 
StopIteration.)


> I'm pretty sure I know the answer to my question, though - it's hard to
> implement, so it's not implemented.
> 
> Jean-Paul


-- 
Steven