[Python-Dev] PEP 0424: A method for exposing a length hint

[Mark Shannon]

Brett Cannon wrote:
> 
> 
> On Sun, Jul 15, 2012 at 10:39 AM, Mark Shannon <mark at hotpy.org 
> <mailto:mark at hotpy.org>> wrote:
> 
>     Nick Coghlan wrote:
> 
>         Right, I agree on the value in being able to return something to
>         say "this cannot be converted to a concrete container".
> 
>         I still haven't seen a use case where the appropriate response
>         to "I don't know" differs from the appropriate response to a
>         hint of zero - that is, you don't preallocate, you just start
>         iterating.
> 
> 
>     There seem to be 5 possible classes values of __length_hint__ that an
>     iterator object can provide:
> 
>     1. Don't implement it at all.
> 
>     2. Implement __length_hint__() but don't want to return any value.
>        Either raise an exception (TypeError) -- As suggested in the PEP.
>        or return NotImplemented -- my preferred option.
> 
>     3. Return a "don't know" value:
>        Returning 0 would be fine for this, but the VM might want to respond
>        differently to "don't know" and 0.
>             __length_hint__() == 0             container should be
>     minimum size.
>             __length_hint__() == "unknown"     container starts at
>     default size.
> 
> 
>     4. Infinite iterator:
>        Could return float('inf'), but given this is a "hint" then
>        returning sys.maxsize or sys.maxsize + 1 might be OK.
>        Alternatively raise an OverflowError
> 
> 
> I am really having a hard time differentiating infinity with "I don't 
> know" since they are both accurate from the point of view of 
> __length_hint__ and its typical purpose of allocation. You have no clue 
> how many values will be grabbed from an infinite iterator, so it's the 
> same as just not knowing upfront how long the iterator will be, infinite 
> or not, and thus not worth distinguishing.
>  
> 
> 
>     5. A meaningful length. No problem :)
> 
>     Also, what are the allowable return types?
> 
>     1. int only
>     2. Any number (ie any type with a __int__() method)?
>     3. Or any integer-like object (ie a type with a __index__() method)?
> 
>     My suggestion:
> 
>     a) Don't want to return any value or "don't know": return NotImplemented
>     b) For infinite iterators: raise an OverflowError
>     c) All other cases: return an int or a type with a __index__() method.
> 
> 
> I'm fine with (a), drop (b), and for (c) use what we allow for __len__() 
> since, as Nick's operator.length_hint pseudo-code suggests, people will 
> call this as a fallback if __len__ isn't defined.

So how does an iterator express infinite length?

What should happen if I am silly enough to do this:
 >>> list(itertools.count())

This will fail; it should fail quickly.

Cheers,
Mark.