[C++-sig] Re: Adding __len__ to range objects

[David Abrahams]

Raoul Gough <RaoulGough at yahoo.co.uk> writes:

> David Abrahams <dave at boost-consulting.com> writes:
>
>> Raoul Gough <RaoulGough at yahoo.co.uk> writes:
> [snip]
>>> in template <...> struct iterator_range, add
>>>
>>>   typename std::iterator_traits<Iterator>::difference_type
>>>   distance () const {
>>>     return std::distance (m_start, m_finish);
>>>   }
> [snip]
>>
>> It only works for random-access iterators.  You could fix it so that
>> it would only create the __len__ function for random-access
>> iterators; that would be better.
>
> Yes, that would be easy enough I guess. BTW, do you mean it wouldn't
> work at all for (e.g.) forward_iterators, or just that it isn't as
> efficient? I thought std::distance worked for most (or all)
> categories, but is only constant time for random_access_iterators.
> It's still going to be (a lot) faster than the equivalent code in
> Python.

You're right, it would work for everything input iterator and above,
which is everything that range() works on.

>> Then, also, range() creates an iterator-returning function, not an
>> iteratable-returning function.  There are subtle differences.
>>
>>     for x in some_list:
>>        print x
>>
>>     for x in some_list:
>>        print x  # prints the same elements
>>
>> but:
>>
>>     for x in some_iterator:
>>        print x
>>
>>     for x in some_iterator:
>>        print x  # prints nothing
>
> Yes, I would actually expect this since I know that it is an iterator
> pair :-) 

I think you're slightly confused.  It is that in C++, but in Python it
is an iterator.  Traditionally, len(x) only applies to iterables in
Python.

> On the other hand, the following should work OK, unless I'm
> mistaken:
>
> iterator_copy = some_iterator
> for x in some_iterator: print x
> for x in iterator_copy: print x    # Prints same

You're mistaken.  iterator_copy and some_iterator both refer to the
same object.  Try it:

    >>> i = iter(range(5))
    >>> j = i
    >>> for x in i: print x
    ...
    0
    1
    2
    3
    4
    >>> for x in j: print x
    ...

>> None of the Python iterators support __len__ and I wonder if it's a
>> good idea to add it to an iterator we create.  Your thoughts?
>
> Yes, maybe it isn't such a great idea after all. One real drawback
> that I thought of is that adding this might break existing code. A
> (broken) user-defined iterator might not provide enough smarts for
> std::distance to work, and yet still be compatible with the existing
> range support. 

It would always supply enough smarts.  However, I am nervous about
supplying __len__ for anything below random-access, especially for
input iterators where it is probably destructive.

> Adding len would break this, since the distance function would get
> generated even if it is never used or wanted.

I don't think that's a problem.

> I can think of two ways around this: add a new range_with_len type
> (seems excessive), or provide some way for the client code to access
> the generated class_ object to add their own extensions (probably
> difficult?)

I think you're barking up the wrong tree.  Maybe we ought to simply
change our tune and say that range() produces an iterable-returning
function rather than an iterator-returning function.  If we did that
we could always generate __len__, and for that matter we could also
generate __getitem__/__setitem__ for random-access ranges.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com