[Python-ideas] Introduce collections.Reiterable

[Oscar Benjamin]

On 23 September 2013 11:43, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 23 September 2013 18:55, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:
>> On 23 September 2013 09:44, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>>
>>> The fact Python 3 lets you create ranges you can't directly take the
>>> length of is a bit of a hack, too (because of the pain involved in
>>> defining an alternative __len__ protocol that didn't funnel everything
>>> through an ssize_t value):
>>>
>>>>>> x = range(10**100)
>>>>>> len(x)
>>> Traceback (most recent call last):
>>>   File "<stdin>", line 1, in <module>
>>> OverflowError: Python int too large to convert to C ssize_t
>>>>>> (x.stop - x.start) // x.step
>>> 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
>>
>> I wouldn't call that a "hack". It's clearly a bug.
>
> The hack is that we were able to make those large ranges possible
> *without* fixing the limitation that len() (or, more accurately, the
> CPython tp_len slot) only supports 64-bit containers. Solving the
> latter is a *much* harder problem that would require a PEP to add a
> new type slot, and it's hard to justify doing all that work for such a
> niche use case, especially when the workaround is relatively simple.
>
> If we'd taken the purist approach, then the result would more likely
> have been that ranges would have remained limited to lengths that fit
> in 64 bits rather than that the 64-bit limitation would have been
> removed.

It may not be worth fixing but I still consider it a bug. It also
doesn't work for Python classes:

$ python3
Python 3.3.2 (v3.3.2:d047928ae3f6, May 16 2013, 00:03:43) [MSC v.1600
32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> class myrange:
...   def __len__(self): return 10 ** 1000
...
>>> len(myrange())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: cannot fit 'int' into an index-sized integer


Oscar