[Python-ideas] Introduce collections.Reiterable

[Nick Coghlan]

On 23 September 2013 18:04, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Executive summary:
>
> The ability to create a quick iterable with just a simple __getitem__
> is cool and not a "hack" (ie, no need whatsoever to deprecate it), but
> it is clearly a "consenting adults" construction (which includes
> "knowing where your children are at 10pm").
>
> Steven D'Aprano writes:
>
>  > I agree, and I disagree with Nick's characterization of the
>  > sequence protocol as a "backwards-compatibility hack". It is an
>  > elegant protocol
>
> Gotta disagree with you there (except I agree there's no need for a
> word like "hack").  Because __getitem__ is polymorphic (at the
> abstract level of duck-typing), this protocol is ugly.  The "must
> accept 0" clause is a wart.

I think others object to the word "hack" more than I do (or give it
additional implications like "in danger of being deprecated"). To me
it's just a shorthand for saying "this is a case where practicality
beat purity". Just because something is a hack doesn't mean it isn't
useful and isn't a good idea.

I consider functools.wraps to be a hack that managed to preserve
introspectability of most decorated functions with minimal development
effort. runpy and the -m switch took quite a while to evolve into
something that wasn't a hack (although they still have some hacky
parts due to limitations of the import protocol). The code that makes
objects that override __eq__ without overriding __hash__ non-hashable
(and the associated "__hash__ = None") trick is a hack. Python 3's new
super is incredibly nice and easy to use, but there's also a lot of
hackery lurking behind it.

The fact Python 3 lets you create ranges you can't directly take the
length of is a bit of a hack, too (because of the pain involved in
defining an alternative __len__ protocol that didn't funnel everything
through an ssize_t value):

>>> x = range(10**100)
>>> len(x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OverflowError: Python int too large to convert to C ssize_t
>>> (x.stop - x.start) // x.step
10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

To create a properly defined iterable in modern Python, you must
either implement __iter__ or implement an iter() compatible
__getitem__ and explicitly register with Iterable (to indicate that
your __getitem__ *is* compatible with the fallback protocol in
iter()). Steven's right that I left out that second alternative when
stating what it takes for an item to be considered an iterable, but I
still consider the __getitem__ fallback to be just a neat backwards
compatibility hack for sequences that were defined before the iterator
protocol existed and before the Iterable ABC provided a way to
explicitly declare that your __getitem__ implementation was compatible
with the sequence-iterator protocol.

I was also wrong about iter() checking for __len__ - that's part of
the sequence API fallback in reversed(), rather than the one in
iter():

>>> class InfiniteIter:
...     def __getitem__(self, idx):
...         return idx
...
>>> reversed(InfiniteIter())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: object of type 'InfiniteIter' has no len()

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia