[Python-ideas] __len__() for map()

[Steven D'Aprano]

On Wed, Nov 28, 2018 at 02:53:50PM -0500, Terry Reedy wrote:

> One of the guidelines in the Zen of Python is
> "Special cases aren't special enough to break the rules."
> 
> This proposal claims that the Python 3 built-in iterator class 'map' is 
> so special that it should break the rule that iterators in general 
> cannot and therefore do not have .__len__ methods because their size may 
> be infinite, unknowable until exhaustion, or declining with each 
> .__next__ call.
> 
> For iterators, 3.4 added an optional __length_hint__ method.  This makes 
> sense for iterators, like tuple_iterator, list_iterator, range_iterator, 
> and dict_keyiterator, based on a known finite collection.  At the time, 
> map.__length_hint__ was proposed and rejected as problematic, for 
> obvious reasons, and insufficiently useful.

Thanks for the background Terry, but doesn't that suggest that sometimes 
special cases ARE special enough to break the rules? *wink*

Unfortunately, I don't think it is obvious why map.__length_hint__ is 
problematic. It only needs to return the *maximum* length, or some 
sentinel (zero?) to say "I don't know". It doesn't need to be accurate, 
unlike __len__ itself.

Perhaps we should rethink the decision not to give map() and filter() a 
length hint?


[...]
> What makes the map class special among all built-in iterator classes? 
> It appears not to be a property of the class itself, as an iterator 
> class, but of its name.  In Python 2, 'map' was bound to a different 
> implementation of the map idea, a function that produced a list, which 
> has a length.  I suspect that if Python 3 were the original Python, we 
> would not have this discussion.

No, in fairness, I too have often wanted to know the length of an 
arbitrary iterator, including map(), without consuming it. In general 
this is an unsolvable problem, but sometimes it is (or at least, at first 
glance *seems*) solvable. map() is one of those cases.

If we could solve it, that would be great -- but I'm not convinced that 
it is solvable, since the solution seems worse than the problem it aims 
to solve. But I live in hope that somebody cleverer than me can point 
out the flaws in my argument.


[...]
> If a function is documented as requiring a list, or a sequence, or a 
> length object, it is a user bug to pass an iterator.  The only thing 
> special about map and filter as errors is the rebinding of the names 
> between Py2 and Py3, so that the same code may be good in 2.x and bad in 
> 3.x.
> 
> Perhaps 2.7, in addition to future imports of text as unicode and print 
> as a function, should have had one to make map and filter be the 3.x 
> iterators.

I think that's future_builtins:

[steve at ando ~]$ python2.7 -c "from future_builtins import *; print map(len, [])"
<itertools.imap object at 0xb7ed39ec>

But that wouldn't have helped E. Madison Bray or SageMath, since their 
difficulty is not their own internal use of map(), but their users' use 
of map().

Unless they simply ban any use of iterators at all, which I imagine will 
be a backwards-incompatible change (and for that matter an excessive 
overreaction for many uses), SageMath can't prevent users from providing 
map() objects or other iterator arguments.



-- 
Steve