[Python-ideas] __len__() for map()

[E. Madison Bray]

On Thu, Nov 29, 2018 at 12:21 PM Chris Angelico <rosuav at gmail.com> wrote:
>
> On Thu, Nov 29, 2018 at 10:18 PM E. Madison Bray <erik.m.bray at gmail.com> wrote:
> >
> > On Thu, Nov 29, 2018 at 12:16 PM Chris Angelico <rosuav at gmail.com> wrote:
> > >
> > > On Thu, Nov 29, 2018 at 10:14 PM E. Madison Bray <erik.m.bray at gmail.com> wrote:
> > > > P.S.
> > > >
> > > > > > This is necessary because if I have a function that used to take, say,
> > > > > > a list as an argument, and it receives a `map` object, I now have to
> > > > > > be able to deal with map()s,
> > > > >
> > > > > If a function is documented as requiring a list, or a sequence, or a
> > > > > length object, it is a user bug to pass an iterator.  The only thing
> > > > > special about map and filter as errors is the rebinding of the names
> > > > > between Py2 and Py3, so that the same code may be good in 2.x and bad in
> > > > > 3.x.
> > > >
> > > > It's not a user bug if you're porting a massive computer algebra
> > > > application that happens to use Python as its implementation language
> > > > (rather than inventing one from scratch) and your users don't need or
> > > > want to know too much about Python 2 vs Python 3.  Besides, the fact
> > > > that they are passing an iterator now is probably in many cases a good
> > > > thing for them, but it takes away my ability as a developer to find
> > > > out more about what they're trying to do, as opposed to say just being
> > > > given a list of finite size.
> > >
> > > If that's the case, then it should be no problem to rebind
> > > builtins.map to return a list. Problem solved.
> >
> > Rebind where?  How?  In sage.__init__?  How do you think that will fly
> > with other packages loaded in the same interpreter?
>
> Either this is Python, or it's just an algebra language that happens
> to be implemented in Python. If the former, the Py2/Py3 distinction
> should matter to your users, since they are programming in Python.

Porque no los dos?  Sage is a superset of Python, and while on some
level (in terms of advanced programming constructs) users will need to
care about the distinction.  But most users don't really know exactly
what it does when they pass something like map(a_func, a_list) as an
argument to a function call.  They don't necessarily appreciate the
distinction that, depending on how that function is implemented, an
arbitrary iterable has to be treated very differently than a list.

I certainly don't mind supporting arbitrary iterables--I think they
should be supported.  But now there are optimizations I can't make
that I could have made before when map() just returned a list.  In
most cases I didn't have to make these optimizations manually because
the code is written in Cython. It's true that when a user called map()
previously some opportunities for optimization were already lost, but
now it's even worse because I have to treat a simple map of a list on
par with the necessarily slower arbitrary iterator case, when
technically-speaking there is no reason that has to be the case.
Cython could even handle that case automatically as well by turning a
map(<some_C_function_wrapped_by_cython>, <a_list>) into something
like:

    list = map.iter[0];
    for (idx=0; idx < PyList_Length(list); idx++) {
        wrapped_c_function(PyList_GET_ITEM(list, idx);
    }

> If the latter, it's all about Sage, ergo you can rebind map to mean what
> you expect it to mean. Take your pick.

I'm' still not sure what makes you think one can just blithely replace
a builtin with something that doesn't work how all other Python
libraries expect that builtin to work.  At best I could subclass map()
and add this functionality but now you're adding at least three
pointers to every map() that are not necessary since the information
is already there in the C struct.  For most cases this isn't too bad
in terms of overhead but consider cases (which I've seen plenty of),
like:

list_of_lists = [map(int, x) for x in list_of_lists]

Now the user who previously expected to have a list of lists has a
list of maps.  It's already bad enough that each map holds a pointer
to a function but I wouldn't want to make that worse.

Anyways, I'd love to get off the topic of Sage and just ask why you
would object to useful introspection capabilities?  I don't even care
if it were CPython-specific.