[Python-ideas] Fast sum() for non-numbers

[Sergey]

On Jul 11, 2013 Andrew Barnert wrote:

>>> sum is not the obvious way to concatenate sequences today, and my
>>> preferred way is for that to stay true. So, I'm: [...]
>> 
>> You're saying what IS NOT your preferred way. But I'm asking what IS
>> your preferred way.
> 
> You just quoted it. My preferred way to handle this is what we
> already do: don't encourage people to misuse sum for concatenating
> sequences, encourage them to use something appropriate.

I don't understand it. It makes no sense to me. Do you like having
many broken tools? E.g. would you liked if someone added __mult__
to set()s, but made it O(N*N*N) so that people would not used it too
often?

Anyway, I've got your point. You want sum() to be O(N) for numbers
and some rare/custom containers, but want it to stay O(N*N) for
the most popular container types for some reasons, that are too
hard for me to understand.

>> Do you prefer to have a slow sum in python and people asking why
>> it's slow forever? Do you see that as a best possible case for
>> python?

> Yes. Given that it is impossible to make sum fast for all
> collection types

Technically it may be possible. As if technically it's possible
to have no wars on Earth. It requires a great cooperation of many
people, so it's not probable, but possible. So what? It's (kind of)
impossible to make python fast for all programs, but it does not
mean that python should not be fast for some programs,

Same applies to sum(): even if it's impossible to make if fast for
all collection types, it does not mean that it should not be fast for
some of them, e.g. lists and tuples.

After all it's quite easy to make it fast for most (if not all)
commonly used cases.

> Also, It's less surprising this way, not more. Today, people only
> have to learn one thing: Don't use sum on collections. That's much
> easier than having to learn a complex mishmash like: Don't use sum on
> immutable collections, except for tuple, and also don't use it on
> some mutable collections, but it's hard to characterize exactly
> which, and also don't use it on things that are iterable but that
> you don't want to treat as sequences, and...

It's much easier to just learn: don't use sum().

And it's even easier to learn: use sum(), because you don't have to
learn that. ;)

> Finally, I've ignored your requests for examples because in every
> case you've already been given examples and haven't dealt with any
> of them.

Oh, really? You said, I can't make sum fast for strings and tuples,
so I did that and showed a patch. Then you said that I can't make sum
fast for linked lists, so I suggested how to do that. You did not
liked my linked lists, so I explained how you can do that for your
cons-lists. You did not liked my explanation (and said some weird
things about thread safety) so I showed you two code samples, both
using your cons-lists and sum().

You also said that I can't make tuples __add__ faster without patching
sum() so explained how you can do that (you never answered whether
you like such a patch and whether you would agree to write it).
I even wrote a simple fasttuple proof of concept [1].

What examples I haven't dealt with? ;)

> If I want to concatenate cons lists, chain does it in linear time
> your design does not;

Quite the opposite, my design would give you a list, while chain
won't even give you a correct list, since `next` elements of that
"list" would not point to correct locations. BTW, chain would
not work for your list, because it's not iterable by default,
is it? And even if you implement __iter__ for it, how are you
going to handle modifications of you list while you iterate it?
You don't have those problems with sum().

Hm, that basically marks chain() as error-prone replacement for sum,
that is sometimes harder to implement support for.

> instead of answering that, you just keep arguing that you can sum
> a different kind of linked list in linear time. That doesn't even
> approach answering the objection.

If the "objection" was "you can't make it fast for linked lists" then
"you can" is the exact answer to that objection. :)

-- 
[1] http://bugs.python.org/file30917/fasttuple.py