Pre-pep discussion material: in-place equivalents to map and filter

Arthur Havlicek arthurhavlicek at gmail.com
Thu Nov 3 16:29:06 EDT 2016


I understand that, the cost of change is such that it's very unlikely
something like this ever goes into Python, but I feel like the interest of
the proposition is being underestimated here, that's why I'm going to argue
a few points and give a bit more context as needed.

> While mapping and filtering are common operations, I'm not sure mapping
> and filtering and then reassigning back to the original sequence is
> especially common.

It depends of your context. On the last 3 months, I stumbled across this at
least 3 times, which is 3 times more than I used a lambda or a metaclass or
a decorator or other fancy language feature that we simply avoid whenever
possible. It also happened to my collegue. I remember these examples
because we had a bit of humour about how nice can be inlined ternaries
inside comprehensions, but I could be missing a lot more.

The reason I'm being especially impacted by this is because I am maintainer
of a decent-sized Python application (~50-100K lines of code) that
extensively uses lists and dictionaries. We value "low level" data
manipulation and efficiency a lot more than complex, non-obvious
constructs. In other contexts, it may be very different. Note that my
context is only relevant for illustration here, I don't expect a feature to
save me since we are currently shipping to Centos 6 and thus will not see
the light of Python 3.7 in the next 10 years (optimistic estimation).

> Arthur, I would suggest looking at what numpy and pandas do.

In my example context, their benefits can't apply, because I'm not going to
rewrite code that uses lists for them to uses np.array instead for example.
Although the performance boost is likely to be bigger if used properly, I
would have prefered a lesser boost that comes for (almost) free.

Like most Python programmers, I'm not in the case of needing a performance
boost very bad, but that does not mean I disregard performance entirely.
The need of performance is not so binary that it either don't matter at all
or is enough to motivate a rewrite.

> To my eyes, this proposed syntax is completely foreign

I must admit I don't have much imagination for syntax proposals...all that
mattered to me here was to make it clear you are doing an in-place
modification. Feel free to completely ignore that part. Any proposal
welcomed of course.
About Readability & Redundancy

I have misused the terms here, but I wasn't expecting so much nitpicking. I
should have used the term maintenability, because that one is bland and
subjective enough that nobody would have noticed :D

How about "I find that cooler." Good enough ?

In a less sarcastic tone:

What I truely meant here is that when you contain the behavior of your code
inside a specific keyword or syntax, you are making your intentions clear
to the reader. It may be harder for him to gain access to the knowledge in
the first place, but makes it easier over time.

Famous example:

When you learned programming, you may have had no idea what "+=" was doing,
but now you do, you probably rate "a += 2" syntax to be much better than "a
= a + 2". You make the economy of a token, but more important, you make
your intentions clearer because "+=" rings a bell, wihle "=" is a more
generic syntax with a broader meaning.

> So map() here is less than a factor of two slower. I wouldn't call
> that "especially bad" -- often times, a factor of two is not important.
> What really hurts is O(N**2) performance, or worse.

When you evaluate your application bottleneck or your average daily
algorithmic evaluation, perhaps. When regarding language core features we
are not on the same scale. If a language X is 2 times faster than a
language Y to do the same task, that's a huge seller, and is of real
importance.



More information about the Python-list mailing list