Question regarding a recent article on informit.com

[Alex Martelli]

Peter Hansen wrote:

> Ben S wrote:
>> 
>> Are there any good tutorials of examples of list comprehension use? In
>> particular, I'm interested in how to replicate the functionality of
>> reduce? Map and filter seem easy to do, though.
> 
> Doesn't reduce return a non-list (or at least, isn't it the case that
> it doesn't necessarily return a list the way map and filter do)?

Exactly: reduce CAN return a list (when a list is what's returned by
the function that is its first argument) but that rarely happens.


> An *abuse* of a list comprehension, or a list comprehension in
> conjuction with something else, could probably replicate reduce(),
> but why bother?  reduce() isn't really like map and filter.

Absolutely right.  More often than not, the right replacement for
reduce is just a simple loop.  Consider...:

$ python timeit.py 'reduce(lambda x, y: x+y, range(100))'
10000 loops, time: 81.392 usec
$ python timeit.py -s 'import operator' 'reduce(operator.add, range(100))'
10000 loops, time: 36.212 usec
$ python timeit.py 'x=0' 'for y in range(100): x += y'
10000 loops, time: 32.864 usec

These numbers are very repeatable on my machine (a good, if dated,
Athlon + DDR box) with Python 2.3 (built right out of CVS).

So why would you pay the 10% performance price for using reduce
(assuming you do NOT use a lambda in it -- otherwise, 150% price
or so...)?  Do you REALLY think it makes things so much clearer
and more obvious as to be worth slowing your code down?  In my
opinion, it makes things LESS clear and obvious for most readers.


So, I would normally prefer the simple, obvious loop -- even in
the "sum all of these numbers" case, which is reduce's main use
case.  I think Python would be a marginally but definitely better
language if it had no reduce built-in: it's just one more thing
that the Python newbie has to learn, basically just in order to
be able to understand existing code that uses it -- reduce might
migrate into some "legacy built-ins existing solely for purposes
of backwards compatibility" ("from __past__ import..." ?-) AND
avoid certain "attractive nuisances" such as...:

$ python timeit.py -s 'import operator' 'reduce(operator.add, 
map(str,range(1000)))'
100 loops, time: 2612.000 usec
$ python timeit.py -s 'import operator' '"".join(map(str,range(1000)))'
1000 loops, time: 1430.569 usec

where the performance penalty for using reduce rather than doing
things the right way can reach and exceed 80% (or higher costs too,
for long-enough lists of strings...) -- not necessarily "in the
noise" any more...!


Alex