Debugging reason for python running unreasonably slow when adding numbers

David Raymond David.Raymond at tomtom.com
Wed Mar 15 10:24:34 EDT 2023


> Or use the sum() builtin rather than reduce(), which was
> *deliberately* removed from the builtins. The fact that you can get
> sum() without importing, but have to go and reach for functools to get
> reduce(), is a hint that you probably shouldn't use reduce when sum
> will work.

Out of curiosity I tried a couple variations and am a little confused by the results. Maybe I'm having a brain fart and am missing something obvious?

Each of these was run with the same "data" and "perceptrons" values to keep that fair.
Times are averages over 150 iterations like the original.
The only thing changed in the trainPerceptron function was how to calculate sum_


Original:
sum_ = 0
for key in input:
    v = weights[key]
    sum_ += v
418ms

The reduce version:
sum_ = reduce(lambda acc, key: acc + weights[key], input)
758ms

Getting rid of the assignment to v in the original version:
sum_ = 0
for key in input:
    sum_ += weights[key]
380ms

But then using sum seems to be slower

sum with generator expression:
sum_ = sum(weights[key] for key in input)
638ms

sum with list comprehension:
sum_ = sum([weights[key] for key in input])
496ms

math.fsum with generator expression:
sum_ = math.fsum(weights[key] for key in input)
618ms

math.fsum with list comprehension:
sum_ = math.fsum([weights[key] for key in input])
480ms


I'm not quite sure why the built-in sum functions are slower than the for loop,
or why they're slower with the generator expression than with the list comprehension.


More information about the Python-list mailing list