benchmark

[Chris Mellon]

On Thu, Aug 7, 2008 at 9:09 AM, Steven D'Aprano
<steve at remove-this-cybersource.com.au> wrote:
> On Thu, 07 Aug 2008 06:12:04 -0700, alex23 wrote:
>
>> On Aug 7, 8:08 pm, M8R-n7v... at mailinator.com wrote:
>>> Really how silly can it be when you suggest someone is taking a
>>> position and tweaking the benchmarks to prove a point [...]
>>
>> I certainly didn't intend to suggest that you had tweaked -anything- to
>> prove your point.
>>
>> I do, however, think there is little value in slavishly implementing the
>> same algorithm in different languages. To constrain a dynamic language
>> by what can be achieved in a static language seemed like such an
>> -amazingly- artificial constraint to me.
>
> I don't know about that... it can be very useful to (say) demonstrate
> that Lisp-style lists are fast in Lisp, and slow in Python. Or that
> try...except is fast in Python, and slow in Java.
>

That's true, but note that the original post doesn't attempt to draw
any conclusions about what's fast or slow from the benchmark, which is
one reason why it's a poor example of benchmarking.

> And if your aim is to compare languages, then it's only fair to keep the
> algorithm constant. Imagine how we would holler and shout if the
> benchmark compared Ruby using Quicksort and Python using Bubblesort.
>

That's definitely true, and (for example) the Alioth benchmarks are
intended to benchmark specific algorithms for comparisons sake.

> I guess what some of us are complaining about is that the algorithm
> chosen doesn't suit Python's execution model very well, and hence Python
> is slow. If the algorithm chosen had suited Python, and hence Python came
> up looking really fast, we'd be ecstatic. How about that, hey? *wink*
>

The "best" way to implement this problem, as bitfield manipulation,
would actually show python in even worse light. I suspect the main
thing that this benchmark is actually testing is loop overhead, and
secondarily object allocation speed. Python is pretty slow in the
former and reasonable in the latter, so I don't find the results very
surprising at all.

> ...
>> Honestly, performance benchmarks seem to be the dick size comparison of
>> programming languages.
>
> I can't disagree with that one bit.
>

As with genitals, the important thing about benchmark comparison is
what you're going to do with the results.

>
> --
> Steven
> --
> http://mail.python.org/mailman/listinfo/python-list
>