Interesting speed benchmark

[Greg Copeland]

xyzmats at laplaza.org (Mats Wichmann) writes:

> That's because Java leaves that to the gabage collector. Which is
> quite sophisticated but to us mere mortals often presents the image of
> "yeah, yeah, I'll get around to it. Maybe tomorrow."  But this sounds
> like a place where it would do okay, because the whole wonky chain of
> 10000 objects can be reaped by the gc.
> 
> Nonetheless, I'm not convinced that's what's going on here.
> Mats Wichmann

Actually, I have several friends that live in Java.  For a very long time,
they were always first to point out how fast Java was at performing a task
in a benchmark (home-grown or otherwise).  I was always first to point out
that MOST benchmarks do not run long enough for Java's gc to kick in.  Once
Java's gc kicks in, you often see a *HUGE* correction in performance.  What
does this mean?  Well, it means that Java's gc implementation is AWESOME
for benchmarks.  It also means that anything you benchmark is Java is usually
a worthless indicator of real-world performance if it's going to be in a
long-running application (that is, runs long enough for the gc to start
running).  My friends have finally acknowledged this and now rarely show me
benchmarks unless they know what the JVM's memory settings are  and can
at least guess at if gc is running.  Now that they do this, most of their
boasts have been removed from their sails.  The other metric that often
gets ignored is the huge JVM start up time.  Likewise, people tend to
forget allow python to byte-compile before they measure.  While these may
not be the single attribute that is making the difference here, it is worth
pointing out that it certainly may be a factor.

Please keep in mind that I'm not trying to bash Java or say that Python
is better.  Rather, I'm only saying that when you are comparing Java to
ANY other language, you must make sure you really are comparing apples to
apples.  Likewise, I find that most Java programmers don't understand enough
of the internals to truly be in a position to offer any type of meaningful
comparison because the simply don't understand what's going on under the
covers.  Again, this is not a whack at Java or Java programmers...just some
observations gleaned from my experiences.


-- 
Greg Copeland, Principal Consultant
Copeland Computer Consulting
--------------------------------------------------
PGP/GPG Key at http://www.keyserver.net
DE5E 6F1D 0B51 6758 A5D7  7DFE D785 A386 BD11 4FCD
--------------------------------------------------