[IronPython] CPython 25x faster than IronPython on this code

[Jim Hugunin]

I can make some fairly small changes to this test and find that
IronPython is about 1.3x faster than CPython on roughly the same code.
The fact that I need to make changes is obviously not a good thing.
However, the changes that I made help show the kinds of rough edges we
have left before getting to IronPython 1.0.  Here are the four issues I
found while playing with your program.

1. We're rapidly increasing the number of standard CPython modules that
run in IronPython, but we're probably not making it clear enough to
folks how to use them.  I use the following tiny site.py file so that I
can import from the standard CPython Lib:
---- site.py ----
import sys
sys.path.append('c://Python24//Lib')
-----------------
The random module is working in the IronPython 0.9.2 and later, so if
you drop in this site.py, your CPy version will run just fine in
IronPython.

2. The first thing you'll notice if you do that is that the numbers look
horrible.  IronPython appears to be about 8 orders of magnitude slower
than CPython.  This is a stupid and outrageous bug in time.time which
we'll fix where it's reporting 100 nanosecond ticks as seconds.  For
now, you should use time.clock instead which will work correctly on both
systems.

3. The next issue is that the max builtin in IronPython is horribly
slow.  It's more than 25x slower than the max in CPython.  This is the
primary cause of your slow-down.  We need to fix this.  The reason for
the slow-down is that max has an overload which accepts a params
object[] (the Python equivalent of *args) and we wind up going through
an extremely slow method dispatch path to figure out which overload of
max to call.  This is more work than fixing time.time, but not that
much.  The work-around for now is to define a fast_max function in
Python.

4. After fixing the perf issue with max, I find that IronPython is still
about 1.6x slower than CPython running this test.  This final
performance hit turns out to come from random.random.  This is another
case of a slow path where IronPython doesn't do so well looking up
members on external modules.  CPython isn't lightning fast at this
either, and you can speed up both implementations by putting
random.random into a variable called rand.  We'll be fixing this too,
but these kinds of slow-downs are obviously of a lower priority than the
jaw-dropping 25x one you found for max.

After the changes above, IronPython on my box is about 1.3x faster than
CPython.

#1 and 2 are simple fixes.
#3 and 4 will require bigger fixes, but these are the kinds of things
we're addressing for IronPython 1.0.

Here's my final version of the code with the changes mentioned above,
plus a few small changes to make the timing and reporting a little
friendlier by running for more iterations and collecting the stop time
before printing anything.
-----------------------------
import time
import random

def fast_max(x,y):
    if x > y: return x
    else: return y

kTotalReps = 8000
kNumReps = 100

start = time.clock()
blah = 0
highnum = 0.0
rand = random.random
for i in range(kTotalReps):
	myt = []
	for j in range(kNumReps):
		toadd = rand()
		myt.append(toadd)
		highnum = fast_max(highnum, toadd)
		blah = blah + 1

end = time.clock()
print highnum
print '%.3f seconds for %d iterations' % (end-start, blah)
-------------------------------

Thanks for the entertaining example - Jim

-----Original Message-----
From: users-bounces at lists.ironpython.com
[mailto:users-bounces at lists.ironpython.com] On Behalf Of Chris Trimble
Sent: Wednesday, October 12, 2005 9:52 AM
To: users at lists.ironpython.com
Subject: [IronPython] CPython 25x faster than IronPython on this code

This same program in interpreted C# or Java (as opposed to JITed) runs
about the same speed as CPython (~110ms on my machine).  IronPython
takes almost 3 seconds in the inner loops.  Any ideas why?

IronPy version:
------------------------
import System

kTotalReps = 800
kNumReps = 100

start = System.Environment.TickCount
blah = 0
randgen = System.Random()
highnum = 0.0
for i in range(kTotalReps):
	myt = []
	for j in range(kNumReps):
		toadd = randgen.NextDouble()
		myt.append(toadd)
		highnum = max(highnum, toadd)
		blah = blah + 1

print blah
print highnum
print (System.Environment.TickCount-start)
----------------------


CPy version:
----------------------
import time
import random

kTotalReps = 800
kNumReps = 100

start = time.time()
blah = 0
highnum = 0.0
for i in range(kTotalReps):
	myt = []
	for j in range(kNumReps):
		toadd = random.random()
		myt.append(toadd)
		highnum = max(highnum, toadd)
		blah = blah + 1

print blah
print highnum
print (time.time()-start)*1000
-----------------------
_______________________________________________
users mailing list
users at lists.ironpython.com
http://lists.ironpython.com/listinfo.cgi/users-ironpython.com