[IronPython] Performance of IronPython 2 Beta 4 and IronPython 1

[Michael Foord]

It looks like 2-3 seconds of the 5-6 seconds slowdown in dependency 
analysis is due to the way I resolved a weird cyclic import problem (one 
that we didn't have in IP1 but due to the spaghetti like imports in that 
part of our code I can't reproduce on its own with IP2). I moved an 
import into the body of a function that is used heavily - moving that 
and resolving the problem another way saves about 2-3 seconds.

In IronPython (1 & 2) reimporting a module that has already been 
imported is much more expensive than in CPython.

That now accounts for almost all of the extra time in the dependency 
analysis - now onto tracking down what looks like overhead in our main 
execution loop. (Executing the spreadsheet code itself is no slower in 
IP2 but the framework around it seems to be suffering.)

Michael

Michael Foord wrote:
> Hello all,
>
> I've ported Resolver One to run on IronPython 2 Beta 4 to check for 
> any potential problems (we will only do a *proper* port once IP 2 is 
> out of beta).
>
> The basic porting was straightforward and several bugs have been fixed 
> since IP 2 B3 - many thanks to the IronPython team.
>
> The good news is that Resolver One is only 30-50% slower than Resolver 
> One on IronPython 1! (It was 300 - 400% slower on top of IP 2 B3.) 
> Resolver One is fairly heavily optimised around the performance 
> hotspots of IronPython 1, so we expect to have to do a fair bit of 
> profiling and refactoring to readjust to the performance profile of IP 2.
>
> Having said that, there are a few oddities (and the areas that slow 
> down vary tremendously depending on which spreadsheet we use to 
> benchmark it - making it fairly difficult to track down the hotspots).
>
> We have one particular phase of spreadsheet calculation that takes 
> 0.4seconds on IP1 and around 6 seconds on IP2, so I have been doing 
> some micro-benchmarking to try and identify the hotspot. I've 
> certainly found part of the problem.
>
> For those that are interested I've attached the very basic 
> microbenchmarks I've been using. The nice thing is that in *general* 
> IP2 does outperform IP1.
>
> The results that stand out in the other direction are:
>
> Using sets with custom classes (that define '__eq__', '__ne__' and 
> '__hash__') seems to be 6 times slower in IronPython 2.
>
> Adding lists together is about 50% slower.
>
> Defining functions seems to be 25% slower and defining old style 
> classes about 33% slower. (Creating instances of new style classes is 
> massively faster though - thanks!)
>
> The code I used to test sets (sets2.py) is as follows:
>
> from System import DateTime
>
> class Thing(object):
>    def __init__(self, val):
>        self.val = val
>      def __eq__(self, other):
>        return self.val == other.val
>
>    def __neq__(self):
>        return not self.__eq__(other)
>          def __hash__(self):
>        return hash(self.val)
>             def test(s):
>    a = set()
>    for i in xrange(100000):
>        a.add(Thing(i))
>        a.add(Thing(i+1))
>        Thing(i) in a
>        Thing(i+2) in a
>    return (DateTime.Now -s).TotalMilliseconds
>   s = DateTime.Now
> print test(s)
>
>
> Interestingly the time taken is exactly the same if I remove the 
> definition of '__hash__'.
>
> The full set of results below:
>
> Results in milliseconds with a granularity of about 15ms and so an 
> accuracy of +/- ~60ms.
> All testing with 10 000 000 operations unless otherwise stated.
>
> Empty loop (overhead):
>    IP1: 421.9
>    IP2: 438
>   Create instance newstyle:
>    IP1: 20360
>    IP2: 1109
>   Create instance oldstyle:
>    IP1: 3766
>    IP2: 3359
>   Function call:
>    IP1: 937
>    IP2: 906
>   Create function: 25% slower
>    IP1: 2828
>    IP2: 3640
>   Define newstyle (1 000 000):
>    IP1: 42047
>    IP2: 20484
>   Define oldstyle (1 000 000): 33% slower
>    IP1: 1781
>    IP2: 2671
>
> Comparing (== and !=):
>    IP1: 278597
>    IP2: 117662
>   Sets (with numbers):
>    IP1: 37095
>    IP2: 30860
>
> Lists (10 000): 50% slower
>    IP1: 10422
>    IP2: 16109
>
> Recursion (10 000):
>    IP1: 1125
>    IP2: 1000
>  
> Sets2 (100 000): 600% slower
>    IP1: 4984
>    IP2: 30547
>
>
> I'll be doing more as the 600% slow down for sets and the 50% slow 
> down for lists accounts for some of the dependency analysis problem 
> but not all of it.
>
> Many Thanks
>
> Michael Foord
> -- 
> http://www.resolversystems.com
> http://www.ironpythoninaction.com
>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Users mailing list
> Users at lists.ironpython.com
> http://lists.ironpython.com/listinfo.cgi/users-ironpython.com


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/
http://www.trypython.org/
http://www.ironpython.info/
http://www.resolverhacks.net/
http://www.theotherdelia.co.uk/