Python and STL efficiency

[Ames Andreas]

> -----Original Message-----
> From: python-list-bounces+andreas.ames=comergo.com at python.org 
> [mailto:python-list-bounces+andreas.ames=comergo.com at python.or
> g] On Behalf Of Ray
> Sent: Wednesday, August 23, 2006 4:28 PM
> Subject: Re: Python and STL efficiency
> 
> 
> Tim N. van der Leeuw wrote:
> > With the nr of loops corrected, Python on my laptop 
> performs worse than
> > C++ under all circumstances, by a factor of about 2:
> 
> *Phew*
> 
> Great to know that my model of how the world works is still correct!
> (at least in relation to Python and C++!) :)

If you're really eager to see python succeed with this 'test' or 'benchmark', if you will, you can look at the results with VC7.1.  It clearly shows that much depends on the STL implementation, you use.

I use the implementations posted by Maric Michaud in http://mail.python.org/pipermail/python-list/2006-August/357593.html.  The tests, for which I appended 'with hash', use a hash_set instead of a plain set.

1) vc71 with dinkumware STL (which is vc's default STL):

print_occurence_of_strings
What do you know?
chicken crosses road
fool
so long...
print_occurence_of_unique_strings
What do you know?
chicken crosses road
fool
so long...
print_occurence_of_unique_strings_compared_by_address
What do you know?
chicken crosses road
fool
so long...
print_occurence_of_strings_with_hash
What do you know?
chicken crosses road
fool
so long...
print_occurence_of_unique_strings_with_hash
What do you know?
chicken crosses road
fool
so long...
strings : 10.921                         <---- Oops ...
unique strings : 1.047
compared by address : 0.328
strings with hash : 11.484               <---- Oooops ...
unique strings with hash : 1.016

2) vc7.1 with STLport 5.02:

print_occurence_of_strings
What do you know?
chicken crosses road
fool
so long...
print_occurence_of_unique_strings
What do you know?
chicken crosses road
fool
so long...
print_occurence_of_unique_strings_compared_by_address
What do you know?
chicken crosses road
fool
so long...
print_occurence_of_strings_with_hash            <--- reorders
fool
What do you know?
chicken crosses road
so long...
print_occurence_of_unique_strings_with_hash     <--- reorders
so long...
What do you know?
chicken crosses road
fool
strings : 2.156
unique strings : 0.953
compared by address : 0.187
strings with hash : 2.078
unique strings with hash : 0.922

This seems to substantiate this post http://sourceforge.net/mailarchive/forum.php?thread_id=30201462&forum_id=46178.

3) python 4.2 (python test.py; interestingly this is faster than with -OO)

so long...
What do you know
fool
chicken crosses road
Elapsed: 2.278332 seconds

It seems the unification of the strings via hash is not significantly better (compared to memcpy or whatever is used).

The results for the dinkumware STL are dominated by the sheer number of calls to HeapAlloc/HeapFree and to EnterCriticalSection/LeaveCriticalSection (I compiled with multithreading and linked statically).  The python version won't be concerned with the latter, I guess (although just by importing 'threading' performance gets about 3.5% worse).

STLport uses for example InterlockedExchange (obviously in the default allocator) and several orders of magnitude fewer calls to the heap allocation API.

It all boils down to the fact that I can write C++ or even assembler programs that perform worse (by any degree I want) than a functionally equivalent python program.  That's a trivial insight and it certainly doesn't provide evidence that python 'is faster' than c++/assembler.  Even if that's just because it *isn't* and can't be.

I always wonder where this rage comes from to prove oneself or whomever that python is the best programming language in any regard as if it hadn't enough other advantages.  It is certainly not the fastest/most memory efficient tool out there.  One of the biggest advantages of python to me is that, even if I run into a hotspot, the transition from python to C/C++ is so easy.  That means that in spite of its warts (GIL, memory efficiency ...) I don't easily come to a dead end using python.  This fact lets me use python's great strengths (like programmer efficiency etc.) without tossing and turning sleeplessly in my bed.

cheers,

aa

-- 
Andreas Ames | Programmer | Comergo GmbH |
Voice:  +49 69 7505 3213 | ames AT avaya DOT com