lists, performance..

[John Hunter]

>>>>> "gabor" == gabor  <gabor at realtime.sk> writes:

    gabor> On Thu, 2002-10-31 at 15:26, holger krekel wrote:
    >> the fastest method i know for your problem is:
    >> 
    >> >>> a,b=[1,2,3],[2,3,4] >>> n=[] >>> filter(n.append, a) [] >>>
    >> filter(n.append, b) [] >>> n [1, 2, 3, 4, 5, 6]
    >> >>>
    >> 
    >> It's very fast because 'list.append' and 'filter' both have
    >> c-level implementations.
    >> 
    >> > but maybe this is too slow.. or isn't? ideally i'd like to
    >> have > something like reserve in c++....  > so let's say i have
    >> a list which len() is 30. now i want to insert >
    >> 20elements... wouldn't it be faster to somehow resize the list
    >> directly > to 50, and then add the elements?  > i'm worried
    >> about how many times would the list resize itself to be able >
    >> to contain the additional 20elements if i would add them
    >> one-by-one
    >> 
    >> time the above solution and see if it's fast enough for you.

    gabor> btw. is there a timing function in python?  like 'time' in
    gabor> unix..

the time module will help

In [1]: import time 

In [2]: time.time?
Type:		builtin_function_or_method
Base Class:	<type 'builtin_function_or_method'>
String Form:	<built-in function time>
Namespace:	Currently not defined in user session.
Docstring:
    time() -> floating point number
    
    Return the current time in seconds since the Epoch.
    Fractions of a second may be present if the system clock provides them.


In [3]: time.clock? 
Type:		builtin_function_or_method
Base Class:	<type 'builtin_function_or_method'>
String Form:	<built-in function clock>
Namespace:	Currently not defined in user session.
Docstring:
    clock() -> floating point number
    
    Return the CPU time or real time since the start of the process or
    since the first call to clock().  This has as much precision as
    the system records .

Here is a little utility class I sometimes use for timing code


import time
class Timer:
    """Record events and how long it takes.  The time between event is
    recorded"""

    def __init__(self):
        self.events = []
        self.last = time.clock()

    def event(self, s):
        now = time.clock()
        self.events.append( (s, now - self.last) )
        self.last = time.clock()

    def report(self):
        if len(self.events)==0:
            print "No events recorded"
        else:
            print '%s: %1.3fs' % self.events[-1]
        
        
    def __repr__(self):
        s = ''
        for event in self.events:
            s += '%s: %1.3fs\n' % event
        return s


You can use it like:

t = Timer()

some_code_that_takes_a_while()
t.event('Ran some code')

some_other_code_that_takes_a_while()
t.event('Ran some other code')

print t


This method is really only useful if you are reasonably sure that the
time it takes to make the function calls in the timer class are
negligible compared to the time it takes to execute the code you are
timing.  But it's easy....

Tim Peters discusses timing in his introduction to the algorithms
chapter in The Python Cookbook, where he warns about lots of little
gotchas in timing code.

John Hunter