Python too slow for real world

[Christian Tismer]

Arne Mueller wrote:

> def read_write(i, o, exclude):
>     name = compile('^>(\S+)') # regex to fetch the identifier
>     l = i.readline()
>     while l:
>         if l[0] == '>': # are we in new dataset?
>             m = name.search(l)
>             if m and exclude.has_key(m.group(1)): # excluding current
> dataset?
>                 l = i.readline()
>                 while l and l[0] != '>':  # skip this dataset
>                     l = i.readline()
>                     pass
>         o.write(l)
>         l = i.readline()
> 
> f = open('my_very_big_data_file','r') # datafile with ~300000 records
> read_write(f, stdout, {}) # for a simple test I don't exclude anything!
> 
> It took 503.90 sec on a SGI Power Challange (R10000 CPU). An appropiate
> perl script does the same job in 32 sec (Same method, same loop
> structure)!

Without changing the program structure, I could make it run about 3 or 4
times faster by using string.split instead of a regex here.
To get more speed, one would have to do more.

Summarizing: Stay with the Perl code, if you need it so fast. 
Perl is made for this real world low-level stuff. 
Python is for the real world high level stuff.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer at appliedbiometrics.com>
Applied Biometrics GmbH      :     Have a break! Take a ride on Python's
Kaiserin-Augusta-Allee 101   :    *Starship* http://starship.python.net
10553 Berlin                 :     PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint       E182 71C7 1A9D 66E9 9D15  D3CC D4D7 93E2 1FAE F6DF
     we're tired of banana software - shipped green, ripens at home