python performance

[Padraig Brady]

Terry Reedy wrote:
> "Padraig Brady" <padraig at linux.ie> wrote in message
> news:3D84E66F.6020701 at linux.ie...
> 
>>I was wondering about the performance characteristics
>>of python and ran a simple test. The 2 programs
>>below are functionally equivalent and just read
>>in fields into a list. file.fields contains
>>720 fields in each of 405 lines of the form of
>>repeating: oneoneone twotwo "thre e three"
>>
>>The time to run is shown above each program
>>from which I've inferred the following:
>>
>>1. The function call version is (6.3%) faster because
>>    the cumulative cost of parsing the simpler expressions
>>    and function call overhead is smaller than parsing the
>>    1 single complex expression?
> 
> 
> Calling functions adds overhead.  Running code within functions
> reduces overhead for object access.

Ah OK, this because it has to search a smaller namespace?
Wouldn't compiling overcome this? I suppose python is too
dynamic.

> See end for third version to
> test.

results below.
thanks!
Pádraig.

>>Or the function is parsed
>>    only once and doesn't have to be reparsed. This would
>>    suggest that top level code is parsed for each iteration?
> 
> 
> No.  All code is parsed only once.  See above and below.
> 
> 
>>2. Anyway I thought that parsing affects would be removed by doing
> 
> the
> 
>>    parsing only once, i.e. compiling the code to .pyc (I used
>>    py_compile.compile()). However this makes no difference at all?
>>    Surely compiling is not just for code obfuscation.
> 
> 
> No.  The one-time parsing of 10 lines of code is extremely fast.  If
> it were parsed over again for each line, then you would notice the
> parsing time.
> 
> 
> 
>>Note I did do the test several times and averaged the results.
>>----------------------------
>>2.514s
>>----------------------------
>>#!/usr/bin/env python2.2
>>import re
>>
>>reFieldFinder = re.compile('[^ "]+|"[^"]+"') #unquoted|quoted
>>def getFields(line):
>>     fields = reFieldFinder.findall(line)
>>     return [field.replace('"', '') for field in fields]
>>
>>for line in open("file.fields").readlines():
>>     listLine = getFields(line[:-1])
>>
>>----------------------------
>>2.672s
>>----------------------------
>>#!/usr/bin/env python2.2
>>import re
>>
>>reFieldFinder = re.compile('[^ "]+|"[^"]+"') #unquoted|quoted
>>for line in open("file.fields").readlines():
>>     listLine = [field.replace('"', '') for field in
>>reFieldFinder.findall(line[:-1])]
> 
> 
> Try one more test to run at function speed without repeated
> getsfields():
> 

----------------------------
2.498s
----------------------------
> def mytest:
>     reFieldFinder = re.compile('[^ "]+|"[^"]+"') #unquoted|quoted
>     for line in open("file.fields").readlines():
>         listLine = [field.replace('"', '') for field in
>     reFieldFinder.findall(line[:-1])]
> 
> mytest()