[New-bugs-announce] [issue11258] ctypes: Speed up find_library() on Linux by 500%

[Jonas H.]

New submission from Jonas H. <jonas at lophus.org>:

(This applies to all versions of Python I investigated, although the attached patch is for Python 2.7)

I wondered why `import uuid` took so long, so I did some profiling.

It turns out that `find_library` wastes at lot of time because of this crazy regular expression in `_findSoname_ldconfig`.

A quick look at the ldconfig source (namely, the print_cache routine which is invoked when you call `ldconfig -p`, http://sourceware.org/git/?p=glibc.git;a=blob;f=elf/cache.c#l127) confirmed my suspicion that the ldconfig's output could easily be parsed without such a regex monster.

I attached two patches that fix this problem. Choose one! ;-)

The ctypes tests pass with my fixes, and here comes some benchmarking:

$ cat benchmark_ctypes.py 
from ctypes.util import find_library
for i in xrange(10):
  for lib in ['mm', 'c', 'bz2', 'uuid']:
    find_library(lib)

# Current implementation
$ time python benchmark_ctypes.py 
real    0m11.813s
...
$ time python -c 'import uuid'
real    0m0.625s
...

# With my patch applied
$ cp /tmp/ctypesutil.py ctypes/util.py
$ time python benchmark_ctypes.py 
real    0m1.785s
...
$ time python -c 'import uuid'
real    0m0.182s
...

----------
assignee: theller
components: ctypes
files: faster-find-library1.diff
keywords: patch
messages: 128910
nosy: jonash, theller
priority: normal
severity: normal
status: open
title: ctypes: Speed up find_library() on Linux by 500%
type: performance
versions: Python 2.5, Python 2.6, Python 2.7, Python 3.1, Python 3.2, Python 3.3
Added file: http://bugs.python.org/file20808/faster-find-library1.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11258>
_______________________________________