[New-bugs-announce] [issue10160] operator.attrgetter slower than lambda after adding dotted names ability

[Χρήστος Γεωργίου (Christos Georgiou)]

New submission from Χρήστος Γεωργίου (Christos Georgiou) <tzot at users.sourceforge.net>:

(Discovered in that StackOverflow answer: http://stackoverflow.com/questions/3940518/3942509#3942509 ; check the comments too)

operator.attrgetter in its simplest form (i.e. with a single non-dotted name) needs more time to execute than an equivalent lambda expression.

Attached file so3940518.py runs a simple benchmark comparing: a list comprehension of plain attribute access; attrgetter; and lambda. I will append sample benchmark times at the end of the comment.

Browsing Modules/operator.c, I noticed that the dotted_getattr function was using PyUnicode_Check and (possibly) splitting on dots on *every* call of the attrgetter, which I thought to be most inefficient.

I changed the py3k-daily-snapshot source to make the PyUnicode_Check calls in the attrgetter_new function; also, I modified the algorithm to pre-parse the operator.attrgetter functions for possible dots in the names, in order for the dotted_getattr function to become speedier.

The only “drawback” is that now operator.attrgetter raises a TypeError on creation, not on subsequent calls of the attrgetter object; this shouldn't be a compatibility problem. However, I obviously had to update both Doc/library/operator.rst and Lib/test/test_operator.py .

I am not sure whether I should attach a zip/tar file with both the attachments (the sample benchmark and the diff); so I'll attach the diff in a further comment.

On the Ubuntu server 9.10 where I made the changes, I ran the so3940518.py sample benchmark before and after the changes.

Run before the changes (third column is seconds, less is better):

list comp 0.40999999999999925 1000000
map attrgetter 1.3899999999999997 1000000
map lambda 1.0099999999999998 1000000

Run after the changes:

list comp 0.40000000000000036 1000000
map attrgetter 0.5199999999999996 1000000
map lambda 0.96 1000000

----------
assignee: docs at python
components: Documentation, Library (Lib), Tests
files: so3940518.py
messages: 119247
nosy: docs at python, tzot
priority: normal
severity: normal
status: open
title: operator.attrgetter slower than lambda after adding dotted names ability
versions: Python 3.2
Added file: http://bugs.python.org/file19310/so3940518.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10160>
_______________________________________