[Python-Dev] New syntax for 'dynamic' attribute access
Ben North
ben at redfrontdoor.org
Mon Feb 12 00:45:05 CET 2007
Hi,
A few days ago I posted to python-ideas with a suggestion for some new
Python syntax, which would allow easier access to object attributes
where the attribute name is known only at run-time. For example:
setattr(self, method_name, getattr(self.metadata, method_name))
from Lib/distutils/dist.py could be rewritten
self.(method_name) = self.metadata.(method_name)
The new syntax would also be usable in augmented assignments, as in
obj.(attr_name) += 1
There was some discussion on python-ideas, which I've linked to in the
draft PEP below. In particular, Guido van Rossum wrote:
> I've thought of the same syntax. I think you should submit this to the
> PEP editor and argue on Python-dev for its inclusion in Python 2.6 --
> there's no benefit that I see of waiting until 3.0.
so here I am. Does anybody have any opinions/suggestions, particularly
on the "open questions" referred to in the draft PEP? To summarise
these open questions:
* The draft currently allows a two-argument form, to supply a default
value if the object has no attribute of that name. This mimics the
behaviour of the three-argument form of getattr, but looks a bit wrong:
s = obj.(attr_name, 'default string')
I agree that it looks odd, but perhaps the extra expressive power
gained might be worth the oddness.
* The draft implementation (a sourceforge patch, linked to in the draft
PEP) may have a general performance penalty of around 1%, although my
initial measurements were quite noisy. Josiah Carlson thought this
would not be too substantial, but he did suggest a couple of other
implementation routes which could be explored. The general
performance hit is offset by a speed gain of around 40% for attribute
access using the new syntax over getattr etc. Is 1% "too much" for
this feature?
Ben.
- - - - 8< - - - -
PEP: 363 [PROVISIONAL NUMBER]
Title: Syntax For Dynamic Attribute Access
Version: $Revision$
Last-Modified: $Date$
Author: Ben North <ben at redfrontdoor.org>
Status: Draft
Type: Standards Track
Content-Type: text/plain
Created: 29-Jan-2007
Post-History:
Abstract
Dynamic attribute access is currently possible using the "getattr"
and "setattr" builtins. The present PEP suggests a new syntax to
make such access easier, allowing the coder for example to write
x.('foo_%d' % n) += 1
z = y.('foo_%d' % n).('bar_%s' % s)
instead of
attr_name = 'foo_%d' % n
setattr(x, attr_name, getattr(x, attr_name) + 1)
z = getattr(getattr(y, 'foo_%d' % n), 'bar_%s' % s)
Note
(I wrote this patch mostly to advance my own understanding of and
experiment with the python language, but I've written it up in the
style of a PEP in case it might be a useful idea.)
Rationale
Dictionary access and indexing both have a friendly invocation
syntax: instead of x.__getitem__(12) the coder can write x[12].
This also allows the use of subscripted elements in an augmented
assignment, as in "x[12] += 1". The present proposal brings this
ease-of-use to dynamic attribute access too.
Attribute access is currently possible in two ways:
* When the attribute name is known at code-writing time, the
".NAME" trailer can be used, as in
x.foo = 42
y.bar += 100
* When the attribute name is computed dynamically at run-time, the
"getattr" and "setattr" builtins must be used:
x = getattr(y, 'foo_%d' % n)
setattr(z, 'bar_%s' % s, 99)
The "getattr" builtin also allows the coder to specify a default
value to be returned in the event that the object does not have
an attribute of the given name:
x = getattr(y, 'foo_%d' % n, 0)
This PEP describes a new syntax for dynamic attribute access ---
"x.(expr)" --- with examples given in the Abstract above. The new
syntax also allows the provision of a default value in the "get"
case, as in:
x = y.('foo_%d' % n, None)
This 2-argument form of dynamic attribute access is not permitted as
the target of an (augmented or normal) assignment. Also, this part
of the new syntax was not as well received [6] in initial
discussions on python-ideas, and I agree that it does not fit so
cleanly. I'm happy to prepare a revised PEP/patch without the
2-argument form if the consensus is that this would be preferred.
Finally, the new syntax can be used with the "del" statement, as in
del x.(attr_name)
Impact On Existing Code
The proposed new syntax is not currently valid, so no existing
well-formed programs have their meaning altered by this proposal.
Across all "*.py" files in the 2.5 distribution, there are around
600 uses of "getattr", "setattr" or "delattr". They break down as
follows (figures have some room for error because they were
arrived at by partially-manual inspection):
c.300 uses of plain "getattr(x, attr_name)", which could be
replaced with the new syntax;
c.150 uses of the 3-argument form, i.e., with the default
value; these could be replaced with the 2-argument form
of the new syntax (the cases break down into c.125 cases
where the attribute name is a literal string, and c.25
where it's only known at run-time);
c.5 uses of the 2-argument form with a literal string
attribute name, which I think could be replaced with the
standard "x.attribute" syntax;
c.120 uses of setattr, of which 15 use getattr to find the
new value; all could be replaced with the new syntax,
the 15 where getattr is also involved would show a
particular increase in clarity;
c.5 uses which would have to stay as "getattr" because they
are calls of a variable named "getattr" whose default
value is the builtin "getattr";
c.5 uses of the 2-argument form, inside a try/except block
which catches AttributeError and uses a default value
instead; these could use 2-argument form of the new
syntax;
c.10 uses of "delattr", which could use the new syntax.
As examples, the line
setattr(self, attr, change_root(self.root, getattr(self, attr)))
from Lib/distutils/command/install.py could be rewritten
self.(attr) = change_root(self.root, self.(attr))
and the line
setattr(self, method_name, getattr(self.metadata, method_name))
from Lib/distutils/dist.py could be rewritten
self.(method_name) = self.metadata.(method_name)
Performance Impact
Initial pystone measurements are inconclusive, but suggest there may
be a performance penalty of around 1% in the pystones score with the
patched version. One suggestion is that this is because the longer
main loop in ceval.c hurts the cache behaviour, but this has not
been confirmed. (Maybe a tool like valgrind [2] could help here?)
On the other hand, measurements suggest a speed-up of around 40--45%
for dynamic attribute access.
Discussion To Date
Initial posting of this PEP in draft form was to python-ideas on
20070209 [4], and the response was generally positive:
I've thought of the same syntax. I think you should submit this
to the PEP editor and argue on Python-dev for its inclusion in
Python 2.6 -- there's no benefit that I see of waiting until
3.0. --- Guido van Rossum [5]
Wow! I have to say this is a compelling idea. The syntax is a
bit foreign looking, but [...] I feel like I could learn to like
it anyway. --- Greg Falcon [6]
I look forward to seeing this in Python 2.6. --- Josiah
Carlson, further down the thread [8]
with Greg Falcon expressing the reservations about the 2-argument
form already noted above, and Josiah Carlson raising a query about
performance:
My only concern with your propsed change is your draft
implementation. [...] Specifically, your changes [...] may
negatively affect general Python performance. --- Josiah
Carlson [7]
Some initial measurements (see above) suggest the performance
penalty is small, and Josiah Carlson said of such cost that it
"isn't really substantial". [8]
Questions To Be Resolved
* Whether to allow the 2-argument form for default arguments.
* Whether the performance impact is real; whether it is acceptable;
whether alternative implementations might improve this aspect.
Alternative Syntax For The New Feature
Other syntaxes could be used, for example braces are currently
invalid in a "trailer", so could be used here, giving
x{'foo_%d' % n} += 1
My personal preference is for the
x.('foo_%d' % n) += 1
syntax though: the presence of the dot shows there is attribute
access going on; the parentheses have an analogous meaning to the
mathematical "work this out first" meaning. This is also the
syntax used in the language Matlab [1] for dynamic "field" access
(where "field" is the Matlab term analogous to Python's
"attribute").
Discussions on python-ideas (see above) made no comment on the brace
alternative, and the .() notation was well-enough received, so the
brace alternative should be considered rejected, I think.
Error Cases
Only strings are permitted as attribute names, so for instance the
following error is produced:
>>> x.(99) = 8
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: attribute name must be string, not 'int'
This is handled by the existing PyObject_GetAttr function.
Draft Implementation
A draft implementation adds a new alternative to the "trailer"
clause in Grammar/Grammar; a new AST type, "DynamicAttribute" in
Python.asdl, with accompanying changes to symtable.c, ast.c, and
compile.c, and three new opcodes (load/store/del) with
accompanying changes to opcode.h and ceval.c. The patch consists
of c.180 additional lines in the core code, and c.100 additional
lines of tests. It is available as sourceforge patch #1657573 [3].
References
[1] Using Dynamic Field Names :: Data Types (MATLAB Programming)
http://www.mathworks.com/access/helpdesk/help/techdoc/matlab_prog/f2-41859.html
[2] Valgrind: "suite of tools for debugging and profiling Linux
programs"
http://www.valgrind.org/
[3] Sourceforge patch #1657573
http://sourceforge.net/tracker/index.php?func=detail&aid=1657573&group_id=5470&atid=305470
[4]
http://mail.python.org/pipermail/python-ideas/2007-February/000210.html
[5]
http://mail.python.org/pipermail/python-ideas/2007-February/000211.html
[6]
http://mail.python.org/pipermail/python-ideas/2007-February/000212.html
[7]
http://mail.python.org/pipermail/python-ideas/2007-February/000213.html
[8]
http://mail.python.org/pipermail/python-ideas/2007-February/000227.html
Copyright
This document has been placed in the public domain.
More information about the Python-Dev
mailing list