[PEP 203] Augmented Assignment
Thomas Wouters
thomas at xs4all.net
Mon Aug 14 12:03:28 EDT 2000
I've attached the current draft for the Augmented Assignment proposal. Due
to popular demand <wink> I've added a Rationale section that tries to
(*briefly*) explain why augmented assignment is a Good Thing. If you prefer
HTML over text, you can find the HTML version on the python PEP page:
http://python.sourceforge.net/peps/pep-0203.html
This is your moment, people. Now you can leave a lasting impression on
Python itself ! Augmented assignment is scheduled to go in soon (well,
before 2.0b1 at least) and if you don't spot the loony now, we'll have to
live with it forever :)
Other comments, from spelling fixes to raving mad mobs bashing my door in,
are also greatly welcome. It'll be one more reason to find myself a real
house :)
Takes-a-loony-to-spot-one-ly y'rs,
--
Thomas Wouters <thomas at xs4all.net>
Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
-------------- next part --------------
PEP: 203
Title: Augmented Assignments
Version: $Revision: 1.5 $
Owner: thomas at xs4all.net (Thomas Wouters)
Python-Version: 2.0
Status: Draft
Created: 13-Jul-2000
Type: Standard
Introduction
This PEP describes the `augmented assignment' proposal for Python
2.0. This PEP tracks the status and ownership of this feature,
slated for introduction in Python 2.0. It contains a description
of the feature and outlines changes necessary to support the
feature. This PEP summarizes discussions held in mailing list
forums, and provides URLs for further information, where
appropriate. The CVS revision history of this file contains the
definitive historical record.
Proposed semantics
The proposed patch that adds augmented assignment to Python
introduces the following new operators:
+= -= *= /= %= **= <<= >>= &= ^= |=
They implement the same operator as their normal binary form,
except that the operation is done `in-place' when the left-hand
side object supports it, and that the left-hand side is only
evaluated once.
They truly behave as augmented assignment, in that they perform
all of the normal load and store operations, in addition to the
binary operation they are intended to do. So, given the expression:
x += y
The object `x' is loaded, then `y' is added to it, and the
resulting object is stored back in the original place. The precise
action performed on the two arguments depends on the type of `x',
and possibly of `y'.
The idea behind augmented assignment in Python is that it isn't
just an easier way to write the common practice of storing the
result of a binary operation in its left-hand operand, but also a
way for the left-hand operand in question to know that it should
operate `on itself', rather than creating a modified copy of
itself.
To make this possible, a number of new `hooks' are added to Python
classes and C extention types, which are called when the object in
question is used as the left hand side of an augmented assignment
operation. If the class or type does not implement the `in-place'
hooks, the normal hooks for the particular binary operation are
used.
So, given an instance object `x', the expression
x += y
tries to call x.__add_ab__(y), which is the `in-place' variant of
__add__. If __add_ab__ is not present, x.__add__(y) is
attempted, and finally y.__radd__(x) if __add__ is missing too.
There is no `right-hand-side' variant of __add_ab__, because that
would require for `y' to know how to in-place modify `x', which is
an unsafe to say the least. The __add_ab__ hook should behave exactly
like __add__, returning the result of the operation (which could
be `self') which is to be stored in the variable `x'.
For C extention types, the `hooks' are members of the
PyNumberMethods and PySequenceMethods structures, and are called
in exactly the same manner as the existing non-inplace operations,
including argument coercion. C methods should also take care to
return a new reference to the result object, whether it's the same
object or a new one. So if the original object is returned, it
should be INCREF()'d appropriately.
Rationale
There are two main reasons for adding this feature to Python:
simplicity of expression, and support for in-place operations. The
end result is a tradeoff between simplicity of syntax and
simplicity of expression; like most new features, augmented
assignment doesn't add anything that was previously impossible. It
merely makes these things easier to do.
Adding augmented assignment will make Pythons syntax more complex.
Instead of a single assignment operation, there are now twelve
assignment operations, eleven of which also perform an binary
operation. However, these eleven new forms of assignment are easy
to understand as the coupling between assignment and the binary
operation, and they require no large conceptual leap to
understand. Furthermore, languages that do have augmented
assignment have shown that they are a popular, much used feature.
Expressions of the form
<x> = <x> <operator> <y>
are common enough in those languages to make the extra syntax
worthwhile, and Python does not have significantly less of those
expressions. Quite the opposite, in fact, since in Python you can
also concatenate lists with a binary operator, something that is
done quite frequently. Writing the above expression as
<x> <operator>= <y>
is both more readable and less error prone, because it is
instantly obvious to the reader that it is <x> that is being
changed, and not <x> that is being replaced by something almost,
but not quite, entirely unlike <x>.
The new in-place operations are especially useful to matrix
calculation and other applications that require large objects. In
order to efficiently deal with the available program memory, such
packages cannot blindly use the current binary operations. Because
these operations always create a new character, adding a single
item to an existing (large) object would result in copying the
entire object (which may cause the application to run out of
memory), add the single item, and then possibly delete the
original object, depending on reference count.
To work around this problem, the packages currently have to use
methods or functions to modify an object in-place, which is
definately less readable than an augmented assignment expression.
Augmented assignment won't solve all the problems for these
packages, since some operations cannot be expressed in the limited
set of binary operators to start with, but it is a start. A
different PEP[3] is looking at adding new operators.
New methods
The proposed implementation adds the following 11 possible `hooks'
which Python classes can implement to overload the augmented
assignment operations:
__add_ab__
__sub_ab__
__mul_ab__
__div_ab__
__mod_ab__
__pow_ab__
__lshift_ab__
__rshift_ab__
__and_ab__
__xor_ab__
__or_ab__
The `__add_ab__' name is one proposed by Guido[1], and stands for
`and becomes'. Other proposed names include `__iadd__',
`__add_in__' and `__inplace_add__'. A firm decision by the BDFL is
probably needed to finalize this issue.
For C extention types, the following struct members are added:
To PyNumberMethods:
binaryfunc nb_inplace_add;
binaryfunc nb_inplace_subtract;
binaryfunc nb_inplace_multiply;
binaryfunc nb_inplace_divide;
binaryfunc nb_inplace_remainder;
binaryfunc nb_inplace_power;
binaryfunc nb_inplace_lshift;
binaryfunc nb_inplace_rshift;
binaryfunc nb_inplace_and;
binaryfunc nb_inplace_xor;
binaryfunc nb_inplace_or;
To PySequenceMethods:
binaryfunc sq_inplace_concat;
intargfunc sq_inplace_repeat;
In order to keep binary compatibility, the tp_flags TypeObject
member is used to determine whether the TypeObject in question has
allocated room for these slots. Until a clean break in binary
compatibility is made (which may or may not happen before 2.0)
code that wants to use one of the new struct members must first
check that they are available with the `PyType_HasFeature()'
macro:
if (PyType_HasFeature(x->ob_type, Py_TPFLAGS_HAVE_INPLACE_OPS) &&
x->ob_type->tp_as_number && x->ob_type->tp_as_number->nb_inplace_add) {
/* ... */
This check must be made even before testing the method slots for
NULL values! The macro only tests whether the slots are available,
not whether they are filled with methods or not.
Implementation
The current implementation of augmented assignment[2] adds, in
addition to the methods and slots alread covered, 13 new bytecodes
and 13 new API functions.
The API functions are simply in-place versions of the current
binary-operation API functions:
PyNumber_InPlaceAdd(PyObject *o1, PyObject *o2);
PyNumber_InPlaceSubtract(PyObject *o1, PyObject *o2);
PyNumber_InPlaceMultiply(PyObject *o1, PyObject *o2);
PyNumber_InPlaceDivide(PyObject *o1, PyObject *o2);
PyNumber_InPlaceRemainder(PyObject *o1, PyObject *o2);
PyNumber_InPlacePower(PyObject *o1, PyObject *o2);
PyNumber_InPlaceLshift(PyObject *o1, PyObject *o2);
PyNumber_InPlaceRshift(PyObject *o1, PyObject *o2);
PyNumber_InPlaceAnd(PyObject *o1, PyObject *o2);
PyNumber_InPlaceXor(PyObject *o1, PyObject *o2);
PyNumber_InPlaceOr(PyObject *o1, PyObject *o2);
PySequence_InPlaceConcat(PyObject *o1, PyObject *o2);
PySequence_InPlaceRepeat(PyObject *o, int count);
They call either the Python class hooks (if either of the objects
is a Python class instance) or the C type's number or sequence
methods.
The new bytecodes are:
INPLACE_ADD
INPLACE_SUBTRACT
INPLACE_MULTIPLY
INPLACE_DIVIDE
INPLACE_REMAINDER
INPLACE_POWER
INPLACE_LEFTSHIFT
INPLACE_RIGHTSHIFT
INPLACE_AND
INPLACE_XOR
INPLACE_OR
ROT_FOUR
DUP_TOPX
The INPLACE_* bytecodes mirror the BINARY_* bytecodes, except that
they are implemented as calls to the `InPlace' API functions. The
other two bytecodes are `utility' bytecodes: ROT_FOUR behaves like
ROT_THREE except that the four topmost stack items are rotated.
DUP_TOPX is a bytecode that takes a single argument, which should
be an integer between 1 and 5 (inclusive) which is the number of
items to duplicate in one block. Given a stack like this (where
the right side of the list is the `top' of the stack):
[1, 2, 3, 4, 5]
"DUP_TOPX 3" would duplicate the top 3 items, resulting in this
stack:
[1, 2, 3, 4, 5, 3, 4, 5]
DUP_TOPX with an argument of 1 is the same as DUP_TOP. The limit
of 5 is purely an implementation limit. The implementation of
augmented assignment requires only DUP_TOPX with an argument of 2
and 3, and could do without this new opcode at the cost of a fair
number of DUP_TOP and ROT_*.
Open Issues
The PyNumber_InPlacePower() function only takes two arguments, not
one like PyNumber_Power(). This is because there is no way to do
an inplace three-argument-power trough the augmented assignment
syntax or the power() function.
Possibly a more obvious name for the Python hooks can be found.
`_ab_' is what Guido proposed[1] as a working name, and comes from
an old Algol-68 naming convention.
Documentation needs to be written. The reference manual, the `dis'
section of the library manual, and possibly the tutorial.
The DUP_TOPX bytecode is a conveniency bytecode, and is not
actually necessary. It should be considered whether this bytecode
is worth having. There seems to be no other possible use for this
bytecode at this time.
The standard library should be adjusted to provide augmented
assignment hooks, where sensible.
It is not possible to do an inplace operation in the variant of
<builtin type> += <instance object>
Instead, the instance objects' __radd__ hook is called, with the
builtin type as argument. The same goes for the other operations.
It might necessary to add a right-hand version of __add_ab__ after
all, to support something like that.
Copyright
This document has been placed in the public domain.
References
[1] http://www.python.org/pipermail/python-list/2000-June/059556.html
[2]
http://sourceforge.net/patch?func=detailpatch&patch_id=100699&group_id=5470
Local Variables:
mode: indented-text
indent-tabs-mode: nil
End:
More information about the Python-list
mailing list