PEP 335: Overloadable Boolean Operators - Official Posting

[John Roth]

It's certainly an interesting PEP. I've got some comments off the
top.

One is that I'm not at all certain that I like the notion of
'and' and 'or' sometimes short circuiting and sometimes
not, especially since you can't reliably predict when it will
and when it won't if the program is playing fast and loose
with typing.

A second comment is that I think you need seven slots - the
'rand2' and 'ror2' slots seem to be missing.

A third comment is that I think the 'and1' and 'or1'
should return either True or False, meaning 'short
circuit with the current value on top of the stack',
or 'continue with the evaluation with the current
value on top of the stack'. I'd like to see a discussion
of why this (IMO conceptually simpler) option
wasn't chosen. In particular, I really don't like the
idea that and1 and or1 can arbitrarily change the result
of the first expression evaluation.

John Roth


"Greg Ewing" <greg at cosc.canterbury.ac.nz> wrote in message 
news:2qccb9Fuac61U1 at uni-berlin.de...
> PEP: 335
> Title: Overloadable Boolean Operators
> Version: $Revision: 1.2 $
> Last-Modified: $Date: 2004/09/09 14:17:17 $
> Author: Gregory Ewing <greg at cosc.canterbury.ac.nz>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 29-Aug-2004
> Python-Version: 2.4
> Post-History: 05-Sep-2004
>
>
> Abstract
> ========
>
> This PEP proposes an extension to permit objects to define their own
> meanings for the boolean operators 'and', 'or' and 'not', and suggests
> an efficient strategy for implementation.  A prototype of this
> implementation is available for download.
>
>
> Background
> ==========
>
> Python does not currently provide any '__xxx__' special methods
> corresponding to the 'and', 'or' and 'not' boolean operators.  In the
> case of 'and' and 'or', the most likely reason is that these operators
> have short-circuiting semantics, i.e. the second operand is not
> evaluated if the result can be determined from the first operand.  The
> usual technique of providing special methods for these operators
> therefore would not work.
>
> There is no such difficulty in the case of 'not', however, and it
> would be straightforward to provide a special method for this
> operator.  The rest of this proposal will therefore concentrate mainly
> on providing a way to overload 'and' and 'or'.
>
>
> Motivation
> ==========
>
> There are many applications in which it is natural to provide custom
> meanings for Python operators, and in some of these, having boolean
> operators excluded from those able to be customised can be
> inconvenient.  Examples include:
>
> 1. Numeric/Numarray, in which almost all the operators are defined on
>    arrays so as to perform the appropriate operation between
>    corresponding elements, and return an array of the results.  For
>    consistency, one would expect a boolean operation between two
>    arrays to return an array of booleans, but this is not currently
>    possible.
>
>    There is a precedent for an extension of this kind: comparison
>    operators were originally restricted to returning boolean results,
>    and rich comparisons were added so that comparisons of Numeric
>    arrays could return arrays of booleans.
>
> 2. A symbolic algebra system, in which a Python expression is
>    evaluated in an environment which results in it constructing a tree
>    of objects corresponding to the structure of the expression.
>
> 3. A relational database interface, in which a Python expression is
>    used to construct an SQL query.
>
> A workaround often suggested is to use the bitwise operators '&', '|'
> and '~' in place of 'and', 'or' and 'not', but this has some
> drawbacks.  The precedence of these is different in relation to the
> other operators, and they may already be in use for other purposes (as
> in example 1).  There is also the aesthetic consideration of forcing
> users to use something other than the most obvious syntax for what
> they are trying to express.  This would be particularly acute in the
> case of example 3, considering that boolean operations are a staple of
> SQL queries.
>
>
> Rationale
> =========
>
> The requirements for a successful solution to the problem of allowing
> boolean operators to be customised are:
>
> 1. In the default case (where there is no customisation), the existing
>    short-circuiting semantics must be preserved.
>
> 2. There must not be any appreciable loss of speed in the default
>    case.
>
> 3. If possible, the customisation mechanism should allow the object to
>    provide either short-circuiting or non-short-circuiting semantics,
>    at its discretion.
>
> One obvious strategy, that has been previously suggested, is to pass
> into the special method the first argument and a function for
> evaluating the second argument.  This would satisfy requirements 1 and
> 3, but not requirement 2, since it would incur the overhead of
> constructing a function object and possibly a Python function call on
> every boolean operation.  Therefore, it will not be considered further
> here.
>
> The following section proposes a strategy that addresses all three
> requirements.  A `prototype implementation`_ of this strategy is
> available for download.
>
> .. _prototype implementation:
>    http://www.cosc.canterbury.ac.nz/~greg/python/obo//Python_OBO.tar.gz
>
>
> Specification
> =============
>
> Special Methods
> ---------------
>
> At the Python level, objects may define the following special methods.
>
> ===============  =================  ========================
> Unary            Binary, phase 1    Binary, phase 2
> ===============  =================  ========================
> * __not__(self)  * __and1__(self)   * __and2__(self, other)
>                  * __or1__(self)    * __or2__(self, other)
>                                     * __rand2__(self, other)
>                                     * __ror2__(self, other)
> ===============  =================  ========================
>
> The __not__ method, if defined, implements the 'not' operator.  If it
> is not defined, or it returns NotImplemented, existing semantics are
> used.
>
> To permit short-circuiting, processing of the 'and' and 'or' operators
> is split into two phases.  Phase 1 occurs after evaluation of the first
> operand but before the second.  If the first operand defines the
> appropriate phase 1 method, it is called with the first operand as
> argument.  If that method can determine the result without needing the
> second operand, it returns the result, and further processing is
> skipped.
>
> If the phase 1 method determines that the second operand is needed, it
> returns the special value NeedOtherOperand.  This triggers the
> evaluation of the second operand, and the calling of an appropriate
> phase 2 method. During phase 2, the __and2__/__rand2__ and
> __or2__/__ror2__ method pairs work as for other binary operators.
>
> Processing falls back to existing semantics if at any stage a relevant
> special method is not found or returns NotImplemented.
>
> As a special case, if the first operand defines a phase 2 method but
> no corresponding phase 1 method, the second operand is always
> evaluated and the phase 2 method called.  This allows an object which
> does not want short-circuiting semantics to simply implement the
> relevant phase 2 methods and ignore phase 1.
>
>
> Bytecodes
> ---------
>
> The patch adds four new bytecodes, LOGICAL_AND_1, LOGICAL_AND_2,
> LOGICAL_OR_1 and LOGICAL_OR_2.  As an example of their use, the
> bytecode generated for an 'and' expression looks like this::
>
>             .
>             .
>             .
>             evaluate first operand
>             LOGICAL_AND_1  L
>             evaluate second operand
>             LOGICAL_AND_2
>        L:   .
>             .
>             .
>
> The LOGICAL_AND_1 bytecode performs phase 1 processing.  If it
> determines that the second operand is needed, it leaves the first
> operand on the stack and continues with the following code.  Otherwise
> it pops the first operand, pushes the result and branches to L.
>
> The LOGICAL_AND_2 bytecode performs phase 2 processing, popping both
> operands and pushing the result.
>
>
> Type Slots
> ----------
>
> A the C level, the new special methods are manifested as five new
> slots in the type object.  In the patch, they are added to the
> tp_as_number substructure, since this allowed making use of some
> existing code for dealing with unary and binary operators.  Their
> existence is signalled by a new type flag,
> Py_TPFLAGS_HAVE_BOOLEAN_OVERLOAD.
>
> The new type slots are::
>
>     unaryfunc nb_logical_not;
>     unaryfunc nb_logical_and_1;
>     unaryfunc nb_logical_or_1;
>     binaryfunc nb_logical_and_2;
>     binaryfunc nb_logical_or_2;
>
>
> Python/C API Functions
> ----------------------
>
> There are also five new Python/C API functions corresponding to the
> new operations::
>
>     PyObject *PyObject_LogicalNot(PyObject *);
>     PyObject *PyObject_LogicalAnd1(PyObject *);
>     PyObject *PyObject_LogicalOr1(PyObject *);
>     PyObject *PyObject_LogicalAnd2(PyObject *, PyObject *);
>     PyObject *PyObject_LogicalOr2(PyObject *, PyObject *);
>
>
> Copyright
> =========
>
> This document has been placed in the public domain.
>
>
> ..
>    Local Variables:
>    mode: indented-text
>    indent-tabs-mode: nil
>    sentence-end-double-space: t
>    fill-column: 70
>    End:
>