[Python-Dev] PEP 457: Syntax For Positional-Only Parameters

Eric Snow ericsnowcurrently at gmail.com
Wed Oct 9 07:48:18 CEST 2013 

On Tue, Oct 8, 2013 at 5:33 PM, Larry Hastings <larry at hastings.org> wrote:
>
> I've contributed a new PEP to humanity.  I include the RST for your reading
> pleasure below, but you can also read it online here:
>
> http://www.python.org/dev/peps/pep-0457/
>
>
> Discuss,

The PEP doesn't mention anything about inspect.Signature, which
already supports positional-only parameters.  It should.

Further comments in-line.

-eric

>
>
> /arry
>
> -----
>
> PEP: 457
> Title: Syntax For Positional-Only Parameters
> Version: $Revision$
> Last-Modified: $Date$
> Author: Larry Hastings <larry at hastings.org>
> Discussions-To: Python-Dev <python-dev at python.org>
> Status: Draft
> Type: Informational
> Content-Type: text/x-rst
> Created: 08-Oct-2013
>
>
> ========
> Overview
> ========
>
> This PEP proposes a syntax for positional-only parameters in Python.
> Positional-only parameters are parameters without an externally-usable
> name; when a function accepting positional-only parameters is called,
> positional arguments are mapped to these parameters based solely on
> their position.
>
> =========
> Rationale
> =========
>
> Python has always supported positional-only parameters.
> Early versions of Python lacked the concept of specifying
> parameters by name, so naturally all parameters were
> positional-only.  This changed around Python 1.0, when
> all parameters suddenly became positional-or-keyword.
> But, even in current versions of Python, many CPython
> "builtin" functions still only accept positional-only
> arguments.
>
> Functions implemented in modern Python can accept
> an arbitrary number of positional-only arguments, via the
> variadic ``*args`` parameter.  However, there is no Python
> syntax to specify accepting a specific number of
> positional-only parameters.  Put another way, there are
> many builtin functions whose signatures are simply not
> expressable with Python syntax.

Definitely the key point.

This also has an impact on other Python implementations, where
maintaining the same positional-only semantics for "builtins" is
complicated (at least a little) by the lack of a positional-only
syntax in Python.

>
> This PEP proposes a backwards-compatible syntax that should
> permit implementing any builtin in pure Python code.
>
> -----------------------------------------------------
> Positional-Only Parameter Semantics In Current Python
> -----------------------------------------------------
>
> There are many, many examples of builtins that only
> accept positional-only parameters.  The resulting
> semantics are easily experienced by the Python
> programmer--just try calling one, specifying its
> arguments by name::
>
>     >>> pow(x=5, y=3)
>     Traceback (most recent call last):
>       File "<stdin>", line 1, in <module>
>     TypeError: pow() takes no keyword arguments
>
> In addition, there are some functions with particularly
> interesting semantics:
>
>   * ``range()``, which accepts an optional parameter
>     to the *left* of its required parameter. [#RANGE]_
>
>   * ``dict()``, whose mapping/iterator parameter is optional and
>     semantically must be positional-only.  Any externally
>     visible name for this parameter would occlude
>     that name going into the ``**kwarg`` keyword variadic
>     parameter dict! [#DICT]_
>
> Obviously one can simulate any of these in pure Python code
> by accepting ``(*args, **kwargs)`` and parsing the arguments
> by hand.  But this results in a disconnect between the
> Python function's signature and what it actually accepts,
> not to mention the work of implementing said argument parsing.
>
> ==========
> Motivation
> ==========
>
> This PEP does not propose we implement positional-only
> parameters in Python.  The goal of this PEP is simply
> to define the syntax,

This would be worth noting in the "Overview" section.  It completely
changes the intent of the proposal.

> so that:
>
>     * Documentation can clearly, unambiguously, and
>       consistently express exactly how the arguments
>       for a function will be interpreted.
>
>     * The syntax is reserved for future use, in case
>       the community decides someday to add positional-only
>       parameters to the language.
>
>     * Argument Clinic can use a variant of the syntax
>       as part of its input when defining
>       the arguments for built-in functions.
>
> =================================================================
> The Current State Of Documentation For Positional-Only Parameters
> =================================================================
>
> The documentation for positional-only parameters is incomplete
> and inconsistent:
>
> * Some functions denote optional groups of positional-only arguments
>   by enclosing them in nested square brackets. [#BORDER]_
>
> * Some functions denote optional groups of positional-only arguments
>   by presenting multiple prototypes with varying numbers of
>   arguments. [#SENDFILE]_
>
> * Some functions use *both* of the above approaches. [#RANGE]_ [#ADDCH]_
>
> One more important idea to consider: currently in the documentation
> there's no way to tell whether a function takes positional-only
> parameters.  ``open()`` accepts keyword arguments, ``ord()`` does
> not, but there is no way of telling just by reading the
> documentation that this is true.
>
> ====================
> Syntax And Semantics
> ====================
>
> From the "ten-thousand foot view", and ignoring ``*args`` and ``**kwargs``
> for now, the grammar for a function definition currently looks like this::
>
>     def name(positional_or_keyword_parameters, *, keyword_only_parameters):
>
> Building on that perspective, the new syntax for functions would look
> like this::
>
>     def name(positional_only_parameters, /,
> positional_or_keyword_parameters,
>              *, keyword_only_parameters):
>
> All parameters before the ``/`` are positional-only.  If ``/`` is
> not specified in a function signature, that function does not
> accept any positional-only parameters.
>
> Positional-only parameters can be optional, but the mechanism is
> significantly different from positional-or-keyword or keyword-only
> parameters.  Positional-only parameters don't accept default
> values.  Instead, positional-only parameters can be specified
> in optional "groups".  Groups of parameters are surrounded by
> square brackets, like so::
>
>     def addch([y, x,] ch, [attr], /):

This is for the sake of only a couple unfortunate builtins, right?
Parameter grouping will undoubtedly be the most controversial part of
this PEP.  I'm not convinced this is a capability we need to encourage
in any way.  Is there some alternate one-off we can use for range,
addch, et al.?

>
> Positional-only parameters that are not in an option group are
> "required" positional-only parameters.  All "required" positional-only
> parameters must be contiguous.
>
> Parameters in an optional group accept arguments in a group; you
> must provide arguments either for all of the them or for none of them.
> Using the example of ``addch()`` above, you could not call ``addch()``
> in such a way that ``x`` was specified but ``y`` was not (and vice versa).
> The mapping of positional parameters to optional groups is done
> based on fitting the number of parameters to groups.  Based on the
> above definition, ``addch()`` would assign arguments to parameters
> in the following way:
>
>     +-------------------+------------------------------+
>     |Number of arguments|Parameter assignment          |
>     +-------------------+------------------------------+
>     |0                  |*raises an exception*         |
>     +-------------------+------------------------------+
>     |1                  |``ch``                        |
>     +-------------------+------------------------------+
>     |2                  |``ch``, ``attr``              |
>     +-------------------+------------------------------+
>     |3                  |``y``, ``x``, ``ch``          |
>     +-------------------+------------------------------+
>     |4                  |``y``, ``x``, ``ch``, ``attr``|
>     +-------------------+------------------------------+
>     |5 or more          |*raises an exception*         |
>     +-------------------+------------------------------+

What are the names bound to for unfulfilled optional groups?  Not
bound (yuck)?  What is wrong with default values within optional
groups (other than how messy the syntax starts to become)?

>
>
> More semantics of positional-only parameters:
>
>   * Although positional-only parameter technically have names,
>     these names are internal-only; positional-only parameters
>     are *never* externally addressable by name.  (Similarly
>     to ``*args`` and ``**kwargs``.)
>
>   * It's possible to nest option groups.
>
>   * If there are no required parameters, all option groups behave
>     as if they're to the right of the required parameter group.
>
>   * For clarity and consistency, the comma for a parameter always
>     comes immediately after the parameter name.  It's a syntax error
>     to specify a square bracket between the name of a parameter and
>     the following comma.  (This is far more readable than putting
>     the comma outside the square bracket, particularly for nested
>     groups.)

Good catch.  Presumably parsing this won't be a pain.

>
>   * If there are arguments after the ``/``, then you must specify
>     a comma after the ``/``, just as there is a comma
>     after the ``*`` denoting the shift to keyword-only parameters.
>
>   * This syntax has no effect on ``*args`` or ``**kwargs``.
>
> It's possible to specify a function prototype where the mapping
> of arguments to parameters is ambiguous.  Consider::
>
>     def range([start,] stop, [range], /):
>
> Python disambiguates these situations by preferring optional groups
> to the *left* of the required group.

What about this:

def spam([a,] b, [c,] d, /):

Using my intuitive ambiguity-resolution heuristics, I would guess it
would resolve spam(1, 2, 3) by binding 3 to d.  However, more
complicated signatures, particularly involving optional groups, would
require even more elaborate ambiguity resolution, wouldn't they?
While this proposal isn't about an actual implementation, it does
imply that any future implementation would be reasonable.  Do you feel
that's the case here?

>
> ======================
> Additional Limitations
> ======================
>
> Argument Clinic uses a form of this syntax for specifying
> builtins.  It imposes further limitations that are
> theoretically unnecessary but make the implementation
> easier.  Specifically:
>
> * A function that has positional-only parameters currently
>   cannot have any other kind of parameter.  (This will
>   probably be relaxed slightly in the near future.)
>
> * Multiple option groups on either side of the required
>   positional-only parameters must be nested, with the
>   nesting getting deeper the further away the group is
>   from the required positional-parameter group.

Did I miss it or did you already specify that there can only be 1
required positional-only group?  It certainly simplifies things a
bunch and lessens my concerns about ambiguity-resolution.

>
>   Put another way:
>   all the left-brackets for option groups to the
>   left of the required group must be specified contiguously,
>   and
>   all the right-brackets for option groups to the
>   right of the required group must be specified contiguously.
>
>
> ==============================
> Notes For A Future Implementor
> ==============================
>
> If we decide to implement positional-only parameters in a future
> version of Python, we'd have to do some additional work to preserve
> their semantics.  The problem: how do we inform a parameter that
> no value was passed in for it when the function was called?
>
> The obvious solution: add a new singleton constant to Python
> that is passed in when a parameter is not mapped to an argument.
> I propose that the value be called called ``undefined``,

I've called this "NOT_SET" when I've had the need.

> and be a singleton of a special class called ``Undefined``.
> If a positional-only parameter did not receive an argument
> when called, its value would be set to ``undefined``.
>
> But this raises a further problem.  How do can we tell the
> difference between "this positional-only parameter did not
> receive an argument" and "the caller passed in ``undefined``
> for this parameter"?
>
> It'd be nice to make it illegal to pass ``undefined`` in
> as an argument to a function--to, say, raise an exception.
> But that would slow Python down, and the "consenting adults"
> rule appears applicable here.  So making it illegal should
> probably be strongly discouraged but not outright prevented.
>
> However, it should be allowed (and encouraged) for user
> functions to specify ``undefined`` as a default value for
> parameters.

Too bad that None doesn't fit here.  This is where allowing defaults
in optional groups would be handy.

>
> ====================
> Unresolved Questions
> ====================
>
> There are three types of parameters in Python:
>
>   1. positional-only parameters,
>   2. positional-or-keyword parameters, and
>   3. keyword-only parameters.
>
> Python allows functions to have both 2 and 3.  And some
> builtins (e.g. range) have both 1 and 3.  Does it make
> sense to have functions that have both 1 and 2?  Or
> all of the above?

I'm pretty sure I've done this before (using *args and **kwargs).
However, I don't have any concrete examples.

>
>
> ======
> Thanks
> ======
>
> Credit for the use of '/' as the separator between positional-only and
> positional-or-keyword
> parameters goes to Guido van Rossum, in a proposal from 2012. [#GUIDO]_
>
> Credit for making left option groups higher precedence goes to
> Nick Coghlan. (Conversation in person at PyCon US 2013.)
>
> .. [#DICT]
>     http://docs.python.org/3/library/stdtypes.html#dict
>
> .. [#RANGE]
>     http://docs.python.org/3/library/functions.html#func-range
>
> .. [#BORDER]
>     http://docs.python.org/3/library/curses.html#curses.window.border
>
> .. [#SENDFILE]
>     http://docs.python.org/3/library/os.html#os.sendfile
>
> .. [#ADDCH]
>     http://docs.python.org/3/library/curses.html#curses.window.addch
>
> .. [#GUIDO]
>    Guido van Rossum, posting to python-ideas, March 2012:
>    http://mail.python.org/pipermail/python-ideas/2012-March/014364.html
>    and
>    http://mail.python.org/pipermail/python-ideas/2012-March/014378.html
>    and
>    http://mail.python.org/pipermail/python-ideas/2012-March/014417.html
>
> =========
> Copyright
> =========
>
> This document has been placed in the public domain.
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/ericsnowcurrently%40gmail.com
>

More information about the Python-Dev mailing list