Notice: While Javascript is not essential for this website, your interaction with the content will be limited. Please turn Javascript on for the full experience.

PEP 570 -- Python Positional-Only Parameters

PEP:570
Title:Python Positional-Only Parameters
Author:Larry Hastings <larry at hastings.org>, Pablo Galindo <pablogsal at gmail.com>, Mario Corchero <mariocj89 at gmail.com>
Discussions-To:Python-Dev <python-dev at python.org>
Status:Draft
Type:Standards Track
Created:20-Jan-2018

Overview

This PEP proposes a syntax for positional-only parameters in Python. Positional-only parameters are parameters without an externally-usable name; when a function accepting positional-only parameters is called, positional arguments are mapped to these parameters based solely on their position.

Rationale

Python has always supported positional-only parameters. Early versions of Python lacked the concept of specifying parameters by name, so naturally all parameters were positional-only. This changed around Python 1.0, when all parameters suddenly became positional-or-keyword. This allowed users to provide arguments to a function both positionally or referencing the keyword used in the definition of it. But, this is not always desired nor available as even in current versions of Python, many CPython "builtin" functions still only accept positional-only arguments.

Users might want to restrict their API to not allow for parameters to be referenced via keywords, as that exposes the name of the parameter as part of the API. If a user of said API starts using the argument by keyword when calling it and then the parameter gets renamed, it will be a breaking change. By using positional-only parameters the developer can later change the name of an arguments or transform them to *args without breaking the API.

Even if making arguments positional-only in a function can be achieved by using *args parameters and extracting them one by one, the solution is far from ideal and not as expressive as the one proposed in this PEP, which targets providing syntax to specify accepting a specific number of positional-only parameters. Also, it makes the signature of the function ambiguous as users won't know how many parameters the function takes by looking at help() or auto-generated documentation.

Additionally, this will bridge the gap we currently find between builtin functions that can specify positional-only parameters and pure Python implementations that lack the syntax for it. The '/' syntax is already exposed in the documentation of some builtins and interfaces generated by the argument clinic. Making positional-only arguments a possibility in Python will make the language more consistent and make it clearer to users that positional-only arguments are allowed in builtins and argument clinic C interfaces.

We can find positional-only arguments useful in several situations. Example: we want to create a function that converts from one type to another:

def as_my_type(x):
    ...

The name of the parameter provides no value whatsoever, and forces the developer to maintain its name forever, as users might pass x as a keyword.

Another good example is an API that wants to transmit the feeling of ownership through positional arguments, see:

class MyDecorator:
    def __init__(self, original_function):
        ...

Again we get no value from using keyword arguments here and it can limit future evolution of the API. Say at a later time we want the decorator to be able to take multiple functions, we will be forced to always keep the original argument or we would potentially break users. By being able to define positional-only arguments we can change the name of those at will or even change them to *args.

Positional-Only Parameter Semantics In Python Today

There are many, many examples of builtins that only accept positional-only parameters. The resulting semantics are easily experienced by the Python programmer -- just try calling one, specifying its arguments by name:

>>> help(pow)
...
pow(x, y, z=None, /)
...
>>> pow(x=5, y=3)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: pow() takes no keyword arguments

pow() clearly expresses that its arguments are only positional via the / marker, but this at the moment is only documentational, Python developers cannot write such syntax.

In addition, there are some functions with particularly interesting semantics:

  • range(), which accepts an optional parameter to the left of its required parameter. [2]
  • dict(), whose mapping/iterator parameter is optional and semantically must be positional-only. Any externally visible name for this parameter would occlude that name going into the **kwarg keyword variadic parameter dict! [1]

Obviously one can simulate any of these in pure Python code by accepting (*args, **kwargs) and parsing the arguments by hand. But this results in a disconnect between the Python function signature and what the function actually accepts, not to mention the work of implementing said argument parsing and the lack of clarity in the resulting signature.

Motivation

The new syntax will allow developers to further control how their API can be consumed. It will allow restricting the usage of keyword arguments by adding the new type of positional-only ones.

A similar PEP with a broader scope (PEP 457) was proposed earlier to define the syntax. This PEP builds partially on top of that, to define and provide an implementation for the / syntax in function signatures.

Providing positional-only arguments will allow for maintaining the interface when creating pure Python implementation of C modules, which provides not only the API benefits outlined in this document but it is also faster. See this thread about converting keyword arguments to positional: [11] and PEP-399 [8], which requires the same API for C accelerators as the Python implementation.

There have been multiple changes in builtin functions that moved away from keyword arguments, like bool, float, list, int, tuple which is a non-backward compatible change. By having proper support for positional-only arguments, these kind of APIs, where it is clear that passing a keyword argument provides no clarity, it would be possible to follow a similar approach as these builtins, without breaking users.

This is a well discussed, recurring topic on the Python mailing lists:

The Current State Of Documentation For Positional-Only Parameters

The documentation for positional-only parameters is incomplete and inconsistent:

  • Some functions denote optional groups of positional-only arguments by enclosing them in nested square brackets. [3]
  • Some functions denote optional groups of positional-only arguments by presenting multiple prototypes with varying numbers of arguments. [4]
  • Some functions use both of the above approaches. [2] [5]

One more important idea to consider: currently in the documentation there is no way to tell whether a function takes positional-only parameters. open() accepts keyword arguments, ord() does not, but there is no way of telling just by reading the documentation.

Syntax And Semantics

From the "ten-thousand foot view", and ignoring *args and **kwargs for now, the grammar for a function definition currently looks like this:

def name(positional_or_keyword_parameters, *, keyword_only_parameters):

Building on that perspective, the new syntax for functions would look like this:

def name(positional_only_parameters, /, positional_or_keyword_parameters,
         *, keyword_only_parameters):

All parameters before the / are positional-only. If / is not specified in a function signature, that function does not accept any positional-only parameters. The logic around optional values for positional-only argument remains the same as the one for positional-or-keyword. Once a positional-only argument is provided with a default, the following positional-only and positional-or-keyword argument needs to have a default as well. Positional-only parameters that don’t have a default value are "required" positional-only parameters. Therefore the following are valid signatures:

def name(p1, p2, /, p_or_kw, *, kw):
def name(p1, p2=None, /, p_or_kw=None, *, kw):
def name(p1, p2=None, /, *, kw):
def name(p1, p2=None, /):
def name(p1, p2, /, p_or_kw):
def name(p1, p2, /):

Whilst the followings are not:

def name(p1, p2=None, /, p_or_kw, *, kw):
def name(p1=None, p2, /, p_or_kw=None, *, kw):
def name(p1=None, p2, /):

Full grammar specification

A draft of the proposed grammar specification is:

new_typedargslist:
  tfpdef ['=' test] (',' tfpdef ['=' test])* ',' '/' [',' [typedargslist]] | typedargslist

new_varargslist:
  vfpdef ['=' test] (',' vfpdef ['=' test])* ',' '/' [',' [varargslist]] | varargslist

It will be added to the actual typedargslist and varargslist but for easier discussion it is presented as new_typedargslist and new_varargslist. Also, notice that using a construction with two new rules (new_varargslist and new_varargslist) is not possible with the current parser as the rule is not LL(1). This is the reason the rule needs to be included in the existing typedargslist and varargslist (in the same way keyword-only arguments were introduced).

Implementation

An initial implementation that passes the CPython test suite is available for evaluation [10].

The advantages of this implementation involve speed, consistency with the implementation of keyword-only parameters as in PEP 3102 and a simpler implementation of all the tools and modules that will be impacted by this change.

Rejected Ideas

Do Nothing

Always an option, just not adding it. It was considered though that the benefits of adding it is worth the complexity it adds to the language.

After marker proposal

A complaint against the proposal is the fact that the modifier of the signature impacts the tokens already passed.

This might make it confusing to users to read functions with many arguments. Example:

def really_bad_example_of_a_python_function(fist_long_argument, second_long_argument,
                                            third_long_argument, /):

It is not until reaching the end of the signature that the reader realizes the /, and therefore the fact that the arguments are position-only. This deviates from how the keyword-only marker works.

That said we could not find an implementation that would modify the arguments after the marker, as that will force the one before the marker to be position-only as well. Example:

def (x, y, /, z):

If we define that / makes only z position-only it won't be possible to call x and y via keyword argument. Finding a way to work around it will add confusion given that at the moment keyword arguments cannot be followed by positional arguments. / will therefore make both the preceding and following parameters position-only.

Per-argument marker

Using a per-argument marker might be an option as well. The approach basically adds a token to each of the arguments that are position only and requires those to be placed together. Example:

def (.arg1, .arg2, arg3):

Note the dot on arg1 and arg2. Even if this approach might look easier to read, it has been discarded as / goes further in line with the keyword-only approach and is less error-prone.

There are some libraries that use leading underscore[#leading-underscore]_ to mark those arguments as positional-only.

Using decorators

It has been suggested on python-ideas [9] to provide a decorator written in Python as an implementation for this feature. This approach has the advantage that keeps parameter declaration more easy to read but also introduces an asymmetry on how parameter behaviour is declared. Also, as the / syntax is already introduced for C functions, this inconsistency will make it more difficult to implement all tools and modules that deal with this syntax including but not limited to, the argument clinic, the inspect module and the ast module. Another disadvantage of this approach is that calling the decorated functions will be slower than the functions generated if the feature was implemented directly in C.

Source: https://github.com/python/peps/blob/master/pep-0570.txt