[Python-ideas] keyword arguments everywhere (stdlib) - issue8706

[Guido van Rossum]

On Sat, Mar 3, 2012 at 7:46 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Sun, Mar 4, 2012 at 6:39 AM, Guido van Rossum <guido at python.org> wrote:
>> Yeah, so it does make sense to standardize on a solution for this. Let
>> it be @positional(N). Can you file an issue?
>
> How could that even work?
>
> Consider the following subset of the Mapping API:
>
>    class C:
>
>        @positional(2):
>        def __init__(self, data=None, **kwds):
>            self._stored_data = stored = {}
>            if data is not None:
>                stored.update(data)
>            stored.update(kwds)
>
>        @positional(2):
>        def update(self, data=None, **kwds):
>            stored = self._stored_data
>            if data is not None:
>                stored.update(data)
>            stored.update(kwds)
>
> Without gross hacking of the function internals, there's no way for a
> decorator to make the following two calls work properly:
>
>    x = C(self=5, data=10)
>    x.update(self=10, data=5)

I am very well aware of this (it occurs in two different places in the
NDB library that I've been developing for Google App Engine).

But either I missed some messages in the thread (quite possible) or
you're bringing this up for the first time now -- the @positional
decorator wasn't meant to solve this case (which only occurs when
**kwds is used in this particular way).

*If* you want to solve this I agree that some actual new syntax is
probably needed.

> Both will complain about duplicate "self" and "data" arguments, unless
> the "positional" decorator truly rips the function definition apart
> and creates a new one that alters how the interpreter maps arguments
> to parameters.
>
> As Simon Sapin pointed out, the most correct way to write such code
> currently is to accept *args and unpack it manually, which is indeed
> exactly how the Mapping ABC implementation currently works [1]. While
> the Mapping implementation doesn't currently use it, one simple way to
> write such code is to use a *second* parameter binding step like this:
>
>    class C:
>
>        def _unpack_args(self, data=None):
>            return self, data
>
>        def __init__(*args, **kwds):
>            self, data = C._unpack_args(*args)
>            self._stored_data = stored = {}
>            if data:
>                stored.update(data)
>            stored.update(kwds)
>
>        def update(*args, **kwds):
>            self, data = C._unpack_args(*args)
>            stored = self._stored_data
>            if data is not None:
>                stored.update(data)
>            stored.update(kwds)

Nice; that idiom should be more widely known.

> The downside, of course, is that the error messages that come out of
> such a binding operation may be rather cryptic (which is why the
> Mapping ABC instead uses manual unpacking - so it can generate nice
> error messages)

Still, a naming convention for the helper function can probably make
this fairly painless -- perhaps you'll need a separate helper function
for each API function, named in a systematic fashion.

> The difficulty of implementing the Mapping ABC correctly in pure
> Python is the poster child for why the lack of positional-only
> argument syntax is a language wart - we define APIs (in C) that work
> that way, which people then have to reconstruct manually in Python.

Nobody else seems to have seen the importance of solving *this*
particular issue directly in the function signature -- but I
personally support trying!

> My proposal is that we simply added a *third* alternative for "*args":
> a full function parameter specification to be used to bind the
> positional-only arguments.
>
> That is:
>
> 1. '*args' collects the additional positional arguments and places
> them in a tuple
> 2. '*' disallows any further positional arguments.
> 3. '*(SPEC)' binds the additional positional arguments according to
> the parameter specification.
>
> In all 3 cases, any subsequent parameter definitions are keyword only.
>
> The one restriction placed on the nested SPEC is that it would only
> allow "*args" at the end. The keyword only argument and positional
> only argument forms would not be allowed, since they would make no
> sense (as all arguments to the inner parameter binding operation are
> positional by design).
>
> Then the "_unpack_args" hack above would be unnecessary, and you could
> just write:
>
>    class C:
>
>        def __init__(*(self, data=None), **kwds):
>            self._stored_data = stored = {}
>            if data:
>                stored.update(data)
>            stored.update(kwds)
>
>        def update(*(self, data=None), **kwds):
>            stored = self._stored_data
>            if data is not None:
>                stored.update(data)
>            stored.update(kwds)
>
> The objection was raised that this runs counter to the philosophy
> behind PEP 3113 (which removed tuple unpacking from function
> signatures). I disagree:
> - this is not tuple unpacking, it is parameter binding
> - it does not apply to arbitrary arguments, solely to the "extra
> arguments" parameter, which is guaranteed to be a tuple
> - it allows positional-only arguments to be clearly expressed in the
> function signature, allowing the *interpreter* to deal with the
> creation of nice error messages
> - it *improves* introspection, since the binding of positional only
> arguments is now expressed clearly in the function header (and
> associated additional metadata on the function object), rather than
> being hidden inside the function implementation

+1. This is certainly the most thorough solution for both problems at
hand (simply requiring some parameters to be positional, and the
specific issue when combining this with **kwds).

> Regards,
> Nick.
>
> [1] http://hg.python.org/cpython/file/e67b3a9bd2dc/Lib/collections/abc.py#l511

-- 
--Guido van Rossum (python.org/~guido)