[Python-ideas] kwargs for return

[Cameron Simpson]

On 27Jan2019 02:30, Steven D'Aprano <steve at pearwood.info> wrote:
>On Sat, Jan 26, 2019 at 03:29:59PM +0100, Anders Hovmöller wrote:
>> > I don't see anything here that can't be done by returning a dict, a
>> > namedtuple (possibly with optional fields), or some other object with
>> > named fields. They can be optional, they can have defaults, and you can
>> > extend the object by adding new fields without breaking backwards
>> > compatibility.
>>
>> That assumes you knew before hand to do that. The question is about
>> the normal situation when you didn't.
>
>Exactly the same can be said about the given scenario with or
>without this hypthetical "kwargs for return".
>
>Thomas talks about having to change a bunch of backends. Okay, but he
>still has to change them to use "kwargs for return" because they're not
>using them yet. So there is no difference here.

I don't think so. It looks to me like Thomas' idea is to offer a 
facility a little like **kw in function, but for assignment.

So in his case, he wants to have one backend start returning a richer 
result _without_ bringing all the other backends up to that level. This 
is particularly salient when "the other backends" includes third party 
plugin facilities, where Thomas (or you or I) cannot update their 
source.

So, he wants to converse of changing a function which previously was 
like:

  def f(a, b):

into:

  def f(a, b, **kw):

In Python you can freely do this without changing _any_ of the places 
calling your function.

So, for assignment he's got:

  result = backend.foo()

and he's like to go to something like:

  result, **kw = richer_backend.foo()

while still letting the older less rich backends be used in the same 
assignment.

>The point is, you can future-proof your API *right now*, today, without
>waiting for "kwargs for return" to be added to Python 3.8 or 3.9 or
>5000. Return a dict or some object with named fields. [...]

Sure, but Thomas' scenario is where nonfutureproof API is already in the 
wild.

>> Also you totally disregarded the call site where there is no way to 
>> do a nice dict unpacking in python.
>
>It wasn't clear to me that Thomas is talking about dict unpacking. It
>still isn't. He makes the analogy with passing keyword arguments to a
>function where they are collected in a **kwargs dict. That parameter
>isn't automatically unpacked, you get a dict.

Yeah, but with a function call, not only do you not need to unpack it at 
the function receiving end, you don't even need to _supply_ it at the 
calling end, and you can still use **kw at the function receiving end; 
it will simply be empty.

>So I expect that "kwargs
>for return" should work the same way: it returns a dict. If you want to
>unpack it, you can unpack it yourself in anyway you see fit.

Yeah. Or even *a. Like this:

    Python 3.6.8 (default, Dec 30 2018, 12:58:01)
    [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> def f(): return 3
    ...
    >>> def f2(): return 3, 4
    ...
    >>> *x = f()
      File "<stdin>", line 1
    SyntaxError: starred assignment target must be in a list or tuple
    >>> a, *x = f()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: 'int' object is not iterable
    >>> a, *x = f2()

Of course this can't work out of the box with current Python because 
argument unpacking expected the right hand side to be a single 
unpackable entity, and:

    a, *x = ...

is just choosing to unpack only the first of the values, dropping the 
rest into x. Of course a **kw analogue is better because it lets one 
associate names with values.

In terms of syntax, we can't go with:

  a, *x = ...

because precedence lets us write "bare" tuples:

  a, *x = 1, 2, 3

so the right hand side isn't 3 distinct expressions, it is one tuple 
(yes, made of 3 expressions) and it is the left side choosing to unpack 
it directly.

However,

  a, **kw = ...

is an outright syntax error, leaving a convenient syntactic hole to 
provide Thomas' notion. In current syntax, the right hand side remains a 
single expression, and kw will always be an empty dict. The tricky bit 
isn't the left side, it is what to provide on the right.

Idea: what if **kw mean to unpack RHS.__dict__ (for ordinary objects) 
i.e. to be filled in with the attributes of the RHS expression value.

So, Thomas' old API:

    def foo():
      return 3

and:

  a, **kw = foo()

get a=3 and kw={}. But the richer API:

  class Richness(int):

    def __init__(self, value):
      super().__int__(value)
      self.x = 'x!'
      self.y = 4

  def foo_rich():
    return Richness(3)

  a, **kw = foo_rich()

gets a=3 and kw={'x': 'x!', 'y': 4}.

I've got mixed mfeelings about this, but it does supply the kind of 
mechanism he seems to be thinking about.

Cheers,
Cameron Simpson <cs at cskk.id.au>