[Python-ideas] Proposal: Use mypy syntax for function annotations

[Steven D'Aprano]

On Fri, Aug 22, 2014 at 10:31:18PM -0400, Antoine Pitrou wrote:

> >>Python has one of the most powerful and user-friendly function call
> >>syntaxes around, why reinvent something clearly inferior and alien?
> >
> >I wouldn't say it is alien. abc[xyz] is clearly Python syntax.
> 
> But it is completely uncommon for anything else than subscripting and 
> indexing containers. 

And function call syntax is completely uncommon for anything else than 
calling functions (including types and methods). One way or the other, 
we're adding a new use, type declarations.


> Python isn't known for reusing operators for 
> completely unrelated things, e.g. it doesn't overload ">>" for I/O 
> (thank goodness).

I see you've forgotten Python 2's 

    print >>sys.stderr, x, y, z

:-)


> >How complex will the static type declarations
> >need to be for us to want to accept arbitrary keyword arguments, for
> >example? I think limiting the syntax possible is a good thing, it will
> >discourage people from trying to declare arbitrarily complex information
> >in the type notations.
> 
> I think this is an incredibly weak argument, and the proof is that it 
> hasn't been brought for anything else in the stdlib. 

We frequently reject syntax because it will be (allegedly) confusing or 
hard to read. E.g. try...except expressions, anonymous code blocks, 
multiline lambda. Especially multiline lambda. Even something which I 
think is a no-brainer, allowing the with statement to use parentheses 
around the context managers:

    with (this_manager() as m1,
          that_manager() as m2):
        ...

was criticised by Nick on the basis that if you can't fit the context 
managers all on one line, you ought to refactor your code. I'm sure if 
you check the rejected PEPs and Python-Ideas archives, you'll find 
many examples of my argument applied to the standard library.

I don't think it's unPythonic to argue that type hinting syntax should 
by default use an annotation style which is not arbitrarily complex. The 
annotations themselves will remain arbitrary Python expressions, so if 
you really need a complex type declaration, you can create a helper and 
call it with whatever signature you like.[1]


> Why the typing 
> module should be any different and use a deliberately repressive 
> parameter-passing syntax is beyond me - even though it's an advanced, 
> optional, feature and will be put in the hands of consenting adults.

I can't speak for the author of mypy, Jukka Lehtosalo, but for me, I 
think the answer is that type declarations make a *mini-language*, not 
full-blown Python. It is unclear to me just how powerful the type 
language will be, but surely we don't expect it to evaluate arbitrarily 
complex Python expressions at compile time? Do we?

I find it unlikely that we expect a static linter to make sense of this:

def bizarre(param:int if random.random() > 0.5 else str)->float:

except to say it returns a float and takes who-knows-what as argument. 
An extreme case, I grant, but I expect that there are going to be limits 
to what can be checked at compile time based on static analysis. I admit 
that I don't fully understand all the implications of static versus 
runtime type checking, but it doesn't seem unreasonable to expect static 
declarations to be considerably simpler than what can be expressed at 
runtime. Hence, a simpler mini-language is appropriate.


> Really, for it to be powerful enough, the type description system has to 
> use its own set of classes. It can't try to hack away existing builtins 
> and ABCs in the hope of expressing different things with them, or it 
> *will* hit a wall some day (and, IMO, sooner rather than later).

How powerful is "powerful enough"? Powerful enough for what?

You've probably heard of the famous "the compiler found my infinite 
loop", where the ML type checker was able to detect that code would 
never terminate. I find that remarkable, and even more astonishing that, 
according to some, solving the halting problem is "nothing special" for 
type systems.[2] But many languages make do with less powerful type systems, 
and tools such as IDEs and editors surely don't need something that 
powerful. So:

- how powerful do we expect the type system to be?

- and how much of that power needs to be expressed using 
  function annotations?

The second question is critical, because there are alternatives to 
function annotations: decorators, docstring annotations, and external 
stub files.




[1] Although if you do so, it is not clear to me how much the static 
checker will be able to use it.


[2] http://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/


-- 
Steven