Python Sanity Proposal: Type Hinting Solution

[Steven D'Aprano]

Mario Figueiredo wrote:

> In article <4b3b498a-c9b0-443d-8514-87ccd8e98f43 at googlegroups.com>,
> rantingrickjohnson at gmail.com says...
>> 
>>     (Example modified for PEP8 compliance ;-)
>> 
>>     @typehint(arg1:str, arg2:int, returns:bool)
>>     def myfunction(arg1, arg2):
>>         return True
>>         
>> Of course "@typehint" could be an extension of the decorator
>> syntax, a keyword, a build-in function or whatever, i don't
>> care.
> 
> I'd rather it'd be a docstring parameter.
> 
> - Decorators are transformation tools. Which type hints are not.

But they can be. Python annotations are available at runtime, they are not
mere declarations for the compiler like in Pascal or C:

py> def spam(x:int, y:float)->float:
...     return x+y
...
py> spam.__annotations__
{'x': <class 'int'>, 'y': <class 'float'>, 'return': <class 'float'>}


They're writable too:

py> def eggs(x, y):
...     return x + y
...
py> eggs.__annotations__ = spam.__annotations__
py> eggs.__annotations__['x']
<class 'int'>


So we can write a transformation tool which adds, modifies or deletes
annotations:

@annotate(x=int, y=float)  # return is a little tricky...
def eggs(x, y):
    return x+y

eggs.__annotations__['x']
# returns int


Except we don't need such a tool. Python 3 already has syntax for it, and it
supports annotating the return result too. Annotation syntax is just
syntactic sugar for a transformation.


> - keywords should be few and used only for language features. Static
> analysis isn't and should never be a language feature. It's an
> extension, perhaps.

I'm not sure if you're making a general observation or one which is specific
to Python. Plenty of languages have static analysis as a language feature.
Are you arguing they are wrong to do so?

In the case of Python, there are no plans for CPython to include static
analysis, or for it to be general language feature. Other implementations,
such as MyPy, and other tools, are intended to perform the analysis, not
the CPython reference implementation.


> - Same with built-in functions... although a case could be made for a
> static analysis package, hmmm.

I'm not sure what you mean here.



> So I'd rather see:
> 
>      def myfunction(arg1, arg2):
>      """
>      Normal docstring.
>      """
>      "@typehint: (str, int) -> bool"
>          return True

By the rules of implicit (compile-time) concatenation of string literals,
that is equivalent to:

def myfunction(arg1, arg2):
    """
    Normal docstring.
    @typehint: (str, int) -> bool"""
    return True

One of the problems with this is that it put the information about
parameters far away from the parameter list itself. Most docstrings are
considerable more than a single line:

py> from random import sample
py> len(sample.__doc__.split('\n'))
16


I consider 16 lines a short docstring! I've written docstrings which are 40
or 50 lines long, so potentially the annotations are almost a full page
away from the function signature.

The usual convention among document generator tools is that parameter
metadata is at the start of the docstring, not the end:

def myfunction(arg1, arg2):
    """
    One line summary.

    @param arg1: str
    @param arg2: int
    @returns bool

    Longer discussion of the function.
    """
    return True

I'm not keen on that, but at least it puts the types close to the function
signature, and it's a defacto standard.


> I removed the arguments names on purpose. They are only necessary on the
> PEP because type hinting is a part of the function header there.
> However, when using a documentation like pattern as above (or as in your
> own example), they can be safely removed, with the added benefit of
> making the syntax simpler.

Only at the cost of making it hard to read. I go to look up a function which
I am unfamiliar with, and find the function signature:

def open(file, mode='r', buffering=-1, encoding=None,
        errors=None, newline=None, closefd=True, opener=None):
    """Open a file.

    blah blah blah blah...
    thirty lines of documentation...


My documentation view shows only the first thirty lines, so I hit the Page
Down key and see:

    blah blah blah blah...
    thirty lines of documentation...
    
I page down again:

    blah blah blah...
    @typehint: (str, str, int, Optional[str], Optional[str], 
                Optional[str], bool, Callable)


Well that's rather less than useful. Which argument is newline again? Number
3 or 4 or 5?

Syntax which appears reasonable with toy examples (one-line docstring, two
arguments) is less reasonable with real-world cases (docstrings can be a
page or more in length, there may be five or a dozen arguments to a
function).


> Alternatively, because dangling strings are always considered
> documentation and completely ignored by the interpreter (PEP 258), one
> could also do:
> 
>      "@typehint: (str, int) -> bool"
>      def myfunction(arg1, arg2):

That's somewhat better, in that at least the hint is close to the function
signature. But it has a lot of disadvantages: it is compile-time only, the
type hints aren't available at runtime. It requires extra complexity to the
parser, so that decorators may be separated from the function by a hint:

@decorate
"@typehint: (str, int) -> bool"
def myfunction(arg1, arg2):


No doubt some people will get them the wrong way around, and the type
checker may silently ignore their hints:

"@typehint: (str, int) -> bool"
@decorate
def myfunction(arg1, arg2):

And others will write:

@decorate
@typehint(str, int) -> bool
def myfunction(arg1, arg2):


and be annoyed or perplexed by the syntax error.

Some syntax will be a bug magnet. This is one.


-- 
Steven