[scikit-learn] Is there any official position on PEP484/mypy?

[Daniel Moisset]

@Andreas, @Gael:

This indeed is something that could be included in the CI, and you could
ensure that the annotations have both internal consistency (i.e., what they
say matches what the implementation is doing) and external consistency (the
way callers are using it matches the way they call it).

To clarify a bit, there are 2 things involved here:

* PEP-484 provides a standard way to add anotations. Those have some syntax
but it's just metadata that gets stored but have no effect whatsoever on
runtime, kind of like a structured docstring. These have no protection by
themselves against bitrot (in the same way that docstrings may rot)
* mypy is a tool that can be run on a linter, it parses the code and the
annotations, and verify that there's consistency. It's something that you
can use on CI or whle developer (comparable to a linter like flake8, but
doing a deeper analysis).

The annotations described by PEP484 is "gradual" (you don't have to cover
the whole code, only the parts where static typing makes sense, and
"unchecked" code is not modified). mypy respects that and also provides a
way to silence the checker for situations where the type system is
oversensitive but you know you're right (similar to flake8's "# noqa").

@Sebastian

I had heard the podcast and it makes a very strong argument argument for
using it, thanks for recommending it (people in dropbox are using this on
their production codebase).

I do believe that end users will start getting benefits from this that are
stronger than docstrings, specially when this tooling starts to get
integrated in code editors (pycharm is already doing this) so they can get
inline checking and detection of errors when they call the scikit-learn
API, and better context-aware completion. That's not counting those users
that want to use mypy in their own codebases and would get a better
advantage if SKL supported it (that's the situation I am in, together with
some colleagues).

Regarding syntax, if we add inline annotations (which IMO is the best path
forward if they have a chance of getting integrated), the only option is
using the 2.x compatible annotations (which are comments). That one is
different to your 2 examples, that would be:

def hello(r, c=5):
    # type: (int, int) -> str
    s = 'hello'
    return '(%d + %d) times %s' % (r, c, s)

(note that your "  # type: str" is valid but not required, given that s can
be obviously inferred to be a string)

Another possible syntax (both are valid, this one makes sense for longer
signatures) is:

def hello(r,  # type: int
          c=5):
    # type: (...) -> str
    s = 'hello'
    return '(%d + %d) times %s' % (r, c, s)

(in this case there's again no need to specify a type for c given that it
can be inferred as an int)

These 2 variants work well in 2.x and 3.x

Best,
   D.

P.S.: In my last email I forgot to put this link describing some of the
things that I've found on real code
http://www.machinalis.com/blog/a-day-with-mypy-part-1/



On Thu, Jul 28, 2016 at 5:49 PM, Andreas Mueller <t3kcit at gmail.com> wrote:

>
>
> On 07/28/2016 12:43 PM, Gael Varoquaux wrote:
>
>> On Thu, Jul 28, 2016 at 12:04:48PM -0400, Andreas Mueller wrote:
>>
>>> If you find some bugs with the annotations and mypy, that would probably
>>> prove its value to some degree [and if you don't, I might be inclined to
>>> argue it's not working well ;]
>>> Joel, Olivier, Gael, anyone else?: opinions?
>>>
>> The only reserve that I might have is with regards to the maintainability
>> of these annotation. I am afraid that they coderot.
>>
>> Daniel, any comments on that concern?
>>
> We can put mypy in the CI, right? Shouldn't that prevent it from rotting?
> [I don't actually know. Daniel?]
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>



-- 
Daniel F. Moisset - UK Country Manager
www.machinalis.com
Skype: @dmoisset
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20160729/d70b80bc/attachment.html>