[Python-Dev] typeshed for 3rd party packages

[Steven D'Aprano]

On Fri, Apr 24, 2015 at 03:44:45PM +0100, Cory Benfield wrote:
> On 24 April 2015 at 15:21, Steven D'Aprano <steve at pearwood.info> wrote:
> 
> > If the type hints are wrong, there are two errors: false positives, when
> > code which should be allowed is flagged as a type error; and false
> > negatives, when code which should be flagged as an error is not.
> > Ideally, there should be no false positives. But false negatives are not
> > so important, since you will still be doing runtime checks. All that
> > means is that the static type-checker will be a little less capable of
> > picking up type errors at compile time.
> 
> I think that's a rational view that will not be shared as widely as I'd like.

I can't tell if you are agreeing with me, or disagreeing. The above 
sentence seems to be agreeing with me, but you later end your message 
with "do it properly or not at all" which disagrees. So I'm confused.


> Given that the purpose of a type checker is to catch bugs caused by
> passing incorrectly typed objects to a function, it seems entirely
> reasonable to me to raise a bug against a type hint that allows code
> that was of an incorrect type where that incorrectness *could* have
> been caught by the type hint.

Of course it is reasonable for people to submit bug reports to do with 
the type hints. And it is also reasonable for the package maintainer to 
reject the bug report as "Won't Fix" if it makes the type hint too 
complex.

The beauty of gradual typing is that unlike Java or Haskell, you can 
choose to have as little or as much type checking as works for you. You 
don't have to satisfy the type checker over the entire program before 
the code will run, you only need check the parts you want to check.


> Extending from that into the general
> ratio of "reports that are actually bugs" versus "reports that are
> errors on the part of the reporter", I can assume that plenty of
> people will raise bug reports for incorrect cases as well.

Okay. Do you get many false positive bug reports for your tests too?


> From the perspective of sustainable long-term maintenance, I think the
> only way to do type hints is to have them be sufficiently exhaustive
> that a user would have to actively *try* to hit an edge case false
> negative. I believe that requests' API is too dynamically-typed to fit
> into that category at this time.

I think we agree that, static type checks or no static type checks, 
requests is going to need to do runtime type checks. So why does it 
matter if it misses a few type errors at compile time?

I think we're all in agreement that for extremely dynamic code like 
requests, you may not get as much value from static type checks as some 
other libraries or applications. You might even decide that you get no 
value at all. Okay, that's fine. I'm just suggesting that you don't have 
just two choices, "all or nothing". The whole point of gradual typing is 
to give developers more options.


> PS: I should mention that, as Gary Bernhardt pointed out at PyCon,
> people often believe (incorrectly) that types are a replacement for
> tests.

They *can* be a replacement for tests. You don't see Java or Haskell
programmers writing unit tests to check that their code never tries to 
add a string to a float. Even if they could write such as test, they 
don't bother because the type checker will catch that sort of error.

The situation in Python is a bit different, and as Antoine points out, 
libraries cannot rely on their callers obeying the type restrictions of 
the public API. (Private functions are different -- if you call my 
private function with the wrong type and blow up your computer, it's 
your own fault.) For libraries, I see type checks as complementing 
tests, not replacing them.

But for application code, type checks may replace unit tests, provided 
that nobody checks in production code until both the type checker and 
the unit tests pass. If you work under that rule, there's no point in 
having the unit tests check what the type checker already tested.


> For that reason I feel like underspecified type hints are
> something of an attractive nuisance. Again, I really think this is a
> case of do it properly or not at all.

In my opinion, underspecified type hints are no more of an attractive 
nuisance than a test suite which doesn't test enough. Full coverage is 
great, but 10% coverage is better than 5% coverage, which is better than 
nothing. That applies whether we are talking about tests, type checks, 
or documentation.


-- 
Steve