[Python-ideas] Why is design-by-contracts not widely adopted?

[Stephen J. Turnbull]

Marko Ristin-Kaufmann writes:

 > I annotated pathlib with contracts:
 > https://github.com/mristin/icontract-pathlib-poc. I zipped the HTML docs

Thank your for completing this task so quickly.

I'm sorry, but I didn't find it convincing.  I'll leave others to
discuss the docs, as they are clearly "proof of concept" and I expect
will be improved greatly.

Part of the problem is the style of contracts, which can probably be
improved with syntactic support.  For example, the many double
negatives of the form "not (not (X)) or Y".  I guess the idea is to
express all implications "X implies Y" in the equivalent form "not X
or Y".  I tried a few in the form "Y if X else True" but didn't find
that persuasive.  I also found the lambdas annoying.

I conclude that there's good reason to prefer the style where the
condition is expressed as a str, and eval'ed by the contract machinery.
This would get rid of the lambdas, and allow writing a contract parser
so you could write implications in a more natural form.  I haven't
tried rewriting that way, and of course, I don't have a parser to
actually run stuff.

 >    - not list(self.iterdir()) (??? There must be a way to check
 >      this more optimally)

You might define a function like this:

    def is_empty_dir(d):
        for _ in os.scandir(d):
            return False
        return True

or the function could catch the StopIteration from next() directly
(but I think that's considered bad style).  I don't think there's any
way to do this without a function call.  Note that the implementation
of iterdir calls os.listdir, so this function isn't necessarily much
more efficient than "not list(self.iterdir())".  (I'm pretty sure that
in this case the iterator keeps a copy of the list internally, and
listification probably involves object creation overhead, a memory
allocation and a machine-language block copy.)  I'm not sure whether
scandir is any better.  And I don't know how big a directory needs to
be before the overhead of function call and exception handling become
great enough to make it worthwhile.

 > [You] want to convey the message: dear user, if you are iterating
 > through a list of paths, use this function to decide if you should
 > call rmdir() or unlink(). Analogously with the first contract: dear
 > user, please check if the directory is empty before calling rmdir()
 > and this is what you need to call to actually check that.

I don't understand this logic.  I don't think I'd be looking at random
contracts for individual to learn how to handle filesystem objects.
I'd be more likely to do this kind of thing in most of my code:

    def rmtree(path: Path) -> None:
        try:
            path.unlink()
        except PermissionError:
            for p in path.iterdir():
                rmtree(p)
            path.rmdir()

I wrote all of that without being a Path user and without checking
docs.  (I cheated on PermissionError by testing in the interpreter,
I'd probably just use Exception if I didn't have an interpreter to
hand already.)

Note that this function is incorrect: PermissionError could occur
because I don't have write permission on the parent directory.  I also
don't learn anything about PermissionError from your code, so your
contract is incomplete.  *DbC is just not a guarantee of anything* --
if there's a guarantee, it derives from the quality of the development
organization that uses DbC.

 > I also finally assembled the puzzle. Most of you folks are all
 > older and were exposed to DbC in the 80ies championed by DbC
 > zealots who advertised it as *the *tool for software
 > development. You were repulsed [...].

Please don't make statements that require mind-reading.  You do not
know that (I for example am 60, and have *heard of* design-by-contract
before but this is the first time I've seen it *seriously proposed for
use* or implementation in any project I participate in).

More important, several posters have explained why they don't see a
need for DbC in Python.  Two common themes are proliferation of
conditions making both code and documentation hard to read, and
already using methods of similar power such as TDD or other systems
using unit tests.  It's bad form to discount such explicit statements.

 > And that's why I said that the libraries on pypi meant to be used by
 > multiple people and which already have type annotations would obviously
 > benefit from contracts -- while you were imagining that all of these
 > libraries need to be DbC'ed 100%, I was imagining something much more
 > humble. Thus the misunderstanding.

No, the misunderstanding is caused by your wording and by your lack of
attention to the costs of writing contracts.  Criticizing yourself is
a much more effective strategy: you can control your words, but not
how others understand them.  Your best bet is to choose words that are
hard to misunderstand.

 > Similarly with rmdir() -- "the directory must be empty" -- but how
 > exactly am I supposed to check that?

There's no reason to suppose the contract is a good place to look for
that.  "not list(path.iterdir())" is the obvious and easy-to-read test
(but in your style, a nonempty condition would be "not not
list(path.iterdir()", which is not not not the way to do it in code
;-).  But this may not be efficient for "large enough" directories.
However, if contracts are expected to be disabled in production code,
there's strong reason to write readable rather than efficient code for
contracts.

Please trim unneeded text.  Fixed it for you this time.