[Python-ideas] Pre-conditions and post-conditions

[Marko Ristin-Kaufmann]

Hi,

@David Mertz, @Paul Moore: first of all, thank you very much for your
thorough answers!

@Paul Moore <p.f.moore at gmail.com>: Thanks in particular for suggesting a
road map. I still think that we need somewhat broader agreement even on
this list that contracts are useful at an abstract level before we dig in
and find concrete examples in existing code bases, have a more detailed
estimation about the costs and how much conflicts the new keyword would
cause, analyze how hard it would be to teach the concepts *etc.* So far, my
impression is that the majority rejects outright the idea that the
design-by-contract is useful and actually sees it as a redundant approach
to software correctness at best, and an useless concept at worst.

Of course, it would be much more convincing if I already performed a much
more thorough analysis as you suggested. Unfortunately, I don't have the
eloquence nor the resources to accomplish that at this moment. I'd imagine
such an analysis to be a collaborative project of multiple interested
people. I'd find it a futile effort before we have at least a vague
agreement here that design-by-contract would be useful.

Please let me address here the concrete points you raised in your message.

1. How exactly do these differ from simple assertions? I don't
> understand the hints about "relaxing" and "strengthening" contracts in
> subclasses, in particular I've no idea how I'd express that in actual
> syntax.
>

The contracts to not apply only to a concrete function, but also extend to
inherited methods.

Let me make an illustration on a stripped-down example of three classes A,
B and C, where B and C inherit from A.

class A:
    # A.some_func operates on sorted lists of integers and
    # it returns an integer larger than the length of
    # the input list.
    @icontract.pre(lambda lst: lst == sorted(lst))
    @icontract.post(lambda result: result > len(lst))
    def some_func(self, lst: List[int]) -> int:
        # ...

class B(A):
    # B.some_func inherits contracts from A, but overrides the implementation ->
    #   B.some_func also operates only on sorted lists of integers
    #   and needs to return an integer larger than the length
    #   of the input list.
    def some_func(self, lst: List[int]) -> int:
        # ...


class C(A):
    # C.some_func also inherits the contracts from A.
    #   It weakens the precondition:
    #       it operates either on sorted lists OR
    #           the lists that are shorter than 10 elements.
    #
    #  It strenghthens the postcondition:
    #       It needs to return an integer larger than
    #       the length of the input list AND
    #           the result needs to be divisible by 2.
    @icontract.post(lambda result: result % 2 == 0)
    def some_func(self, lst: List[int]) -> int:
        # ...

Assume that the function A.some_func needs sorted integers as input (*e.g.*
because it uses binary search).

Now, class B inherits from A -- say B.some_func also uses binary search.

The things get really interesting in the case of class C and its method
some_func: to satisfy polymorphism, C.some_func needs to work in all cases
where an instance of A could work as well. However, it can do more -- it
can also operate in situations where the input list has few elements, but
is not necessarily sorted. This is called "weakening" or "relaxing" of a
contract.

The "strengthening" or "tightening" of a contract is an analogous concept
for the post-conditions. Due to polymorphism, C.some_func needs to return a
result that satisfies all the postconditions of its ancestors so that we
can safely use it in all the situations where we would also use an instance
of the ancestor class. But we can strengthen it in C.some_func: it needs to
return the result satisfying the ancestors' postcondition and than be
constrained some more.

I omitted the invariants for brevity. They behave the same as the
post-conditions: C needs to satisfy all its own invariants as well as all
the invariants of its ancestor classes (A in this case).

2. Are we talking here about adding checks that would run in
> production code? Wouldn't that slow code down? How do I deal with the
> conflict between wanting tighter checks but not wanting to pay the
> cost at runtime of checking things that should never go wrong
> (contracts, as I understand it, are for protecting against code bugs,
> that's why there's no facility for trapping them and producing "user
> friendly" error messages)? I use assertions very sparingly in
> production code for precisely that reason - why would I be more
> willing to use contracts?


There is no way you can use contracts without slowing down the program (as
they still needs to be verified at run-time). People who need the
production code to run as fast as possible need to disable the contracts
(and assertions) altogether. If contracts were fast to be verified (for
example, by in-lining the checks into the function body instead of calling
a separate function), the cost of many contracts might be still acceptable
in the production code. Mind that there is also a gradation: there might be
contracts which are simply always too slow to run in production (*e.g., *check
that the input or output is sorted) while others are almost always
acceptable (*e.g., *check that the input arguments are positive integers or
one argument is smaller than the other). There is usually a switch
mechanism that allows the developer to turn off certain contracts while
still enforcing the others.

Please mind that the benefits of the contracts are threefold and are not
only reduced to "better assertions":
* They allow us to formally write down the assumptions about the input and
output of the functions and invariants of the data structures.
* They verify these assumptions automatically in all possible cases at
run-time (be it during the development or during the production). A smaller
set of unit tests hence tests a larger portion of the code then if there
were no contracts.
* They can be used by downstream tools to automatically generate unit tests
and perform static analysis. Imagine if you could just click on a class in
Pycharm and generate unit tests for all of its methods automatically within
couple of seconds. While these might not cover every use case, this feature
would allow you to cover many cases with practically zero work. I don't see
how such a tool could be developed without a standardized approach to
contracts in python (be it a library or a language construct).

@David Mertz:

> Related to the text I emphasized, would you mind to explain a bit more
>> in-depth which features you have in mind? I see contracts formally written
>> out and automatically verified as a completely indispensable tool in large
>> projects with multiple people involved.
>>
>
> This much is rather self-evidently untrue.  There are hundreds or
> thousands of large projects involving multiple people, written in Python,
> that have succeeded without using contracts.  That suggests the feature is
> not "completely indispensable" since it was dispensed with in the vast
> majority of large projects.
>

Sorry, David, my bad; English is not my first language. What I meant to say
is that I, as in "personally", could not see how I would tackle a large
project with a bigger team with a variety of skill levels in efficient
manner without a tool to formally write down the assumptions and have them
automatically verified. I do understand that achieving large projects is
evidently possible without contracts (and many other tools such as static
type checks, assertions, and even unit testing). In my case, I would have a
much higher overhead working in a team where the assumptions are written in
the documentation and never verified since many bugs and obsolete
documentation would just go unnoticed. Doing refactoring or adding new
features would be hence much more costly for my team and me.

Maybe this is also due to the fact that you refer to frameworks when you
refer to "large projects" with many experienced and highly skilled
contributors. The developers are most probably keen to update the
documentation and thoroughly test the framework and also test the
assumptions in combinatorial manner. In contrast, I work on a (larger)
project where the comments actually become obsolete almost at the moment we
wrote them and the testing budget is limited due to time constraints and
non-critical application domain. At present, we are not developing a
framework used by many other developers, but a solution based on computer
vision. We can live with incorrect code, but it's good that it is as
correct as it gets.

Personally, I view contracts as a  way to improve substantially the
correctness of the code with very little overhead. In addition, contracts
seem to lead to better and more maintainable code since people actually
need to reflect more on the assumptions and write them down formally. This
is, of course, a completely subjective impression based on my anecdotal
experience.

Let me see if I can make the inheritance work with meta-programming and
extra magical members (__preconditions__, __postconditions__,
__invariants__). Let's consider the icontract library a test case, and then
see how much more work would it be to bring it into the standard library.

@Steve D'Aprano: could you maybe point us to a couple of studies showing
that design-by-contract is beneficial in practice?

Cheers,
Marko

On Thu, 30 Aug 2018 at 13:22, David Mertz <mertz at gnosis.cx> wrote:

> On Thu, Aug 30, 2018 at 3:44 AM Marko Ristin-Kaufmann <
> marko.ristin at gmail.com> wrote:
>
>> Related to the text I emphasized, would you mind to explain a bit more
>> in-depth which features you have in mind? I see contracts formally written
>> out and automatically verified as a completely indispensable tool in large
>> projects with multiple people involved.
>>
>
> This much is rather self-evidently untrue.  There are hundreds or
> thousands of large projects involving multiple people, written in Python,
> that have succeeded without using contracts.  That suggests the feature is
> not "completely indispensable" since it was dispensed with in the vast
> majority of large projects.
>
> I think that most of these large projects succeed in large part because
> they have good *unit tests*.  There are several popular frameworks for
> writing these (some in standard library), but notably none of them require
> specific syntax changes to make them work.  Some of them *do* use DSLs of
> sorts as part of how they operate (or various metaprogramming and
> introspection magic).  There is a whole lot of overlap between what unit
> tests do and what design-by-contract does, enough so that I believe the
> latter adds little to a large project (of course you can come up with some
> specific example that a unit test cannot verify as well as a
> pre/postcondition.
>
> In writing before about "features" (and Paul Moore) does a better job than
> me in writing about *costs*, I wasn't discussing design-by-contract
> specifically.  Various new feature ideas come up here and elsewhere.  A few
> are accepted, most are rejected.  They all need to be compared to costs
> like those I mention before their possible advantages can win out.
>
> Yours, David...
>
> --
> Keeping medicines from the bloodstreams of the sick; food
> from the bellies of the hungry; books from the hands of the
> uneducated; technology from the underdeveloped; and putting
> advocates of freedom in prisons.  Intellectual property is
> to the 21st century what the slave trade was to the 16th.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20180830/0237844e/attachment-0001.html>