[Python-ideas] Pre-conditions and post-conditions
Marko Ristin-Kaufmann
marko.ristin at gmail.com
Tue Aug 28 01:46:02 EDT 2018
Hi,
To clarify the benefits of the contracts, let me give you an example from
our code base:
@icontract.pre(lambda x: x >= 0)
@icontract.pre(lambda y: y >= 0)
@icontract.pre(lambda width: width >= 0)
@icontract.pre(lambda height: height >= 0)
@icontract.pre(lambda x, width, img: x + width <= pqry.opencv.width_of(img))
@icontract.pre(lambda y, height, img: y + height <= pqry.opencv.height_of(img))
@icontract.post(lambda self: (self.x, self.y) in self)
@icontract.post(lambda self: (self.x + self.width - 1, self.y +
self.height - 1) in self)
@icontract.post(lambda self: (self.x + self.width, self.y +
self.height) not in self)
def __init__(self, img: np.ndarray, x: int, y: int, width: int,
height: int) -> None:
self.img = img[y:y + height, x:x + width].copy()
self.x = x
self.y = y
self.width = width
self.height = height
def __contains__(self, xy: Tuple[int, int]) -> bool:
x, y = xy
return self.x <= x < self.x + self.width and \
self.y <= y < self.y + self.height
We use mypy and type annotations for typing, and we don't need contracts
for that. In this particular case, contracts are very handy to formulate
how we deal with pixels. If you carefully look at the contracts, you will
see:
* pixels are indexed starting from 0 (as oppose to starting from 1)
* region-of-interest needs to fit within an image. It can not be outside of
its boundaries
* pixels within the region-of-interest are in [x, x + width), [y, y +
height) (where "[" means inclusive and ")" exclusive)
*Why not types?* These constraints are either impossible or very hard to
write as types. Moreover, I doubt that you could give such types meaningful
names, and I expect the readability to suffer immensely. I suppose any
constraints involving more than a couple of variables is hard to write as a
type.
*Benefits.* You could write all of this in human-readable informal
docstring of a class, but then it would never be verified. This way we are
sure that contracts are always there. Whenever we test, we can be sure that
all the contracted properties hold. Mind that we do not test that they hold
for that single test case -- we automatically test that they hold for all
the test cases.
This is, in my opinion, far superior to ambiguous human documentation or
print statements (which are deleted after the debugging). I suppose *any*
larger code base built by a team of more than one developer needs this kind
of rigor. The contracts are not meant to be used only in high-risk
applications -- they are meant to improve any code base.
*Problem without standard support. *The current libraries (dpcontracts,
icontract) can deal with pre and postconditions and class invariants of a
concrete class.
However, the current libraries break as soon as you have inheritance. There
is no way in python to inherit the function decorators and to modify them.
If we are to inherit from the above-mentioned class ROI, we need to make
sure that the invariants of the parent class hold (*e.g., *what is
contained in a ROI) as well as invariants of the child class. We might want
to weaken the requirements (*e.g., *a ROI that can deal with pixels outside
of an image) such that x and y can be an arbitrary numbers, not restricted
to 0 <= x < img.width and 0 <= y < img.height respectively.
Additionally, without standard approach, we do not know how to deal with
contracts when we have a third-party tool that would like to use them (*e.g.,
*for automatic generation of unit tests, static testing or visualization in
IDE or in documentation).
@Wes Turner <wes.turner at gmail.com>: If I understood you correctly, you are
looking for a library that gives you verbose messages when a contract is
breached. Please have a look at icontract library:
https://github.com/Parquery/icontract
We added informative messages particularly because we wanted to have
verbose output when something goes wrong.This was helpful not only during
the development, but also in production since failure cases were hard to
reproduce and anticipate in the first place (otherwise, they wouldn't have
made it into the production). Here is an example with an error message:
>>> class B:... def __init__(self) -> None:... self.x = 7...... def y(self) -> int:... return 2...... def __repr__(self) -> str:... return "instance of B"...>>> class A:... def __init__(self)->None:... self.b = B()...... def __repr__(self) -> str:... return "instance of A"...>>> SOME_GLOBAL_VAR = 13>>> @icontract.pre(lambda a: a.b.x + a.b.y() > SOME_GLOBAL_VAR)... def some_func(a: A) -> None:... pass...>>> an_a = A()>>> some_func(an_a)
Traceback (most recent call last):
...
icontract.ViolationError: Precondition violated: (a.b.x + a.b.y()) >
SOME_GLOBAL_VAR:SOME_GLOBAL_VAR was 13
a was instance of A
a.b was instance of B
a.b.x was 7
a.b.y() was 2
On Tue, 28 Aug 2018 at 03:19, Wes Turner <wes.turner at gmail.com> wrote:
> Thanks for the explanation.
>
> This may be a bit OT,
> but is there a good way to do runtime assertions (in particular for data
> validation) that's as easy as assert?
>
> - self.assertEqual (unittest.TestCase.assertEqual) is hardly usable in
> other class instances,
> - pytest can be used at runtime but has somewhat nontrivial overhead
> - I always end up writing e.g. _assert() and _assertEqual() that throw
> AssertionErrors and helpful error messages just like unittest.
>
>
> How do contracts differ from checking interfaces with e.g. zope.interface?
> https://zopeinterface.readthedocs.io/en/latest/verify.html
>
> I'll read up a bit more on design by contract. I seem to have
> conceptualized dbc as a composition approach with typed interfaces and type
> and value assertions, but that's not it?
>
> If the parameter value assertions in pycontracts aren't really contracts,
> and the mypy compile-time parameter and return type checks are not really
> contracts, and zope.interface verifications aren't really contracts; what
> does that leave in terms of abstract data types, preconditions,
> postconditions, and invariants?
>
> https://en.wikipedia.org/wiki/Design_by_contract
>
> On Monday, August 27, 2018, Steven D'Aprano <steve at pearwood.info > wrote:
>
>> On Mon, Aug 27, 2018 at 11:01:21AM -0400, Wes Turner wrote:
>> > Runtime checks: data validation & code validation
>> > Compile-time checks: code validation
>> >
>> > What sort of data validation is appropriate for assert statements or
>> > contacts that may be skipped due to trading performance for more risk
>> > ('optimized out')?
>>
>> That depends on what you mean by "data validation".
>>
>> Testing for bad input, or testing for predictable error states such as
>> I/O errors, missing files, permission errors, server down etc is
>> not appropriate for assertions (which includes contracts).
>>
>> The rule I use is that assertions are for:
>>
>> (1) testing your program state, which is under your control; and
>>
>> (2) communicating the intention of your program as executable
>> code rather than comments.
>>
>> The ultimate aim of contracts and assertions is to eventually disable
>> them when the program is bug-free.
>>
>> The Eiffel docs say:
>>
>> It should be clear from the preceding discussion that contracts
>> are not a mechanism to test for special conditions, for example
>> erroneous user input. For that purpose, the usual control
>> structures [...] are available [...] An assertion is instead a
>> correctness condition governing the relationship between two
>> software modules (not a software module and a human, or a
>> software module and an external device). ... Bluntly:
>>
>> Rule -- Assertion Violation: A run-time assertion violation is
>> the manifestation of a bug.
>>
>>
>> https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Assertions_and_Exceptions
>>
>> For detecting *external* error states (anything to do with data that
>> comes from outside your program, like user input) you can never let your
>> guard down and never disable the test, because servers can always go
>> down, users can always give you bad data, files can always be corrupt.
>> It is unsafe to disable these tests and so these should not be
>> assertions.
>>
>> For a library function intended to be called by third-parties, the
>> function arguments aren't under the control of the library author so
>> should not be tested with assertions. But for an application where the
>> author controls those function arguments, they are under the author's
>> control and may be assertions or contracts.
>>
>> Design By Contract is partly a methodology and partly a set of syntax.
>> Its a way of thinking about the design of your program. In practice, you
>> don't have to buy 100% into DBC to get some benefit for it. A study done
>> a few years back looked at 21 large projects in Eiffel, JML (Java) and
>> Code Contracts for C# and found that 33% of the classes used contracts.
>>
>> http://se.ethz.ch/~meyer/publications/methodology/contract_analysis.pdf
>>
>> Like unit testing, you don't need 100% coverage to get benefit. 10% is
>> better than nothing, and 20% is better than 10%.
>>
>> I wrote more about assertions here:
>>
>> https://import-that.dreamwidth.org/676.html
>>
>>
>> --
>> Steve
>> _______________________________________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20180828/ca7b963c/attachment-0001.html>
More information about the Python-ideas
mailing list