[Python-ideas] Specify an alternative exception for "assert"

[Steven D'Aprano]

On Mon, May 02, 2016 at 04:23:14PM +0200, Giampaolo Rodola' wrote:
> assert statement gives the possibility to display the text which goes along
> with the AssertionError exception. Most of the times though, what would be
> more appropriate is to raise a different exception (e.g. ValueError). My
> proposal is to be able to specify an exception as a replacement for
> AssertionError as in:
> 
> >>> assert callable(fun), ValueError("object is not a callable")
> ValueError: object is not a callable

My apologies for the length of this post. For the benefit of those in a 
hurry, here is the executive summary, or TL;DR version:

Strong -1 for this proposal.

Assertions have different semantics to explicit "if condition: raise" 
checks, and it is good and useful that they behave differently. In 
particular, a failed assert should always be a bug, and never an 
expected exception which the user may wish to catch. We should not blur 
the difference between an assert and an explict if...raise just for the 
sake of saving a line of code. If you find yourself wanting assert to 
raise a different exception type (like TypeError), that is a warning 
sign that you shouldn't be using assert: you're probably abusing 
assert for code code that needs the if...raise semantics, not the assert 
semantics.

Longer, and hopefully more articulate explanation (and hopefully not too 
rambling) follows:



I work with some people who insist on always using "assert" for all 
their error checking. ALL of it, whether of public or private functions, 
including end-user data validation. Because "it saves a line" and "it 
makes it easy to remember what exception to catch".[1] So their code is 
riddled with "try...except AssertionError", and heaven help us if the 
end user runs their code with -O. "It doesn't matter, nobody will do 
that." (Fortunately, so far the end users have not been savvy enough to 
know about -O.)

Assertions have a number of uses, such as checked comments, testing 
invariants, contract checking[2], etc. Some time ago, I wrote this to 
try to explain what I believe are good and bad uses of assert:

http://import-that.dreamwidth.org/676.html

I believe that assert (as opposed to an explicit test and raise) 
strongly communicates the intent of the programmer:

    assert condition, "error"

says that this is an internal check which (in a bug-free program) is 
safe to disable. The assertion can be thought of as a form of "trust, 
but verify". There are a few different interpretations of assert, but 
they all agree that *assertions are safe to remove* (provided the 
program is bug-free). Once you are confident that the assertions will 
never trigger, you can safely disable them, and your program should 
still work.

Whereas an explicit test and raise:

    if not condition: raise Exception("error")

strongly says that this is checking *external* input from a source that 
cannot be trusted (say, user supplied input, or external code that 
doesn't obey your code's internal contracts). Rather than "trust but 
verify" it is more "don't trust", and even in a bug-free program, you 
cannot do without this check. You can never disable these checks.

(By trust, I don't necessarily mean that the user or code is actively 
hostile and trying to subvert your function. I just mean that you cannot 
trust that it will provide valid input to your function. You MUST 
perform the check, even in a bug-free program.)

I think that the difference between those two sorts of checks is 
important, and I do not wish to see the distinction weakened or removed. 
I think that this proposal will weaken that distinction by allowing 
assert to raise non-AssertionError and encouraging people to treat it as 
a lazy shortcut for if...raise. I think it will make it harder to 
distinguish between program bugs and expected errors that can be caught. 
More on this below.

(At the risk of weakening my own argument, I acknowledge that there are 
grey areas where there may be legitimate difference of opinion whether a 
particular check is better as an assert or an if...raise. Sometimes it 
comes down to the programmer's opinion. But I think it is still 
important to keep the distinction, and not blur the two cases just for 
the sake of those grey areas, and especially not just to save a line and 
a few keystrokes.)


AssertionError strongly indicates an internal program bug (a logic 
error, a failed checked comment, a contract violation etc). 
Consequently, I should never need to catch AssertionError directly. 
AssertionError is always a failure of "trust, but verify" and therefore 
an internal bug. And importantly, no other exceptions should count as 
this sort of failure.

(I'm giving an ideal, of course, and in real life people vary in how 
rigourously they apply the ideals I describe. Some people seemingly 
choose exceptions arbitrarily. No doubt we all have to deal with badly 
chosen exceptions. But just because people will misuse exceptions no 
matter what we do, doesn't mean we should add syntax to make it easier 
to misuse them.)

For example, contract violations shouldn't raise ValueError, because 
then the caller might treat that contract violation (a bug) as an 
expected error and catch the exception. Same goes for invariants and 
checked comments:

    process(thelist)
    # if we get here, the list has at least two items
    assert len(thelist) >= 2, 'error'


The intent here is that AssertionError is *not* part of the function 
API, it should not be treated as an expected error which the caller can 
catch and deal with. AssertionError signals strongly "don't think about 
catching this, it's a bug that must be fixed, not an expected error".

If the failure *is* expected, then I ought to communicate that clearly 
by using an explicit if...raise with a different exception:

    process(thelist)
    # if we get here and the list has less than two items, that's an error
    if len(thelist) < 2, ValueError('error')


But with the proposed change, the code can send mixed messages:

    assert len(thelist) >= 2, ValueError('error')


The assert says that this can be safely discarded once the program is 
bug-free, i.e. that the exception should never be raised. But the 
ValueError says that the exception is expected and the test shouldn't be 
removed. If ValueError does get raised, is that a failed invariant, i.e. 
a bug? Or an expected error that the caller can and should deal with? 
Who can tell? Is it safe to disable that assertion? The intention of the 
programmer is harder to tell, and there are more ways to get it wrong.





[1] I admit it: sometimes I'm lazy and use assert this way too. We're 
all human. But never in production code.

[2] As in Design By Contract.

-- 
Steve