Is there a more elegant way to handle determing fail status?

[donarb]

On Tuesday, January 15, 2013 3:24:44 PM UTC-8, J wrote:
> Ok, so I have a diagnostic tool, written by someone else. That tool
> 
> runs a series of small tests defined by the user and can simplified
> 
> summary output that can be one of the following:
> 
> 
> 
> FAILED_CRITICAL
> 
> FAILED_HIGH
> 
> FAILED_MEDIUM
> 
> FAILED_LOW
> 
> PASSED
> 
> 
> 
> I also have a wrapper script I wrote to run these tests, summarize the
> 
> results of all tests aggregated and then fail based on a particular
> 
> fail level.
> 
> 
> 
> The idea is that if I run 3 tests with the diagnostic tool and it
> 
> tells me the following:
> 
> 
> 
> testA: PASSED
> 
> testB: FAILED_MEDIUM
> 
> testC: PASSED
> 
> 
> 
> AND I told the wrapper to only fail on HIGH or above, the wrapper will
> 
> tell me that I had a medium failure, but the wrapper will still exit
> 
> with a 0 (success)
> 
> 
> 
> if I get the same results as above, but tell the wrapper to fail on
> 
> LOW, then it will tell me I had that medium failure, but the wrapper
> 
> will exit with a 1 (failure).
> 
> 
> 
> The problem is that my exit determination looks like this:
> 
> 
> 
>     if fail_priority == fail_levels['FAILED_CRITICAL']:
> 
>         if critical_fails:
> 
>             return 1
> 
>     if fail_priority == fail_levels['FAILED_HIGH']:
> 
>         if critical_fails or high_fails:
> 
>             return 1
> 
>     if fail_priority == fail_levels['FAILED_MEDIUM']:
> 
>         if critical_fails or high_fails or medium_fails:
> 
>             return 1
> 
>     if fail_priority == fail_levels['FAILED_LOW']:
> 
>         if critical_fails or high_fails or medium_fails or low_fails:
> 
>             return 1
> 
> 
> 
>     return 0
> 
> 
> 
> So, to explain the above... the fail level can be set by the user when
> 
> running the wrapper using -f (or it defaults to 'high')
> 
> the wrapper assigns a number to each level using this:
> 
> 
> 
>     # Set correct fail level
> 
>     args.fail_level = 'FAILED_%s' % args.fail_level.upper()
> 
> 
> 
>     # Get our failure priority and create the priority values
> 
>     fail_levels = {'FAILED_CRITICAL':4,
> 
>                    'FAILED_HIGH':3,
> 
>                    'FAILED_MEDIUM':2,
> 
>                    'FAILED_LOW':1}
> 
>     fail_priority = fail_levels[args.fail_level]
> 
> 
> 
> the variables critical_fails, high_fails, medium_fails, low_fails are
> 
> all counters that are etiher None, or the number of tests that were
> 
> failed.
> 
> 
> 
> So using this output from the diagnostic tool:
> 
> 
> 
> testA: PASSED
> 
> testB: FAILED_HIGH
> 
> testC: PASSED
> 
> testD: FAILED_MEDIUM
> 
> testE: PASSED
> 
> 
> 
> critical_fails would be None
> 
> high_fails would be 1
> 
> medium_fails would be 1
> 
> low_fails would be None.
> 
> 
> 
> The exit code determination above works, but it just feels inelegant.
> 
> It feels like there's a better way of implementing that, but I can't
> 
> come up with one that still honors the fail level properly (e.g. other
> 
> solutions will fail on medium, but won't fail properly on medium OR
> 
> higher).
> 
> 
> 
> I can provide the full script if necessary, if the above isn't enough
> 
> to point me in a direction that has a better way of doing this...
> 
> 
> 
> Thanks for looking,
> 
> 
> 
> Jeff

My back of the envelope coding would do it this way. Use an array of fail_counters, with PASSED as the first element all the way up to FAILED_CRITICAL as the last element. Then use a simple loop starting from index fail_priority to the end of the list looking for errors. Like this:

# Array of fail counters
fail_counters = [
    0,  # PASSED
    0,  # LOW
    0,  # MEDIUM
    0,  # HIGH
    0   # CRITICAL
]

... run tests, accumulate error counts in fail_counters

for i in range(fail_priority, len(fail_counters)):
    if fail_counters[i]:
        return 1
return 0