unittest for system testing

[Steven D'Aprano]

Sorry for breaking threading, but the original post has not come through 
to me.

> On 18/10/2012 01:22, Rita wrote:
> Hi,
>
> Currently, I use a shell script to test how my system behaves before I
> deploy an application. For instance, I check if fileA, fileB, and fileC
> exist and if they do I go and start up my application.

Do you run the shell script once, before installing the application, or 
every time the application launches?

Do you realise that this is vulnerable to race conditions? E.g:

Time = 10am exactly: shell script runs, fileA etc exist;

Time = 10am and 1 millisecond: another process deletes fileA etc;

Time = 10am and 2 milliseconds: application launches, cannot find 
       fileA etc and crashes.


Depending on what your application does, this could be a security hole.

Regardless of what the shell script reports, to be robust your Python 
application needs to protect against the case that fileA etc are missing. 
Even if all it does is report an error, save the user's work and exit.


> This works great BUT
>
> I would like to use python and in particular unittest module to test my
> system and then deploy my app. I understand unittest is for functional
> testing but I think this too would be a case for it. Any thoughts? I am
> not looking for code in particular but just some ideas on how to use
> python better in situations like this.

Well, you *could* use unittest, but frankly I think that's a case of 
using a hammer to nail in screws. Unittest is awesome for what it does. 
It's not so well suited for this.

Compare these two pieces of code (untested, so they probably won't work 
exactly as given):

# sample 1
import os
import sys
for name in ['fileA', 'fileB', 'fileC']:
    if not os.path.exists(name):
        print('missing essential file %s' % name)
        sys.exit(1)

run_application()



# sample 2
import os
import sys
import unittest

class PreRunTest(unittest.TestCase):
    list_of_files = ['fileA', 'fileB', 'fileC']
    def testFilesExist(self):
        for name in self.list_of_files:
            assertTrue(os.path.exists(name)

total_tests, failed_tests = unittest.testmod()  # I think...

if failed_tests != 0:
    sys.exit(1)

run_application()



I think the first sample is much to be preferred, and not just because it 
is a couple of lines shorter. There's less magic involved.


-- 
Steven