Art of Unit Testing

[Grace]

"David Goodger" <goodger at users.sourceforge.net> wrote in message
news:B7A3F696.16481%goodger at users.sourceforge.net...
> Paul Moore <gustav at morpheus.demon.co.uk> wrote on 2001-08-18 08:19:
> > On Fri, 17 Aug 2001 11:32:49 -0700, Jeff Shannon <jeff at ccvcorp.com>
wrote:
> >> Well, the alternative argument could be that, if you don't need to
setup and
> >> teardown in between various tests, then they could be coded as
subtests of a
> >> single, larger test...  OTOH, if you *do* have high setup/teardown
overhead,
> >> which *does* require being redone for each test, then coding it each
time
> >> would be a pain.  Lumping subtests together is easier than multiple
copies of
> >> setup/teardown...
> >
> > I'm not sure I agree. The long and short of it is that both cases can
happen.
> > But look at my example "for real" (sort of)
> >
> > class testSimpleQueries(unittest.TestCase):
> >     def setUp(self):
> >         # May take seconds to execute...
> >         self.connection = DB.Connect(connect_str)
> >     def tearDown(self):
> >         self.connection.Disconnect()
> >     def exec_query(self, q):
> >         "Trivial helper to run a query"
> >         return self.connection.Execute(q)
> >     def testQ1(self):
> >         "Simple query"
> >         q = "select 1 from dual"
> >         assertEqual(1, self.exec_query(q))
> >     def testQ2(self):
> >         "Exception when no rows returned"
> >         q = "select 1 from dual where 1=0"
> >         assertRaises(DB.Error, self.exec_query(q))
> >     # And 100 more trivial queries...
> >
> > You get the idea. The setup costs a *lot* in relative terms of time
(each test
> > query takes, say, 0.01 second). You really want to only do that once.
But you
> > don't really want to code a single huge test - it destroys the
reporting of
> > the individual test docstrings by unittest.main(). What do you do?
>
> The setUp/tearDown mechanism can ensure that each test is independent of
any
> state left over from any other test. If you know your tests won't leave
any
> state behind, you could put your DB.connect() call in
> testSimpleQueries.__init__. (Don't forget to call
unittest.TestCase.__init__
> though.) Again, this assumes that none of the individual tests will
change
> the state of the connection. To close the connection, you could put the
> self.connection.Disconnect() call in testSimpleQueries.__del__; but
there's
> no guarantee when (or even if?) this will be called.

No, it won't work.

The __init__ is called for each test method for the class that inherits
TestCase -- PyUnit creates as much instaces of TestCase class as the number
of test methods, in this case those starting with "test-".

One of the golden rules of unit testing is "make it fast." Most of the
time, it shouldn't take more than a few mins at one run -- C3 had over 1300
unit tests, performing over 13,000 individual checks altogether, which ran
in about 10 minutes in visual works. The fastness of running unit tests is
necessary for TestDrivenDesign and MercilessRefactoring.

To make it fast, you should do high-cost things such as db connection,
insert/delete, as infrequent as possible. Yet you want to test as much as
possible.

One alternative to using live dbs is using small test dbs, but after all it
can lead to a real mess if a multiple of developers and test cases trying
to do the db test. (you could in-line the db in the test code instead)

Another one is using shunt pattern.

When I look at your code, it seems like it's testing DB itself rather than
the python code. PyUnit is not intended for testing DB engines per se.

"Some tests make sure that the lowest layer of database access works as
planned. Then I write a suite based on the assumptions demonstrated by the
first suite. It assumes that I can get things into and out of the real
database, so I don't have to have the real database there. That way I can
create an in-memory impostor for the database and exercise the higher level
objects." [PPR]

Putting lots of SQL statements at a higher layer would, as one of many
disadvantages, reduce code testability enormously. (refer to
http://www.c2.com/cgi/wiki?BadFormedPersistenceLayer)