unit test strategy

[Aaron Brady]

On Sunday, September 16, 2012 2:42:09 AM UTC-5, Steven D'Aprano wrote:
> On Fri, 14 Sep 2012 19:59:29 -0700, Aaron Brady wrote:
> 
> 
> 
> > Hello,
> 
> > 
> 
> > I've developing a test script.  There's a lot of repetition.  I want to
> 
> > introduce a strategy for approaching it, but I don't want the program to
> 
> > be discredited because of the test script.
> 
> 
> 
> Test scripts should be simple enough that they don't require test scripts 
> 
> of their own. Or at least, not too many "test-the-test" tests. It is 
> 
> possible to avoid repetition without making convoluted, hard to 
> 
> understand code. Based on the tiny code fragments you give below, I 
> 
> suspect that your test script will need more testing than the code it 
> 
> tests!
> 
> 
> 
> 
> 
> > Therefore, I'd like to know
> 
> > what people's reactions to and thoughts about it are.
> 
> 
> 
> I'd love to make some suggestions, but I have *no idea* what you are 
> 
> talking about. See further comments below:
> 
> 
> 
> 
> 
> > The first strategy I used created an iterator and advanced it between
> 
> > each step:
> 
> 
> 
> What are you using an iterator for? What does this have to do with unit 
> 
> testing?
> 
> 
> 
> So far, your explanation is rather lacking. It's a bit like:
> 
> 
> 
> "I want to create an alarm system for my home, so I put in a screw and 
> 
> tightened it after each step."
> 
> 
> 
> Doesn't really help us understand what you are doing.
> 
> 
> 
> 
> 
> > 	self.op_chain(range(5), ('add', 5))
> 
> > 	self.op_chain(range(5), ('add', -2), ('add', -1))
> 
> > 	self.op_chain(range(5), ('discard', -1), ('add', 5))
> 
> > 	self.op_chain_ok(range(5), ('update', [0, 1]))
> 
> > Etc.
> 
> 
> 
> Where is the iterator you created? Where are you advancing it? What's 
> 
> op_chain do?
> 
> 
> 
> 
> 
> > I'm considering something more complicated.  'iN' creates iterator N,
> 
> > 'nN' advances iterator N, an exception calls 'assertRaises', and the
> 
> > rest are function calls.
> 
> [...]
> 
> 
> 
> You've proven that even in Python people can write obfuscated code.
> 
> 
> 
> 
> 
> > Do you think the 2nd version is legible?
> 
> 
> 
> Neither version is even close to legible.
> 
> 
> 
> 
> 
> > Could it interfere with the accuracy of the test?
> 
> 
> 
> Who knows? I have no clue what your code is doing, it could be doing 
> 
> *anything*.
> 
> 
> 
> 
> 
> 
> 
> -- 
> 
> Steven

You are forcing me to explain my test code.

Here is an example of some repetitive code.

for view_meth in [ dict.items, dict.keys, dict.values ]:
	dict0= dict( ( k, None ) for k in range( 10 ) )
	iter0= iter( view_meth( dict0 ) )
	dict.__setitem__( dict0, 0, 1 )
	next( iter0 )
	dict.__setitem__( dict0, 10, 1 )
	self.assertRaises( IterationError, next, iter0 )

	dict0= dict( ( k, None ) for k in range( 10 ) )
	iter0= iter( view_meth( dict0 ) )
	next( iter0 )
	dict.__setitem__( dict0, 0, 1 )
	next( iter0 )
	dict.__setitem__( dict0, 10, 1 )
	self.assertRaises( IterationError, next, iter0 )

	dict0= dict( ( k, None ) for k in range( 10 ) )
	iter0= iter( view_meth( dict0 ) )
	self.assertRaises( KeyError, dict0.__delitem__, 10 )
	next( iter0 )
	dict.__delitem__( dict0, 9 )
	self.assertRaises( IterationError, next, iter0 )

	dict0= dict( ( k, None ) for k in range( 10 ) )
	iter0= iter( view_meth( dict0 ) )
	next( iter0 )
	self.assertRaises( KeyError, dict0.__delitem__, 10 )
	next( iter0 )
	dict.__delitem__( dict0, 9 )
	self.assertRaises( IterationError, next, iter0 )


It makes sense to condense it.  However, it can be condensed rather far, as follows:

dsi= dict.__setitem__
ddi= dict.__delitem__
KE= KeyError
IE= IterationError
chain(range(10), 'i0', (dsi, 0, 1), 'n0', (dsi, 10, 1), (IE, 'n0'))
chain(range(10), 'i0', 'n0', (dsi, 0, 1), 'n0', (dsi, 10, 1), (IE, 'n0'))
chain(range(10), 'i0', (KE, ddi, 10), 'n0', (ddi, 9), (IE, 'n0'))
chain(range(10), 'i0', 'n0', (KE, ddi, 10), 'n0', (ddi, 9), (IE, 'n0'))


The parameters to 'chain' correspond 1-to-1 with the statements earlier.  The view methods are looped over in the 'chain' method instead.  'op_chain' was an earlier version; some midway point in condensing the code could be preferable.

Specifically my questions are, is the code condensed beyond legibility?  Should 'chain' execute the test directly, or act as a metaprogram and output the test code into a 2nd file, or both?  Should 'chain' create the iterators in a dictionary, or in the function local variables directly with 'exec'?