Does automatic golden master unittest generation exist/or is it feasible?

[Steven D'Aprano]

On Tue, 04 Apr 2017 19:40:03 -0700, fleshw wrote:

> People told me that there's a similar concept called "Golden Master
> Testing" where it keeps a "golden record" and runs test case on it. So
> it looks like I'm trying to automate golden master testing.
> 
> So my question is this:
> 
> 1. Is this a concept already explored?

https://duckduckgo.com/?q=Golden+Master+Testing

https://www.google.com.au/search?q=Golden+Master+Testing


Gold master testing is not a substitute for unit tests. The purpose of 
gold master testing is different: not the ensure that the code is 
correct, but to ensure that the behaviour *doesn't change*. Or at least 
not without a human review to ensure the change is desired.

You would use gold master testing when you have a legacy code base that 
needs refactoring. You don't care so much about fixing bugs (either there 
aren't any known bugs, or you have to keep them for backwards 
compatibility, or simply there's no budget for fixing them yet) but you 
care about keeping the behaviour unchanged as you refactor the code.

If that describes your situation, then gold master testing is worth 
pursuing. If not, then forget about it. Just write some unit tests.

I like small functions that take some arguments, do one thing, and return 
a result without any side-effects. They are very easy to test using 
doctests. Doctests make good documentation too, so you can kill two birds 
with one stone. Write doctests.

For anything too complicated for a doctest, or for extensive and detailed 
functional tests that exercise all the corners of your function, write 
unit tests. The `unittest` module can run your doctests too, 
automatically turning your doctests into a test case.

If your code base is well-designed, with lots of encapsulation, then 
writing unit tests are easy.

If its not... well, you have my sympathy.


 
> 2. Is there already a framework for doing this?

This came up in googling. I make no comment about its maturity, 
functionality, bugginess or overall quality, as I have never used it.

https://github.com/approvals/ApprovalTests.Python



-- 
Steve