Monday, 17 November 2014
Combination testing strategies
Problem: Complete testing is impossible.
I had an interesting discussion about test of a critical feature. The discussion was on test coverage and completeness in the planned test. The argument was that this feature was of a criticality that required “complete testing”
Solution: Acknowledge that complete testing is not possible and apply combination testing strategies.
There are enormous numbers of possible tests. To test everything, you would have to:
· Test every possible input to every variable.
· Test every possible combination of inputs to every combination of variables.
· Test every possible sequence through the program.
· Test every hardware / software configuration, including configurations of servers not under your control.
· Test every way in which the user might try to use the program.
No test satisfies all of above. How do we balance them? It is a matter of balancing the tradeoffs vs. risk, meaning that you get the highest coverage. I came across a nice slide set that I need to share, as it details different combination testing strategies: