Supporting your principles of testing

Problem: Your principles of testing cannot stand alone.

Having spend the time defining the principles on which you want you test to run is not enough – It is definitely a good start, but the principles needs to be anchored in the project through activities and action that turns the principle into practice.

Solution: Walk the talk by supporting the principles with action.

One thing is to have a defined test principles, another thing is to support these in practice in the projects. I am going to have a look at some of the supportive actions related to the 7 Principles of Testing from the ISTQB syllabus, as these are commonly accepted.

1) Testing shows presence of defects, but cannot prove that there are no defects. This means that testing needs to be very structured to ensure that testing probes the right areas. Risk based testing can be an approach to testing the right areas, but no matter how you go about this there is two things I would recommend you to inject in the project to address this principle: A strategy that guides your test planning and priorities and expectation management that ensures that the  steering group, project mangers, external stakeholders etc are aware that test will not prove that there are no errors.  

2) Exhaustive testing is impossible and testing everything including all combinations of inputs and preconditions is not possible. Applying different test techniques such as boundary-value analysis, all-pairs testing and state transition tables aim to find the areas most likely to be defective. Use equivalence partitioning as a driver for designing the test to a level of detail that makes sense for the project.

3) Early testing. Remember the principles of verification and verification?  Apply them early and make a plan for when to do it – Some want the V-model to drive early test, others have different approaches, the one that is successful is the one that ensures that testing happens in a timely manner (read: as early as possible). Another thing that facilitates early test is to have a champion in the project from the start – Someone who advocates good quality already from the project planning and onwards.

4) Defect clustering. Does your risk driven approach consider technical complexity / risk? If not, then you might want to consider this in conjunction with some coverage analyses. Another approach is to do experience driven explorative testing, with the purpose of finding candidates for a thorough test – It is a shortcut to finding those troubled areas.

5) Pesticide paradox. Change your approach every now and then – Switch responsibility for functional areas in the test team. Invite new people in, try explorative test, arrange bug hunts, have a bug-off of testers vs. development or business vs. it-team. Point is that you need to refresh yourself now and then, and you might as well put in your plans and have some fun while doing so.

6) Testing is context depending. Base your strategy on the nature of the system and the error tolerance of the receiver. Don’t juse run all test cases just because you can – Run those that are related to the functions delivered.

7) Absence – of – errors fallacy. Invite those users in early, sooner than later, to gather feedback that ensures that you are moving towards ‘fit for purpose’. Involving users requires careful planning, and since you want early involvement this is something that you need to address ASAP in any project.

Again, my point is that you need to have actions that supports the principles, without this, you principles are worthless. Another point is that you need to drive all of the above aided by common sense. Do not invent a wheel unless you have a need for driving somewhere.

Happy testing!

/Nicolai