The quality of testing

Just recently I attended a venue, where the quality of testing was the topic of the day. There was some interesting points about how this could be measured and monitored that I would like to share.

The discussion had the following topics all related to quality of the testing:

• Completeness of the test done
• Credibility of the test documentation
• Reliability
• Quality Parameters to be observed

The discussion was revolving around the parameters that was driving quality of a product, and especially the quality found in the work done by testers. It quickly became evident that the work and quality of it is hard to quantify, and require that you look into the details of the work, looking for completeness, creditability and reliability.

You can see the parameters we discussed below.

Completeness of the test done:

• Coverage vs. objects under test
  • Actual coverage vs. Planned
  • Execution vs. High risk areas first and coverage of same.

Creditability of the test effort

• Test cases are created and executed based on objective foundation
  • Structure in test cases.
  • Independence in review and test.
  • Dedicated resources for testing.
  • Separate estimate / time for test activities.

Reliability in test

• Predictability in test result (based on metrics and pervious experience).
• Status on test can easily and accurately be derived at any time.
• Defects and other observations are handled according to defined process.

Quality Parameters to be observed

• Defined metrics, that are monitored (less is more)
• Approval criterias and compliance in the deliveries (eg. Open defects vs. categories)
• Test efficiency per test phase (& number of defects found after go-live)

…And there are probably a lot more if you really want to scrutinize that test department of yours – Just remember to have a little faith in the work done, before going all in.

Happy testing & Have a nice weekend!

/Nicolai