Metrics - Test Preparation & Planning

Following the defect metrics post I came across my list metrics that I find useful in projects. The metrics focus on 3 areas of the test activities: Test Planning, Test Execution and Test Results – This post will be about Test Preparation (& planning).

For Test Preparation & planning I’ll invite you considering the following:

·         Test Plan Documentation

·         Test Preparation

·         Test Coverage

·         Test Estimates

I have included some recommendations to targets to achieve, risks you face by not monitoring this as inspiration. These are but a few things that you can measure against, but experience is that simple metrics that has a very specific purpose in life will get you far.

Happy testing!

/Nicolai

 

Test Plan Documentation     

Measured as: 

·         Does a test plan document exist?

·         Is the test plan formally approved/agreed with the stakeholders?                         

Purpose / Objective:

·         This metric measure if test plans are documented and discussed and approved with the stakeholders.         

Recommended targets:

·         Test plan is created early.

·         Test plan is formally approved by stakeholders.    

Measurement Frequency

·         One off measurement as part of project start up.

Risks if not monitored

·         Unplanned and unmanageable test.

·         Expectations to test not aligned with stakeholders.

·         Resources & estimates for test not outlined early.

Metric: Test Preparation

Measured as: 

·         Number of test cases per test level.

Purpose / Objective:

·         This metric measure if test plans are documented and discussed and approved with the stakeholders.         

Recommended targets:

·         All test levels must be covered by test cases.

·         Number of test cases should reflect complexity / risk.

·         Test levels with broad scope will often have many cases.

Measurement Frequency

·         Prior to test level readiness review and on ad hoc.

Risks if not monitored

·         Functional areas / requirements not covered by test.

·         No early test execution = cost escalation of defects.

Metric: Test Coverage

Measured as: 

·         Requirements not covered by test cases.                

Purpose / Objective:

·         This metric aims to uncover requirements that are not covered by test, in order to flag risk and assist test team in getting appropriate level of coverage in the test.

Recommended targets:

·         All requirements should be covered by test cases.

·         Traceability between test cases and underlying requirements must be possible to do prober coverage analysis.            

Measurement Frequency

·         Prior to test level readiness review and on ad hoc.

Risks if not monitored

·         Unknown quality in delivery.

·         Untested functionality released to production.

Metric: Test Estimates

Measured as: 

·         Work breakdown of test activities in test plan for all test levels  

Purpose / Objective:

·         This metric aims to verify prober level of work breakdown and to ensure that estimates are detailed enough to describe the activities.

Recommended targets:

·         All test levels must have estimates that quantify the test effort needed for each test level.

·         Test estimates must be detailed enough and broken down to less than one man-week of effort.

Measurement Frequency

·         Prior to negotiation of test budget and planning activities

Risks if not monitored

·         Inaccurate / unrealistic estimates for test activities.

·         Inaccurate planning and inability to deliver against major milestones.