Metrics - Test Progress & Control

Next up in the metrics posts are the metrics for Test Progress & Control, where we will look at metrics for Test Progress, Product Quality and Defect Resolution

I have included some recommendations to targets to achieve, risks you face by not monitoring this as inspiration.

Happy testing!

/Nicolai

Metric: Test Progress

Measured as: 

·         S-curve planned test cases vs. executed test cases

Purpose / Objective:

·         This metric aims to ensure that the test is running as planned, and serves as a tool to see trends of delays for risk mitigation and replanning.

Recommended targets:

·         Test execution must be monitored for each test level.

Measurement Frequency:

·         Weekly while test runs or in agreement with stakeholders

Risks if not monitored:

·         Unmanaged test.

·         Inability to proactively meet delays.

·         Unknown progress.

Metric: Product Quality

Measured as: 

·         Number of failed test cases vs. number of run test cases

Purpose / Objective:

·         This metric gives stakeholders an indication on the product quality, showing the amount of test failure for the object under test.

Recommended targets:

·         Trends of high failure should be followed by a test readiness review ensure that test resources are not wasted testing immature deliveries.

Measurement Frequency:

·         Weekly while test is running and prior to test readiness reviews

Risks if not monitored:

·         Unknown product quality.

·         Inability to mitigate poor quality early.

Metric: Defect Resolution

Measured as: 

·         Defects that are open and closed and the severity of these defects.

·         Measurement should be as progress over time to see trend and a snapshot to get overview of severity of defects.

Purpose / Objective:

·         This metric gives an indication on the amount of rework that is outstanding and shows trends of test completeness.

Recommended targets:

·         Critical defects must be closed prior to release of a product as stated in Nordea’s Corporate test policy.

Measurement Frequency:

·         Weekly while test is running and as part of test evaluation after each test level.

Risks if not monitored:

·         Poor or missing defect management.

·         Unknown size of the work backlog related to defects.

·         Inability to see defect trends and test completeness.

·         Unknown defect severity = inability to prioritize defects for closure.