Wednesday, 29 January 2014

Standard Risk List for Projects

Problem: Risks are cumbersome to identify, and requires grooming before discussion can take place.

Working with risks require that they are well documented and made accessible for the organization to absorb and work with. Grooming the risks takes time, allowing some of the risks to grow into issues, or slip under the radar and avoid getting mitigation tasks in the plans. This means that early identification of risks will make test planning and risk management a lot easier.

Solution: Start with a standard risk list, and build your planning and risk management on that.

In many projects it is the same problems (or variations of same) you will encounter. So you might as well look at the usual suspects. I did that a while back, and concluded that I had to make a standard list to use for input to test plans and risk meetings in the projects I participated in.

Given that test management is my game, the risks focus on test issues and impact on quality, and to make them easily available I divided them into these categories:

Resource related risks:
  • Use of undedicated resources
  • Poor Productivity in long projects
  • Wrong or missing test resources
  • Employee Turnover
Process related risks:
  • Big bang releases
  • Overlapping project phases & activities
  • Configuration and Change Management
  • Risk Management not being done
Delivery related risks:
  • There will be a delay in the release of the product
  • Gold plating
  • Stakeholders and expectations not managed
Documentation related risks:
  • Requirements not complete & contain assumptions
  • Requirements Inflation
  • Specification Breakdown
  • Compromising on designs
  • Technical risks
  • Inter plan dependencies
  • Scope is not closed.
  • Test foundation not in place
  • Test environment & Test data not in place
  • Lack of Test Tool usage

Each detailed with the following information: Risks, Event, Possible causes, Effects, Possible measures to facilitate discussion and mitigation.

You can find my standard risk list for projects here.

Note: This is not an invite to stop inventing risks, and run risk management, but it is a fast-track to getting risk management going early in projects. It is an invitation to get the test and QA related risks on of the list and help you write the best test plan ever.

Have a nice day & Happy testing!


*** EDIT: I came across this very comprehensive list of risks in SDL, something that you might find handy: 

Friday, 24 January 2014

Visual Studio Test Tooling Guides

I need to share this, as it is a real treat for all working with test in MS Visual Studio. We discovered this by accident, while working with coded IU tests this week.

In here you will find comprehencive guides to everything related to test in the MS Visual Studio / Test Manager universe.

It contains the following guides, which are all worth a look:
  • Coded UI Guide
  • DevOps bug resolution using IntelliTrace
  • Better Unit Testing with Microsoft Fakes
  • Microsoft Test Manager Guide
Enjoy & Happy testing!


Monday, 20 January 2014

Automation Considerations

I came across this today:

It underlines the need for Return on Investment discussions and measurements when working with automation. There are so many factors that can kill a good automation case, where running maintenance and gold plating of the solution are some.

I wrote something on automation RoI a while back, and the points from above comic really underlines the points about deploying RoI and proof of concept as drivers for evaluating the results of implementing automated tools.

The previous post on test automation return of investment can be found here:

Have a nice day & happy testing!


Tuesday, 14 January 2014

Test Readiness Review Checklist

A while back I wrote about using test readiness reviews as part of test governance toolbox :

I came across my test readiness review checklist, and thought I would share it:

Deliveries to test:

·        Is feature list complete and updated?

·        Are some features not ready for test?

·        Have previous test phase been completed?

·        Does the delivery contain known errors / shortcomings?

Test documentation:

·        Are test cases ready and reviewed?

·        Have test cases been set up in test tool?

·        Are test cases involving 3rd party stakeholders discussed and agreed?

·        Has test log and reporting templates been prepared for the test?

·        Has invites been sent for test readiness review, and test status meetings for the test to come?

Test Data and Environment:

·        Is test data load needed, planned and detailed?

·        Is test data requirements in place and supported in test environment?

·        Is the environment configured according to expectations?

·        Has deploys of test builds been scheduled?

·        Are there any know environment outages (upgrades etc.) in the test period?


Aside from above you need to obtain a statement of: “We are ready and able to execute test from date X” from all stakeholders in the test phase to come.

Have a nice day & Happy testing!


Friday, 10 January 2014

Test Process Improvement

Problem: Process Improvement without clear priorities and goals will fail

Many organizations do not have sufficient transparency in the test processes deployed and the result of this investment, this makes it hard to deliver the expected result to the business. Much like hiking you require a map, a compass, your destination and most importantly your current location – Without known location and clear goals you will get lost, no matter whether you are hiking or doing process improvement.

Solution: Test Process improvement based on maturity assessments.

Why do maturity assessments?
The test maturity assessments are done in order to guide, develop and prioritize the improvement initiatives in the organization. It allows you to work on your initiatives on a strategic, tactical and operational level in a controlled manner.

·        Create common foundation and visibility on vision and goals for test and QA.
·        Create understanding and commitment for test and quality in the organization.
·        Get insight on cost of quality and RoI in the projects
·        Serve as a communication and governance tool.
·        Set priorities for the initiatives.
·        Guide the initiatives in a controlled manner.
·        Improve process and competences.
·        Optimize use of resources
·        Reduce waste in projects
·        Streamline test effort and approach.

When is a maturity assessment needed?

·        If there is an increased need for streamlining of effort and process. E.g. when complexity increases in IT, dependencies on key resources, use of external resources, working with external partners, offshoring of development and test.

·        Deteriorating quality in the deliveries. E.g. Lots of defects in production, cost escalation, lack of confidence in deliveries, economic consequences to the business.

·        There is a need for information to do decision-making, prioritizing new initiatives etc.

·        Projects are high risk and of a critical nature.

What tools are available? What are the methods?
There are a lot of tools and methods you can use to do maturity assessments, such as TPI, TMMI, CMMI, ISO etc. It really doesn’t matter which one you use as long as it sets the boundaries and priorities for the effort and secure a holistic approach to the improvement activities.

What does it take to get started?
I use TPI (Test Process Improvement) developed by Sogeti, it consists of a field test tool that allows you to make the assessment based on interviews of key people in the organization. The result generates an overview of the situation, and gives a framework that allows a controlled and prioritized approach to improving quality.

The details about how-to is described in the book “TPI NEXT – Business Driven Test Process Improvement“ (ISBN 90-72194-97-7). Read the book and play with the tool, it is a small investment that gives a lot in return. Try answering the questions in the questionnaire, and you will get an instant result.

The TPI materials can be found here:

A word of advice if you are the slightest in doubt about if you can answer positive to a question then it should be no, and that translates into something that you should look into. Using the questionnaire in projects, interviewing people like project manager, test manager, lead developer and business test persons will in the matter of hours give you a clear view of ‘state of the nation’. For starters, you only need to look at the controlled level, which should give you plenty of observations to work on.

Happy testing!