Thursday, 26 September 2013

Testing ROI - what's your story??


There are numerous ROI calculations on various forms of testing. Be it manual or automated testing, in waterfall or agile contexts someone has crunched the numbers for "testing as a whole" or for a specific industry.
It very often leads to some rather academic figures.

And very often to a setup where a lot of assumptions or "model projects" have to form the foundation for the calculations - and then another set of "best practices" on top of this with estimates on how many minutes it takes to do a test script, review and execute it, how many defects/test script. And then you have to put in the human factor - multiply by 1.07 and the ROI is....lost.


Find a good "product" story for your project or for your QA department in your company. You will most likely not be able to do flashy ROI on top of the story. Or only partially.
I've been with a company that was very focused on software- and hardware quality, yet what we needed was not ROI to remind us that test was important. We all shared the same story which was told when you joined the company - two failed product launches and we are out of business.
And QA didn't want to contribute to that in any way. At all. So we tested because we had to do test - because we did not want to fail. And that did not change for years. ROI calculations were updated during that time, better tools to support testing were introduced, the test deparment matured in terms of processeses and participants skills and experience but at the end of the day our story was still the selling point. And it could be shared outside the QA department with other deparments.

I'm not saying that ROI's are useless. Some are obviously so easy to pick apart that they are. On the other hand good ROI's are great input for understanding how much effort and reuse it takes before automation is feasible. And it is also good for telling the story about how many parameters that are impacting testing. In short the ROI process is good for getting discussions and understandings aligned internally. If there is a factor of 20 between two estimates or factors in an ROI where is the consensus and why?

If you are put with the task of doing a ROI to justify a test department in your company you should consider something to be fundamentally wrong. Test departments are needed whenever there is software or hardware development. They can be outsourced with different disasters waiting to happen. Unless you also ROI "communication challenges". And if you do that please share your thoughts on this blog.
But we would much rather hear your product story.


No comments:

Post a Comment