Monday 23 September 2013

Test automation return of investment


Problem: Calculating & using Return of Investment (RoI) in test automation implementation


Engaging in discussions on RoI when implementing test automation is a tricky business. Discussions is often derailed either by incorrect numbers, a strange formula being used or stakeholders who for political or personal reasons wants to influence the decision.

Solution: Expectation management and a simple RoI calculation to steer the discussion

RoI calculations are needed for doing the business case detailing the investment that test automation is. There is really no way around this, but I would recommend that you start your test automation discussion with expectation management.

Expectation management is needed, as there will be many different opinions on what can, and what cannot be done using test automation. Especially the falsely expected benefits are dangerous, as they will pull discussion in the wrong direction, and set the project up for failure. When looking for falsely expected benefits and other mistakes I suggest that you start the discussion with stakeholders listing the tangible and intangible benefits they see. This will serve as input for the business case and give you an idea of the realism in the expectations of the stakeholders.

Test automation is an investment that takes a lot of time to get RoI from. Especially the arguments about savings and cost reduction should be examined closely and that is where a RoI calucation will come in handy. Some years ago I got a copy of Dorothy Graham’s test automation calculator, a nice Excel sheet that allows you to make a simple RoI. You can find it here, along with some notes on the subject on Dorothy's Blog.

In my experience RoI calculations can never stand alone – The numbers in the calculation needs to be backed by the experience you get from doing the actual test automation. When introducing test automation to a project, I suggest that you do a short proof of concept to ensure that the assumptions are right and that estimates are reasonable.
In short:
  • Engage stakeholders in a discussion on expectations and manage those
  • Do a RoI based on a simple calculation – Remember to mention the intangible benefits
  • Make a proof of concept, where you check your numbers and assumptions

Want to know more? Check out Douglas Hoffman’s excellent paper on ’Cost Benefits Analysis of Test Automation’ found here: http://www.softwarequalitymethods.com/papers/star99%20model%20paper.pdf


Happy testing!

/Nicolai
 

1 comment: