Applying test automation

Problem: Test automation is easily applied wrong, resulting in huge cost and little productivity gains.

While reading Brian Marick’s “Classic testing mistakes” I came to think of some of the test automation initiatives I have been involved over the years. Some were successes other were not, and failure was often due to the mistakes Brian outlines in “Theme Five: Technology Run Rampart”:

• Attempting to automate all tests.   
• Expecting to rerun manual tests.   
• Using GUI capture/replay tools to reduce test creation cost.   
• Expecting regression tests to find a high proportion of new bugs.

Solution: Be careful when introducing test automation - Proof of concept can help you in the right direction.

In my experience, there are several things that you can do to increase chances of success when implementing test automation.

Manage expectations with the sponsor – He pays the bills and wants to know when the investment pays off. He I likely to pressure the test automation project for signoff on business cases that shows significant return of investment, and he is not likely to be a testing professional. This means that you need to tell him what to expect – Promises of great wealth is easy, but the truth about the long journey to get there needs to be told up front.

 

Have a strategy for what to automate – Brian is right, automation is not for everything, in fact there are many test cases that are never to be automated. The strategy needs to tell what to automate, what not automate, and more importantly, how to determine what is a candidate for automation. Examples of what to automate could be:

• Repetitive tests that run for multiple builds
• Tests that are highly subject to human error
• Tests that require multiple data sets
• Frequently-used functionality that introduces high risk conditions
• Tests that are impossible to perform manually
• Tests that run on several different hardware or software platforms and configurations
• Tests that take a lot of effort and time when doing manual testing

I would advise you to spend 5 minutes on this slideware it will definitely help you establish a strategy for test automation.

Choose the right tool and operator – "A fool with a tool is still a fool, and the wrong tool will make you a fool!" MAKE A PROOF OF CONCEPT! That goes for both the tool and the operator. Take the most complicated test case (that candidates test automation), and make the operator script it using the tool. If either the tool or the operator fails, then you might want to reconsider if they are right for the job.

Remember test automation is not a silver bullet – It will make money if applied right, but it will be a costly adventure if applied wrong. There is a nice article on this subject here – If you want to explore the topic further.

Have a nice weekend & Happy testing!

/Nicolai