“Without strategy execution is aimless. Without execution, strategy is useless.” Morris Chang CEO of Taiwan Semiconductor Manufacturing Co.

In this agile age, more organizations than ever before are looking at making Test Automation a core part of their product development efforts. That being said, anecdotal evidence would suggest that a surprisingly large number of these test automation efforts don’t deliver the desired results. As a company focused on testing and test automation we have encountered our fair share of client relationships that start on this kind of a note. Clearly there is no lack of great tools and technologies out there and it’s also fair to say that the test automation skills are scarce but available – what then is behind the lower than desired success rate? Over time, we have come to believe that chief among the many reasons test automation “experiments” fail is a poorly defined automation strategy. This post is our attempt to outline, what we believe to be, six critical considerations while devising a solid test automation strategy.
1. Business Buy-in: In many ways this is the most critical piece. Test Automation has to be core and not just a vanity project. The automation strategy has to be aligned with the business objectives and should have the active buy-in of the business to ultimately be successful. This is even more critical in the context of the Agile and Continuous Delivery models of product development. Given the shorter time windows between releases, it becomes essential for the automation strategy to plug into the business and product strategy at a very early stage in order to be effective over the entire product development life cycle.

2. Defined Outcomes: It’s been said in the context of planning to “start with the end in mind”. The call is to make sure that the end objectives are kept in view at every stage of the journey. In the context of a test automation strategy, the need is to have clearly defined objectives that the test automation effort should achieve when done. These goals need to be very specific – “test faster” is not a goal. “Achieve 25% greater test coverage” and “20% faster time to market” are examples of specific goals to aim for.

3. Tool Selection: There is a variety of tools, technologies and platforms out there. Making the right choice is an absolutely key part of the test automation strategy. The tool you pick has to suit your business needs and be appropriate for the achievement of your “defined objectives”. There is also the question of skills – if a synergy can be built with the technology skills already available in the organization then it may be a good idea to pick a test automation tool that allows such synergy.

4. Assigned Stakeholders: Our suggestion is that the test automation effort should not be an “add-on” responsibility of the manual testing team. It is important that this effort be helmed by a defined set of stakeholders with the means and the authority to take decisions and in turn tasked with its ultimate success or failure.

5. Expectation Setting: Walmart founder Sam Walton said, “High expectations are the key to everything.” While that may be true in most fields of human endeavor, in the context of test automation we suggest tempering the expectations to prevent setting unrealistic goals. It’s unlikely you will ever be able to achieve 100% automation and anyway some conditions will always be tested better manually. That being the case we suggest carefully picking the test cases that you would like to automate so the effort can deliver the maximum bang for the automation buck.

6. Evolution: The test automation roadmap is a clearly defined path of growth. It is important to remember that the product will keep evolving and hence room should be made in the test automation strategy for similar evolution. Agile implies faster releases and more iterations of the product under test – the test automation roadmap should be able to deliver RoI at each such stage to be truly successful.

Winston Churchill is reported to have said, “However beautiful the strategy, you should occasionally look at the results.” Our view is that for the right results the strategy has to not only be beautiful but also well thought out!