Some Numbers That Tell the Evolving Software Testing and Automation Story
In today’s rapidly changing world of testing practices, the focus of the software industry is now moving away from the Center of Excellence approach to a more wide-ranging strategy using Agile or DevOps practices. So much so, that based on the World Quality Report for 2016 the budget allocation for testing has increased 9% year-on-year. Considering this shift it would be interesting to see some of the top trends and likely evolutions for testing and test automation and some key numbers that tell the tale:
- Development and Operations; DevOps
DevOps is a software development technique which relies on tight coordination between Development, Testing, and Operations. The goal of DevOps is to improve the communication and collaboration between the departments so as to achieve, almost absurdly, faster software delivery. Out of all the participants of the World Quality Report survey, it was identified that 67% are using DevOps practices. This has a fundamental impact on testing as the basic idea of DevOps practice is to start testing as early as possible in the development cycle. This ensures that issues which are encountered only when the entire development is completed are identified much in advance. DevOps also helps teams focus on quality from day one of the project, instead of finding out about the defects much later in the software development lifecycle.
- Test Driven Development (TDD)
TDD is a methodology that, in many ways, relies on repetition of a short development cycle. The customer requirements are converted into very specific test cases and it involves execution of the new test cases. 39% of the participants in the World Quality Report are using TDD and with good reason too. A paper published in the Empirical Software Engineering journal mentions that ‘TDD seems to be applicable in various domains and can significantly reduce the defect density of developed software without significant productivity reduction of the development team.’ The study also compared four projects at Microsoft and IBM that used TDD with similar projects that did not employ this testing practice and found that defects per thousand lines of code reduced anywhere from 40% – 90% as compared to non-TDD projects.
- Selenium and related tools
Selenium is a software testing framework which is designed primarily for web applications. The advantages of using Selenium is that it is an open source tool which means there is no license cost involved. More and more companies are now embracing this open source technology for their automation testing needs. Gartner’s Magic Quadrant for Software Test Automation makes, the rather bold, claim that by the year 2020, Selenium WebDriver will become THE (emphasis ours) standard for functional test execution. and that this will be the key differentiator between vendors that cannot provide this test functionality.
- Mobile testing in the cloud
Mobile usage is rising, as everyone knows, but it is also true that the demands are becoming more complex. HTTP Archive reported that in the 4 years after 2011, the average size of a page served to the mobile rose from 340 KB to 1080 KB. 20% of these pages also had more than 100 resource requests and nearly 2 in 3 had 10 or more Java Script requests. This suggests a growing emphasis on mobile testing. Mobile testing in the cloud can help the testing teams to be more productive and efficient. Usually, mobile testing lab devices can be established at a centralized location within the organization. With a variety of mobile cloud testing tools, the testing teams can now be spread across different locations where they can all access any of the devices from the mobile testing lab via the cloud. Cloud testing also gives the flexibility of not having to buy the same device more than once; instead you can have one device in the cloud and the testers can reserve times for when they wish to test on them.
- Digital transformation
The CIO Report of 2016 reported that 2 out of 3 CIOs were already measuring their efforts against different KPIs from 12 months ago, directly because of concerns related to Digital Transformation. Digital transformation, today is becoming a key aspect for enterprises to achieve their business goals and to avoid getting disrupted by newer, more digitally adept business models. Many companies are now relying on this transformation approach for their success. With digital transformation the challenge of software quality is a critical factor that organizations need to consider. The digital agenda is focused on user experience testing, as organizations feel the need to provide the best and most innovative digital experience to its end users. Security testing is also a vital consideration when it comes to Digital transformation. The centrality of testing in the new, digitally transformed enterprise is visible in World Quality Report’s projection of a 40% rise in the budgets for QA & Testing by 2018.
Overall, the top priorities for QA and Automation testing executives continue to include Security, Customer experience, and efficient delivery. The main intention of QA and Automation testing is evolving to provide a seamless experience to the end users which in turn helps to improve the corporate image, end user satisfaction, and ultimately business results.