Software Testing is Dead, Long Live Software Testing
Summarize With:
By Rajiv Jain (CEO, ThinkSys)
Forgive the rather corny headline (fans of The Phantom will know it's origin) but as someone who's been in the software testing business for years now, I've grown more than a little weary of these periodic announcements of the death of software testing. I saw a quote from James Bach somewhere that went, 'Pretty good testing is easy to do. That's partly why some people like to say 'testing is dead'— they think testing isn't needed as a special focus because they note that anyone can find at least some bugs some of the time. Excellent testing is quite hard to do.' Bach's assessment is likely true, but I think it is also possible that these prophets of doom aren't really pronouncing the death of testing, rather the end of testing 'as they know it.' This is a statement I am much more likely to get behind. If you look at the testing world around you, it has changed significantly in the last 2 -3 years even. Let me take this chance to highlight a few changes that I have particularly been struck by.
- Agile and DevOps:
Let me take the liberty of lumping both these software product development movements together for the purpose of this post. Shortening development and release cycles pushing into continuous development, continuous integration, and continuous delivery have shaken the fundamentals of how we used to test software.
These are the days of continuous testing — no more waiting for the end of a release, testing it in some kind of staging area and then pushing it back to engineering with bug reports in tow. Apart from these changes to the way we work, one fundamental change this acceleration in the release cycles has wrought is one that I am very happy about. Today, all around we see testing becoming much more tightly coupled to the product and business goals and at a much earlier stage in the product development cycle.
Testing podcaster Trish Khoo had said, 'The more effort I put into testing the product conceptually at the start of the process, the less I effort I had to put into manually testing the product at the end because less bugs would emerge as a result.' Product owners too now recognize that if testing has to deliver in such a time-constrained environment, it has to be involved early, get visibility of the direction the product is likely to take, and the possible roadmap of development. This is great news! - Developers Testing:
I have expressed my opinion in the past on whether developers should test their own code. Well, my opinion is moot— the fact is a blurring of boundaries has already taken place between development and testing and who does what. DevOps teams are routinely staffed with developers, operations, and testing skills as the product rolls out full-speed ahead. The need for greater code coverage in a shorter time has driven even greater use of Test Automation and writing the code or the scripts for automating test cases is a task developers are well suited for. Joe Colantonio says, 'I remember the days when QA testers were treated almost as second-class citizens and developers ruled the software world. But as it recently occurred to me: we're all testers now.' No more are testers looked down upon as children of a lesser God, so to speak. - The Role of Cloud:
The outlook has turned positively cloudy over the last few years. More and more products, or Apps as they choose to be called now, are SaaS-based and consumed on-demand over the 'series of tubes' that make up the web all around us. Testing for these products (ok Apps) is pretty different but most specifically in the emphasis that now has to be placed on security and performance.
Your App is only as secure as the last hack so an almost unreasonable amount of the focus of the testing is on identifying vulnerabilities and in keeping the architecture and the internals of the app safe from all possible attacks. Then there is performance. The days of your enterprise application sitting on a dedicated server on your network seem to have gone for good — when the app is on a multi-tenant infrastructure that is being accessed by hundreds, possibly thousands of other users it can't show any strain from the load and this becomes a key focus area for those testing it. - Mobility:
Mobiles have changed our lives so can software testing be immune? Software products today have no choice but to have a mobile strategy. That customers will access these products from their tablets and smartphones is a given. To those tasked with testing these products, this means a load of added complexity. Now your test plans have to factor in devices of varying form factors and differing hardware and software capability.
There is the access speed and bandwidth to think about. The biggest change possibly, though, is in testing the user experience. The small screen presents differently and is used very differently and the software products that expect to run on that small screen have to be tested with that in mind. All of these are paradigm shifts from the things we used to test for even a few years ago.
Conclusion
Let me end with another quote, this one by Mike Lyles, 'While we may understand the problem, it is critical for us to change our thinking, our practices, and continuously evolve to stay ahead of the problems we face.' The fact is the situation around us is always going to change — as capable software testers, our role will always be to find a way through, and to help build better software in the bargain.