Importance of Shift Left and Shift Right Testing Approaches

Gone were the days, when testing was considered as a separate phase and is carried out post-development of the software application. With the introduction of new and innovative approaches of software development or may be called software models such as agile and DevOps, testers are now able to show their presence and apply their sincere efforts towards the achievement of quality traits in a software application, right from its seeding.

Further, it is also pertinent to mention that despite continuous and strenuous testing efforts put up right from the dawn of the development phase till the production, a software application may lack in some or more qualities from user’s point of view, keeping in view the user’s expectation to operate software functionalities along with its performance in real world conditions and environment. Thus, testing both at the starting & during the development, and after the production is equally important to provide best quality software products to users.

The approaches of testing the software application starting from the initiation of its development and throughout the development, and testing done post-development are generally referred to as shift left testing approach and shift right testing approach.

Shift Left Testing Approach:

Shift left testing approach as the name suggests, shifts the testing process to the left of the development phase, i.e. testing starts at the starting point of the development. Basically, testing process begins with the development and is being carried out throughout the development phase with the vision to prevent defects rather encountering with unexpected defects at a later stage.

Shift left testing is an approach to introduce testers at an early stage of software development to ease the task of developers in developing the required software application of acceptable quality based on the specified requirements and specifications. Based on the requirements and specifications, acceptance criteria both for the software and business requirements is being defined and created. Continuous testing right from the beginning of the development phase and throughout the development helps in identifying, locating and removing or correcting the flaws occurring at each level of the development, which subsequently helps in reaching the acceptance criteria defined for the software. This approach may comprise of different types of testing for a particular job, including regression testing to test the originality after applying patches.

The early introduction of the testing process and not after the software development provides flexibility to the developers to implement dynamic changes throughout the development on the continuous perusal of feedbacks, reviews and reports provided by the testers, which is needed to apply patches or correction to remove bugs or defects as earliest as possible. This approach prevents or minimizes the presence of defects at the end of the development i.e. at the point of delivery or release.

Thus, shift left testing approach may be seen as an easy and economical way of eliminating the efforts, time and cost in correcting the software at a much complex state, when it is completely developed and is ready to release, and is almost technically & economically infeasible to implement changes in the application.

Shift Right Testing Approach:

When it is possible to develop software product of desired quality in an economical and convenient manner using shift left testing approach, then why shift right testing approach?? Well, the developed software adheres to the quality and attributes as specified in the business and software requirements, but what about the functioning and performance under real conditions and in the real world environment. Although, system is completely developed in compliance to stated requirements and specification, but how to encounter with the unexpected events or circumstances occurring with the software such as slow performance, crash down, failure and many such things.

Shift left testing approach is an essential requirement but solely not enough to certify the quality of the software as user experience and reviews is equally important to business/software requirements in determining the software quality.

Shift right testing approach initiates the testing task from the right i.e. post production of the software. The software application present at the right end, i.e. completely built and functioning software application is being tested to ensure performance and usability traits. Feedbacks and reviews are received from targeted users to experience satisfaction in using and managing the software in real world conditions, which thereby helps in improving the quality further.

Although, shift left testing provides the scope of early testing of software to prevent defects, still shift right testing approach has its own unique advantages over shift left testing like

  • Automation of a complete and stable software application is easy to be done in comparison to unstable application (partially or not completed).
  • As compare to shift left, shift right approach provides wider test coverage as testers have the access to test complete system at a much larger time.
  • As targeted users are involved in the approach, the strategy of right shift provides feedback and reviews of the end users, which helps to improve quality of the software at a much larger extent.


In view of the above stated facts, it may be concluded that both shift left and shift right testing approaches are equally important and delivers unique and different way of testing the software application, which are adequate to touch each and every aspect of the application to ensure best possible quality.

4 Crucial Aspects Of Cloud Selection and Implementation

Cloud computing is a new way of doing business. A lot of companies, both big and small, are considering moving to the cloud to make their business more organized and efficient. This allows them to stop worrying about IT infrastructure and concentrate on other aspects that will help in business expansion in the future. But there are some factors that are absolutely important when you are considering this migration. The move will directly or indirectly affect your security and privacy issues, risk management practices, compliance, auditing and many other such issues. Here are four of the most important aspects you need to take into consideration to ensure that the move takes place smoothly:

  1. Reliability and Security:
    Shifting to the cloud is going to be a big change for the systems, data management and overall functionality of your organization. While the model for how IT services are delivered and consumed may be in for a change, the end objectives will be the same, hence it is very important that the new solutions support all the elements that are vital for the end users. Even small glitches in the cloud implementation could lead to major problems in the functioning. Your cloud may be a test bed for new services and applications, developers are working on or it may be running your payroll. No matter what the purpose of the cloud is, users expect it to function perfectly every minute of the day. Hence, it is important that you choose a cloud service that is completely available, reliable and secure. Ensure that it has the capability to continue to operate and keep the data intact in the virtual datacenter even if a failure occurs in some component. Additionally, if the cloud architecture is dealing with a shared resource pool, security and multi-tenancy need to be integrated into all aspects of the process. Services must be seen to be secure and reliable to gain the trust of the users that their data and applications are secure.
  2. Selecting the right provider:
    Know what your exact cloud computing needs are so that you can dictate the type of services you choose from your specific provider. Aspects such as available data storage, pricing structure and accessibility need to be clearly defined before taking the first steps. For basic storage, software-based cloud offerings such as Dropbox may work best. If you are looking for more than basic data storage, with an IT infrastructure and on-demand access to virtual servers, vendors such as IBM, SmartCloud Enterprise, GoGrid and Amazon Web Services would be best. If you want access to specific business solutions you should be turning to the Saleforce and other SaaS providers of the world. In terms of pricing, do thorough research before making a decision and make sure that you are only paying for what you use. Ideally, the pricing scheme should come with options to add services as needed. Pricing for cloud implementation services varies significantly, from as low as $1 per month to $100 a month.
  3. Business advantages:
    You are taking the big step of cloud implementation but what you want to focus on is how the move will affect your business and help expand it. In this case, you need to listen to what your service provider is promising you. Make sure that your provider is not focused only on technology outcomes. It may deliver excellent technology but may not be relevant to your business. Opt for a provider whose solutions can help you with high customer retention or streamlined product delivery. For this, choosing a service provider with your specific vertical market as a focus could help. To gain maximum benefits from cloud implementation, try to communicate your business objectives clearly to your provider. Thus they can be more involved in the process of business expansion and give their inputs during their time of decision-making.
  4. Regulatory compliance:
    The Cloud move can involve the transmission of data across uncontrolled internet connections that are susceptible to interception and monitoring. Most cloud-based services are secure and use different forms of encryption either via web-based communications (eg. SSL or TLS over HTTPS) or through secure applications. However, the effectiveness of the encryption may depend on a number of factors and the actual algorithms may fall short of the Federal Information Processing Standards (FIPS) encryption requirements. At the same time, cloud services that make use of proprietary transmission software may require validation in order to meet the Government standards. Hence it is very important that the cloud provider you choose has all the provisions to maintain regulatory compliance so that it meets any applicable industry regulations, allowing your business to grow smoothly and flourish.

Cloud computing is a big move. Data centers are now delivering highly reliable and highly scalable services to clients, but it is up to those enterprises making the move to pick the right service provider and the right set of features their business demands. Look at the factors of value to your business and make your choice – and if you have already made the move then help the rest of us out – let us know how you went about making your choice?

What CEOs of eCommerce Companies are Thinking This Holiday Season?

“The reason it seems that price is all your customers care about is that you haven’t given them anything else to care about.” – Seth Godin

It’s that time of the year again. As Halloween slips by CEO’s of eCommerce and consumer internet-focused companies may be forgiven for having some scary visions of their own. There is so much at stake for these companies in the period between Thanksgiving and through till Christmas and the New Year, that some may wonder why this is called the Holiday Season. Don’t believe me- look at the numbers. The National Retail Foundation has reported that retailers could make as much as 30% of their entire annual sales in this short period. The number for online sales could rise this year to touch $117 billion, according to the NRF. Clearly, you cannot afford to have any problems slowing you (or your site) down at this time – your whole year could be a wash if something like that happens.

So what are these CEOs with scary visions thinking of at this time? Based on the conversations I have had with people like this over the years, I can narrow it down to 4 areas that seem to be the top priority.

  • Performance and Scalability:
    This is a serious issue considering the site is going to be hit more frequently and in much larger numbers at this time of the year than at any time in its lifetime before that. There is a very real impact if the site slows down, underperforms or crashes at this time. Kissmetrics has reported that each delay of 1 second in page response causes as much as a 7% reduction in conversions. The analysis is that for an ecommerce site selling $ 100,000 per day, this delay of 1 second could cost a whopping $ 2.5 million in lost sales annually – not a hit you want to take. They also report that customers have high expectations of speed, 47% of them want a web page to load in less than 2 seconds and as many as 40% of them will actually bail on the page if it takes more than 3 seconds to load.

    The CEO’s concern thus, is the site should allow for the greatly larger number of users, potentially in concentrated bursts and within a reasonably small time-window. Performance and load testing of the site and every component on it thus becomes critically important. Testing the site so it doesn’t pack up under the pressure of the sharp scaling that it will encounter at this time also becomes key.

  • Security:
    The enemy is at the gate – at least that’s what the consumers think. The highly visible coverage of credit card fraud, data loss, and identity theft has made consumers wary and this is impacting their buying behavior. A 2015 study by Bizrate Insights found that as many as 60% of consumers surveyed believed that online stores were just not doing enough to protect their card and personal information. This lack of confidence reflected in 34% of them expressing a hesitation to buy online.
    Those leading online retail companies have security front and center on their list of priorities. The cloud that hosts their site, the technologies their site is built on, the payment infrastructure, the individual components of the site, and even all the bells and whistles the site employs have all to be designed to be secure and rigorously tested to validate that they are, indeed so.
  • Usability:
    We live in an age of busy people with short attention spans. A famous book on design principles actually propagated the maxim “Don’t make me think.” This is the age of the impatient consumer and the “ease-of-use” factor of the website is an overriding concern. The Baymard Institute’s “Ecommerce Checkout Usability” survey from last year found that over 1 in 4 consumers abandoned their shopping cart without completing the order because they found the checkout process overly long or complicated. It’s not just the loss of revenue due to lost sales that motivate such sites to improve usability – there’s money to be made too. Defaqto Research has reported that 55% of the consumers in their survey wanted a better experience so much that they would be willing to pay more to get it.Taken together this represents a powerful motivator for the CEOs of these eCommerce companies to invest time, money and design effort to make their sites more intuitive, easy to navigate, and friendly. Testing of the UI, obviously plays a big role in that process.
  • Mobile:
    This is almost a foregone conclusion that your consumers are on the mobile. Statista has reported that over 75% of US internet users access the internet from their respective mobile devices. These users are spending money while online too, already about 28% of total online spending is from tablets and smartphones – projected to touch $ 200 billion in 2018. Then, there’s the consequence of not being on the mobile bandwagon. MoPowered found that 30% of all consumers abandoned their transaction if the experience was not optimized for the mobile.The challenge for the CEOs of these ecommerce and consumer internet companies is how to stay in front of this mobile game? There are some many mobile devices out there, multiple operating systems (well, at least 2), device capabilities, form-factors, and other such factors to worry about. Testing whether your site performs well across all of these options has to form a significant time of the testing strategy.

These CEO’s of consumer internet and eCommerce companies obviously believe Seth Godin, that’s why they worry about the entire consumer experience on their site and the value they have to deliver. Given the importance of the holiday shopping season, it’s issues like those listed here, that could well be occupying their hearts and minds, on that quest.

Testing Strategies For The eCommerce Shopping Season

eCommerce is now a high-octane space with almost all retailers vying to make a winning online presence. As the holiday season kicks into gear with Black Friday and Cyber Mondays and continues on to Christmas, online retailers have to ensure that just like their brick and mortar store, their online store too is ready to service the heavy footfalls in the weeks ahead. Large retailers such as Walmart and Target too have worked on their online stores to avert incidents that they faced in 2015 where their eCommerce stores buckled under the pressure of heavy traffic on Black Friday and Cyber Monday. eCommerce companies, both big and small, are working to improve the digital experience provided by their eStore by improving reliability and increasing capacity to handle and manage high traffic. An Adobe Digital Insights report shows that approximately “35% of Americans are ready to shop right at the dinner table to ensure a good deal”. A Synchrony Financial survey shows “that more than half of holiday shoppers say the best deals are online, and 37% report they plan to do more of that this year, given the pretty much, anytime, anywhere convenience.” It thus becomes essential that the online retailers provide a seamless and hassle-free shopping experience for greater profits in the holiday season.

While having a great digital strategy forms an essential part of increasing sales in the holiday season, one way eTailers can ensure that their online store performs optimally is by testing their website. In this blog, we take a look at a few testing strategies for the eCommerce shopping season.

  • Catalog and segment infrastructure – ease of use:
    Given that the number of items and related discounts increases considerably during the shopping season, eTailers must ensure that all these items are displayed correctly on the Products page. Along with this, testing that all products and the associated discounts are reflecting correctly, the cataloging of the products has been done correctly and that product browsing is easy become imperative. Testers also have to ensure that all search options reflect and display correctly, the number of products on a particular page present themselves correctly, there is no duplication of products on the next page and ensure that pagination and filtering options work in harmony so that the user can browse through the website with ease.
  • Load and Performance testing:
    Research from load balancing and cyber security solutions company Radware showed that slow eCommerce websites contribute to 18% of shopping cart abandonments. Thus, testers need to make sure that the website loads fast, especially when the traffic to the website is high. Testers should look at the historical data and then assess the spike in the traffic that can be expected on the website. Along with the traffic, testers need to test the web application components such as the hardware, the database, and the network bandwidth to assess if these can handle the anticipated load and accordingly adjust the application’s performance profile. Additionally, testers will also need to assess how many concurrent requests the system can handle at maximum load, assess if response time for all test paths is acceptable, assess the reason for poor website performance such as large data sets, or browser incompatibility etc. Extensive load testing will also determine if the website needs to deploy more load balancers to eliminate the problem of refused connections that ultimately leads to disgruntled customers.
  • Mobile testing: estimated that Mobile sales grew by almost 53% from 2014 to 2015 on Cyber Monday and accounted for USD $514 million in revenues. According to a report by Dynatrace, over 50% millennials (who are the largest and fastest growing demographic in the US) who use smartphones do more holiday shopping from their mobile devices than in-store. Thus testers have to make sure that they do not ignore performance testing for mobile and ensure that the mobile application or the mobile website does not crash under peak pressure. Along with the overall performance testing for mobile, testers have to also assess problems and get solutions for problems such as mobile latency and conduct mobile network speed simulations for optimal performance.
  • Shopping cart and payments:
    Testers have to make sure that all products in the shopping cart display correctly when the user proceeds to checkout. Given that people are pressed for time, they need to ensure that the checkout process is smooth and that all discount codes reflect correctly. Regression testing with all active and inactive codes thus becomes important. Testers also have to make sure that the discount codes are not putting an undue amount of load on the database as this too, can impact the performance of a website. Finally, they need to check that all the payment systems in use function appropriately even during peak traffic.
  • Security:
    eTailers also must account for the security of their customers. Data from ACI Worldwide reveals that while eCommerce transaction grew by 21% in 2015 from Thanksgiving, the attempts grew by just 8%! Testers thus should make sure that the security layer is not compromised by ensuring secure handling of incoming and outgoing data, doing more penetration tests to identify vulnerabilities and taking a multi-layered approach to security.

With eCommerce sales estimated to cross $414.0 billion by 2018, making sure that your eCommerce website performs according to the expectations of the customer, especially during the holiday shopping season becomes imperative. By taking a methodical and planned approach to eCommerce testing, eTailers can make sure that they can unwrap the holiday season with profits.

Test Design – The Crucial Step to Test Automation

Recently, our test automation experts were having a conversation with an organization which was restarting its test automation project. The discussions started with the company narrating how their earlier test automation project failed miserably after 13 months incurring them huge costs, lot of time, and worse of all, loss of faith of the team in test automation. The company had started their initiative with the purchase of expensive test automation technology. Then they wrote automation scripts for most of the manual test cases and started running those. The end result? A huge number of automated test scripts which require high maintenance, require human intervention to run and are not useful for the product at all.

The problem we see in most scenarios is that the discussions around test automation start around what to automate and what not to automate. Ideally, what needs to be defined is what to test and what not to test. That’s what is called test design. Test design is a crucial phase for testing. Test Design involves analysis of the product specifications and coming up with test cases to validate the product functionality. This is 100% human effort and cannot be automated. It requires the involvement of domain experts, software development experts, and testing experts who need to work together to prepare a test plan with great attention to detail. Test design is what makes or breaks the success of test automation.

“More than the act of testing, the act of designing tests is one of the best bug preventers known. The thinking that must be done to create a useful test can discover and eliminate bugs before they are coded – indeed, test-design thinking can discover and eliminate bugs at every stage in the creation of software, from conception to specification, to design, coding and the rest.”

-Boris Beizer, Software Testing Techniques

An effective test design involves defining the test cases to test the software. The test cases should be created in such a manner that those can easily read, written, and maintained. The objective of the test cases should be to not only find the bugs in the product but also improve the overall product experience for the user. Test case maintenance is one of the crucial aspects which is often overlooked. Test Design needs to consider this aspect and make sure that the test cases are designed in such a way that those are easily maintainable, even by the people who did not create those in the first place. Especially in the case of test automation projects, the test design should aim towards reducing the maintenance costs for test development. The test design needs to align with the business goals such as faster time to market, more test coverage, and increased team confidence.

Similar to a software development project, the test automation project also needs to go through design, development, architecture, and maintenance. The test automation needs to have product development mindset and the test automation suite needs to follow the product roadmap.

Typically, a good test design involves –

  • A detailed thinking and design of what to test, how to test, designing of the test, and the execution plan.
  • Test authoring using method such as Model-based testing, boundary value analysis, action-based testing, error guessing etc. and defining of the keywords and action works.
  • Test case design for bug identification, impact, and maintainability.
  • Having a standard set of guidelines for writing the test cases.
  • Grouping of tests cases into small modules and suites.
  • Test case writing based on test objectives, steps, test data, and validation criteria.


In a nutshell, a good test design is a crucial step in test automation to achieve meaningful test coverage, find defects in the software, and build confidence in the testing team. It achieves more accuracy, is effective, and involves low maintenance. Contrary to common belief, a good test design is not very hard to do. It just requires dedicated thinking, patience, domain knowledge, understanding of good testing practices, and knowledge of design guidelines. Go for it and you will see the returns very soon!

Making Your Global QA Team A Super-Success

“More and more, in any company, managers are dealing with different cultures. Companies are going global, but the teams are being divided and scattered all over the planet.” – Carlos Ghosn

While Ghosn was mainly talking about automobiles, the software development business is one that has really taken to this distributed, global way of working. Software maker Atlassian surveyed 1300 companies engaged in software development and some of the results suggest as much. 72% of the companies confirmed that their software teams were spread across multiple locations and as many of 17% of the surveyed companies said that more than half of their entire team was remotely located. Given the rate at which companies are setting up offshore development centers and outsourcing to locations like India, it’s fair to assume that the locations being spoken of include some global ones. So, when software development, and by extension software testing teams, are global, what can you do to ensure they function effectively?

Here are some things we pay attention to – chances are you will find them useful too. By-the-way those looking for a list of the technical skills that go into the making of a global QA team are likely to be surprised by this post – our experience is that while tech skills are mandatory, other things make the crucial difference.

  • Clear direction:
    If everyone knows exactly where the destination lies it’s more likely they will tread the same path to get there. This identification of the strategic direction and business objectives and a clear articulation of that to the entire team will help ensure that everyone pulls in the same direction. In approaches like Agile or Continuous Development, where the product under development goes through many iterations in short spans of time, it is even more critical that everyone has a clear understanding of the strategic objective that are driving all the smaller changes.
  • Unified team:
    The teams and individual members at the global locations have to feel like they are part of a unified effort and not bit contributors at some outpost. This means that special effort has to be made to make them feel that way. One good way to do that is to apply similar policies across the locations for things like measuring effectiveness and productivity standards, rewards and recognitions, training opportunities and yes, even seemingly trivial things like availability of “swag” from the logo store – this really works. One note of caution, though, there are cultural issues specific to each global location and it is important to make allowance for those while defining.
  • Communication:
    As must already be apparent from the earlier points, communication between the engineering and business leadership and the remote teams is critical. The communication between the development teams and testing teams across locations and even between the elements of the testing team itself that may be spread across global locations has to be clear, frequent, structured and goal-oriented. If the efforts of everyone across the locations has to be aligned, it’s imperative to keep everyone updated on progress, possible obstacles, changes in strategy or operational tactics and especially with follow-throughs on actions involving contributions from other locations. Everyone should be clued into what everyone else is doing – just as would be the case if they were all located at the same place.
  • Process:
    The engineering leadership needs transparent visibility into what’s happening at all times across teams and the teams themselves need help to be able to steer a common course, irrespective of location. Defining (and of course, following) a clear process for all the testing teams to follow and including regular milestones and touchpoints to objectively measure progress along the way is the most effective way to help address both those needs.
  • Give Ownership with Accountability:
    Talking about achieving milestones, something we have found essential always is to inculcate a sense of ownership in the remote, global team. That means they have to own the milestones, along with the accountability for achieving them. This would require an effort to make the remote testing teams relatively self-sufficient or autonomous in day-to-day decision-making and making them responsible for the results they are expected to deliver.
  • Leverage technology:
    Technology can be an extremely valuable tool in achieving each of the points mentioned above. Great tools exist for everything – version control, test-case management, bug tracking, collaboration, and even communication. Many of these tools can be integrated with each other for greater effectiveness. In the Atlassian study quoted earlier, 82% of those surveyed, had integrated their source code management with a build system, issue tracker or both. Whatever be the technology chosen, it is important to bring teams across locations onto the same platforms in the interest of consistency.
    Working with global teams are a foregone conclusion today – chances are if you aren’t already working with them, you soon will be. While the factors highlighted here could help those starting out on this road, let’s ask those who have been doing this a while what are their secrets are? How did you make it work?

A Great Time to be ThinkSys – A Personal Note

Followers of my occasional posts on the ThinkSys blog site may find this post a bit out of character. I have usually spoken of issues that have equal parts technical or business impact, with a slight tilt towards software testing. This one though is more or a personal note. The last few weeks have been great for the entire ThinkSys team and I wanted to take this opportunity to say just why.

I’m just back from very busy few days in the company of the world’s leading software testing minds at TechWell’s StarWest. Like last year, ThinkSys was a platinum sponsor and, like last year, it was well worth the effort. We met a whole bunch of old friends and made a load of new connects. Clearly, the ThinkSys story of a focus on excellence, cost-effectiveness and efficiency in the QA, Test Automation, and web development services we offer has resonance.

It is this focus that has really made the last few weeks extra-special though. Over this time, we were selected to receive 3 major awards that really validate everything we are doing, as far as I am concerned.

First, we were selected for inclusion in the Inc 5000 – an annual compilation by the prestigious business publication, of the fastest-growing private companies in the US. Eric Schurenberg, President and Editor-in-Chief of Inc. has been quoted as saying, “No one makes the Inc. 5000 without building something great — usually from scratch.”. When you consider that companies like Microsoft, Oracle and have been included in this list in the past, you can understand the reason for our pride.

Apart from our presence in the US, our primary development center is in India. The region is important to us – as a market as well as a talent pool to hire from. In the first context, our selection as one among the Red Herring Asia 100 award is very significant for us. This is a list of the fastest growing technology companies in Asia. It’s a great honor to be included in this list because of the onerous selection criteria. Red Herring is known to evaluate companies across over 20 separate factors and the companies go through a 3-step review process, designed to eliminate everyone that don’t fully fit the bill. Red Herring says those that do make it could be in for “explosive growth”. That’s great for ThinkSys, I guess!

Our development center is located in the National Capital Region, just outside of Delhi India. In an accolade that, perhaps, represents our contribution to wider technology services sector in the area, global analysis and analytical research company, Worldwide Achievers, awarded us as the “Best Emerging Software Development Company in Delhi/NCR”. The award is decided after comprehensive market surveys and a great deal of research. We got this award from a representative of the highest level of the Indian government at the Worldwide Achievers Business Leader’s Summit & Awards 2016. My view is that this award is a sign of our growing influence as an employer as well as a thought leader in the technology areas we operate in – at least within the region we operate in.

All in all, this has been a great couple of months for ThinkSys. That said, and as I look at the first of our trophies sitting in our offices with pride, I take inspiration from knowing that the best has still to come. So with a promise to look at the future, let me end by thanking every one of our employees, associates, customers and everyone else who thought well of us – you have an equal share in our success!

TDD – Myths and Misconceptions

Test Driven Development (TDD) which is also referred to as test-driven design is a widely-accepted methodology in the field of software development. It typically relies on repetitive unit tests which are executed on the source code being developed. In a way, this is to start with the end in mind – designing the test cases that have to be “passed” for the code under development to be found “acceptable.” The idea of TDD is to expedite the development and testing process even though the tests may not be perfect in the first iteration. With each cycle of new code development, refactoring is done for the tests and the similar tests are executed. Finally, the iterative process is run till the desired functionality starts to work as expected. Despite being an integral part of the approach adopted by many software development teams there are certain myths or misconceptions that are associated with implementing TDD. Let’s take this chance to look at some of them, and hopefully, put them to rest.

Employing TDD is a time-consuming process

TDD actually comes into its own when it comes to measuring long-term benefits. The focus of every organization’s management team is to be able to deliver the end product within the agreed timelines. Adding TDD to the picture could increase the overall time-estimates and can thus be viewed as a time-consuming process. However, implementing TDD has long term advantages, as it helps to increase the overall productivity of the developers. TDD reduces the number of defects that are detected only once the application is deployed in production. Post production, much more time is spent in investigating, isolating and fixing the issue, which could have been avoided by using TDD. Having production issues also tarnishes the image and reputation of the organization and eventually leads to an unsatisfied customer.

R Jeffries and G Melnik, writing for IEE have reported that TDD helps in decreasing production bugs by 40-80%. Of course, it can add 10-30% to the initial development time and cost but considering the factors like defect fixes, and maintenance it is justified to implement TDD for the first phase of project development. In fact, the benefits of TDD have been tested on real projects by several companies and they found that the process is enormously beneficial.

Writing test cases before the code, is not feasible

With the introduction of the Agile approach, the process of software design has moved from the waterfall model to more iterative model. It is a challenge sometimes, to convince the developers to write test cases as the design is being built, but this helps developers to consider multiple scenarios which may lead to failure situations in real time. The idea of TDD is that it paves a way for the developers to think in a way which eventually leads to better design. TDD adds efficiency in code creation as there is an immediate feedback from the unit test. Also, developing code with unit tests allows parallel development and reduced debugging and testing time.

TDD is a software design method for better design

Technically, TDD is not really a software design methodology but it does lead to better software design! By developing your code using the TDD way, it automatically leads to a better-designed code as it is not possible to create meaningful test cases for poor or low-quality code. But the application design still needs to be catered to by having a clear idea of things like data structures, design patterns, overall system architecture, feasibility, and scalability considerations.

TDD always ensures complete code coverage

The goal of every testing organization implementing TDD is to have 100% code coverage. Code coverage is useful in the context when there are well-written tests. Writing incomplete or bad tests will not help in the above situation. Ideally, code coverage should be measured on a line or snippet if the covering tests are passing. Code coverage cannot guarantee 100% sufficiency. Usually, TTD code is likely to have a near-perfect code coverage but it may possible that a perfectly covered code will not necessarily be sufficiently tested.

James Grenning, who has written a lot about TDD said, “The best TDD can do, is assure that code does what the programmer thinks it should do. That is pretty good BTW.” TDD is a proven concept and is embraced by many developers and organizations. If this is the approach that works for you then go ahead and join the legion of followers – if the objections you encounter are from among those listed above, then you know just how true they really are.

Is Apple All Set to Own Enterprise Mobility With iOS 10?

At the recently concluded Apple’s WWDC, the iPhone7 was the object of everyone’s attention. During the conference, Apple also announced its first iOS double-digit release, the iOS 10. Even though not much time was spent discussing its fantastic capabilities, it seems clear that this major iOS update indicates that Apple is warming up to the enterprise. The business focus that iOS 10 demonstrates is hard to ignore with expanded EMM Enterprise Mobility Management) functionalities and greater device interoperability. Apple had made its enterprise intentions apparent last year by entering into a partnership with Cisco. With this partnership, enterprises can take advantage of the joint solutions that deliver an “experience that wouldn’t have been possible for either company to deliver alone”, according to Rowan Trollope, senior vice president, and general manager, Cisco. This year, with the iOS10 release, we can see how iOS devices will get IT capabilities and transform businesses through mobility. So what makes iOS 10 enterprise ready?

A mature and complete Enterprise Platform to develop fully vetted and integrated enterprise apps

With iOS 10, Apple has introduced a complete enterprise platform. This platform leverages their partner network to provide their clients with an ecosystem to enable business processes with enterprise mobility. Customers can now focus on meeting business objectives and spend less time worrying about backend management. The partnership with Cisco aims to ensure that mobile management is smarter, faster and easier for IT administrators, developers, and end users. Apple is also offering their enterprise customers a variety of integrated apps that meet their business requirements. This includes apps from leading vendors such as MobileIron, Cisco, IBM etc. who are a part of their Mobility Partner Program (MPP). With the help of the AppConfig Community, which is a collection of EMM (Enterprise Mobility Management) vendors and app developers, Apple enables developers to develop, configure, and secure mobile apps for the enterprise by employing the extensive app security and configuration frameworks available in the OS.

New developer features
The iOS 10 adds new features to the developer toolkit making it more enterprise efficient. The CallKit API with Cisco Spark allows VoIP developers to build apps that allow customers to use native iPhone apps to initiate VoIP calls. With this update, Apple demands that all apps submitted to the App Store by 2017 include ATS (App Transport Security) to prevent vulnerabilities and ensure secure and encrypted web connections. iOS 10 also introduces the Messages app store that allows developers to create apps with broader capabilities like powerful animations that can be used in iMessages.

Apps Open to Third-Party Developers
By opening up Siri to developers in iOS 10, Apple is looking to give users the benefit of an even better experience. Sirikit gives developers access to a variety of features as well as the full intelligence already built into Siri. Apple is opening up Siri as well as Maps to third-party developers so that they can bring out new features in these apps. Apple is also giving the Messages app to developers, thus making them compatible with more enterprise apps. Developers can download Sirikit from Apple, which provides an extension that communicates with Siri even if the app isn’t running.

Enhanced interoperability
With the iOS 9.3 update, Apple offered new ways to manage multiple devices. With the iOS 10, Apple enables these devices to work together harmoniously within the enterprise. Downloading new apps and data sharing across all Apple devices becomes much easier and this allows better continuity for users. Features such as Universal Clipboard allow users to move back and forth between Mac and iOS devices seamlessly without using solutions such as AirDrop. To prevent unintended data loss, Apple partners such as MobileIron are employing stricter copy/paste controls. The Auto Unlock feature allows users to unlock a Mac without typing a password, by using Apple Watch. Once the device is unlocked the Watch needs to stay in contact with the user’s skin to stay unlocked. If it loses skin contact then the device locks automatically and will then require the owners’ PIN. The user also has to be within three meters of the Mac to unlock it.

With iOS 10 Apple also takes care of the security of data in motion by supporting ‘VPN IKEv2 EAP-only’ mode. This makes VPN access to access corporate data, a critical component of enterprise mobility, secure.

Unified platform
With the iOS 10, Apple seems to be on the road to, what Gartner calls, “Unified Endpoint Management”. Now enterprise customers can make updates to any device running macOS Sierra using MDM. IT admins will also be allowed to implement policy restrictions to iCloud Keychain sync, iCloud photo library, Apple Music, Notes sharing, Find My Mac etc. by supporting new payloads to configure the IP firewall on Sierra. So we can expect EMM platforms to secure and manage most of the Apple devices, both desktop, and mobile leading the shift to desktop and mobile convergence.

Reduced workloads
With the iOS10, enterprises can expect to see workloads reduce to almost half since managing iPhones and other Apple devices can be done on the corporate network itself. This also helps in reducing network latency, employee downtime and increased helpdesk calls that stem from sluggish app performance due to network incompetencies.

Enchanced Security
Enterprises call for greater security and this is also an area of focus. Now wifi can be controlled (Enabled/disabled) through geo-fencing within MobiControl. Apple is also allowing supervised control over the apps installation and removal and control over system apps like – FaceTime, Siri, iTunes, iCloud, GameCenter, which will further enhance the enterprise user experience.

That apart, there have also been significant enhancements to security and performance improvements in areas like faster roaming, reduced web browsing failures and more reliable calls. It does appear that with the iOS 10, we are witnessing the evolution of the iOS into a mature enterprise-grade platform that offers more variety, ease of use and secure work-ready apps. With its new features and focus on integrating the new iOS into the partner ecosystem, it seems that Apple is looking to establish a beachhead within the enterprise. The world is watching with interest.

Some Numbers That Tell the Evolving Software Testing and Automation Story

In today’s rapidly changing world of testing practices, the focus of the software industry is now moving away from the Center of Excellence approach to a more wide-ranging strategy using Agile or DevOps practices. So much so, that based on the World Quality Report for 2016 the budget allocation for testing has increased 9% year-on-year. Considering this shift it would be interesting to see some of the top trends and likely evolutions for testing and test automation and some key numbers that tell the tale:

  • Development and Operations; DevOps
    DevOps is a software development technique which relies on tight coordination between Development, Testing, and Operations. The goal of DevOps is to improve the communication and collaboration between the departments so as to achieve, almost absurdly, faster software delivery. Out of all the participants of the World Quality Report survey, it was identified that 67% are using DevOps practices. This has a fundamental impact on testing as the basic idea of DevOps practice is to start testing as early as possible in the development cycle. This ensures that issues which are encountered only when the entire development is completed are identified much in advance. DevOps also helps teams focus on quality from day one of the projects, instead of finding out about the defects much later in the software development lifecycle.
  • Test Driven Development (TDD)
    TDD is a methodology that, in many ways, relies on the repetition of a short development cycle. The customer requirements are converted into very specific test cases and it involves execution of the new test cases. 39% of the participants in the World Quality Report are using TDD and with good reason too. A paper published in the Empirical Software Engineering journal mentions that ‘TDD seems to be applicable in various domains and can significantly reduce the defect density of developed software without significant productivity reduction of the development team.’ The study also compared four projects at Microsoft and IBM that used TDD with similar projects that did not employ this testing practice and found that defects per thousand lines of code reduced anywhere from 40% – 90% as compared to non-TDD projects.
  • Selenium and related tools
    Selenium is a software testing framework which is designed primarily for web applications. The advantages of using Selenium is that it is an open source tool which means there is no license cost involved. More and more companies are now embracing this open source technology for their automation testing needs. Gartner’s Magic Quadrant for Software Test Automation makes, the rather bold, claim that by the year 2020, Selenium WebDriver will become THE (emphasis ours) standard for functional test execution. and that this will be the key differentiator between vendors that cannot provide this test functionality.
  • Mobile testing in the cloud
    Mobile usage is rising, as everyone knows, but it is also true that the demands are becoming more complex. HTTP Archive reported that in the 4 years after 2011, the average size of a page served to the mobile rose from 340 KB to 1080 KB. 20% of these pages also had more than 100 resource requests and nearly 2 in 3 had 10 or more JavaScript requests. This suggests a growing emphasis on mobile testing. Mobile testing in the cloud can help the testing teams to be more productive and efficient. Usually, mobile testing lab devices can be established at a centralized location within the organization. With a variety of mobile cloud testing tools, the testing teams can now be spread across different locations where they can all access any of the devices from the mobile testing lab via the cloud. Cloud testing also gives the flexibility of not having to buy the same device more than once; instead, you can have one device in the cloud and the testers can reserve times for when they wish to test on them.
  • Digital transformation
    The CIO Report of 2016 reported that 2 out of 3 CIOs were already measuring their efforts against different KPIs from 12 months ago, directly because of concerns related to Digital Transformation. Digital transformation, today is becoming a key aspect for enterprises to achieve their business goals and to avoid getting disrupted by newer, more digitally adept business models. Many companies are now relying on this transformation approach for their success. With digital transformation, the challenge of software quality is a critical factor that organizations need to consider. The digital agenda is focused on user experience testing, as organizations feel the need to provide the best and most innovative digital experience to its end users. Security testing is also a vital consideration when it comes to Digital Transformation. The centrality of testing in the new, digitally transformed enterprise is visible in World Quality Report’s projection of a 40% rise in the budgets for QA & Testing by 2018.

Overall, the top priorities for QA and Automation testing executives continue to include Security, Customer experience, and efficient delivery. The main intention of QA and Automation testing is evolving to provide a seamless experience to the end users which in turn helps to improve the corporate image, end-user satisfaction, and ultimately business results.

Here’s What We Look for While Hiring Great Software Testing Pros

By: Rajiv Jain (CEO, ThinkSys)

“Great Vision without great people is irrelevant” – Jim Collins

Today the consumers of technology are everywhere…from the biggest enterprises to the youngster with a mobile phone. As we get increasingly conversant with technology, it becomes essential to develop good and robust software products, that are bug-free, error-free and fast. Today, no organization or individual has the time, patience or bandwidth to deal with slow performing, buggy apps that can pose a productivity or security threat. To enable organizations to deliver to the demands of speed and performance one thing climbing up the priority chart is testing. With the proliferation of test automation, many feel that the role of a tester is just that of a facilitator. This could not be further from the truth. Testers are the secret weapons that guarantee that the products that we develop perform to their optimal capacity.

Everyone looks for testing skills, experience, and certifications but there is something more that sets the great software testing pro apart. Here, in no particular order, are some of the things we look out for when hiring these super-testers:

  1. Prioritization Skills:
    A software tester has to have the ability to bite off more than he can chew and yet be able to chew and swallow properly. Software testers have to deal with a lot of workloads such as designing strategy, writing test cases, creating reports etc. In most cases, they have to work against tight timelines. Hence, having great prioritization skills and good time management are the hallmark of a great tester – somehow their day seems to have just that many more hours. He/she needs to understand what needs to be tested and when, which task should rank lower on the priority list, which tasks should be automated and which should be manual and which tasks need to be addressed immediately.
  2. Attention to Detail:
    Testing demands an eye for detail. It can be easy to miss a small bug but that small miss can compound into a bigger problem in a very short span of time. So while identifying glaring issues seems easy enough, a testing pro will be able to identify the not-so-obvious issues, the small stuff that can snowball into a big impact on the application at hand. Sherlock with an instinct for code is what we need.
  3. A Creative Mind:
    Test professionals who can think beyond what the software is expected to do or what the users expect from the software are the ones who truly shine in this area. They should have a creative mind that allows them to think of new ideas to test a product and come up with ideas to use test cases in different scenarios. Coming up with new ideas to test a product ensures that the product performs optimally when it does get stressed.
  4. Curiosity:
    Curiosity is a great trait in a tester. Only when a tester has a keen and curious mind will he/she try to think out of the box, look for problems in the unlikeliest of places and come up with intelligent solutions. A curious mind also gives the tester the ability to see the big picture and connect the dots to see how each action is impacting the project
  5. Ability to think from the user’s perspective:
    A tester should be able to think like a user. In order to achieve end user satisfaction, testers have to think from the perspective of the user. They should be able to get into the user’s shoes and walks all around in them to identify how the users want to communicate and interact with the product. So great testers have the ability to understand their target audience and assess how their user base will be interacting with the application and then develop test cases and test plan to get complete coverage and get an application that performs optimally.
  6. Ability to ask questions:
    A tester-extraordinaire has to learn to ask questions…a lot of questions…even questions that might seem irrelevant to another without feeling awkward or uncomfortable. By learning to ask the right set of questions, a tester can understand the requirement, understand the changes that need to be incorporated and implement them, understand the bigger picture and define the scope of testing.
  7. Data analysis skills:
    Great testers don’t only write test cases but also have the ability read and analyze test data generated from a particular application. If they have identified a ‘non-reproducible bug’ then they should have the capability to analyze the test environment, the test data, the interruptions in code etc. to assess where the bug generated and fix it accordingly. They also need to analyze data generated from testing script execution during test automation to find loopholes and performance gaps and identify ways to increase testing productivity.
  8. Reporting skills:
    Testing demands a lot of reporting. Hence having good reporting skills, possessing the ability to report negative things in a positive way, write status reports for clients and especially say a lot, but in a succinct, crisp manner etc. should come naturally to a tester.
  9. Thirst for knowledge:
    To be a good, well anything, and especially a good software tester, one has to possess a thirst for knowledge. Knowledge, for a tester, does not end with mastering one scripting language but continues as they have to stay in step with the latest technological developments and automation tools to keep coming up with new ideas to test better.
  10. Be a great negotiator:
    Negotiating well should come naturally to a tester as they have to negotiate with different people at different stages of a project. They have to have the skills to convince developers (who usually are quite possessive about the code they develop) that there is a defect in the code, explain its impact and get the defect resolved.

Another thing that a tester has to have in abundance is ‘perseverance’. Only when a tester is patient enough to explore the software constantly to find bugs and make new improvements and take all the testing challenges and complexities in the positive spirit can he/she become a great tester. At the end of the day, we go by the motto, “When we move our focus from completion to contribution, life becomes a celebration”. Our best testers feel the same way!

Node.js–A Great New Way to Build Web Apps

The past few years have seen Node.js explode on the application development scene like a superhero. Despite its humble beginnings with Yammer and Voxer, Node.js managed to establish its authority quite fast and is now seeing great mainstream adoption with giants like Walmart and PayPal putting their trust in it. Netflix too moved its website UI to Node.js from Java. Once the underdog, Node.js has established its credibility and superhero status within the enterprise and has an increasing number of developers adopting it to build fast and scalable web applications with this open source cross-platform runtime environment. Ever since its launch, Node.js has been seen as this cool and trendy server side platform that managed to attract the developer community. The great thing about Node.js is that it’s core functionality has been written in JavaScript making it a great choice for developing real-time applications. Apart from this, Node.js is packed with a host of other features that make it ideal for building web applications. So what makes Node.js so great?

  • Neutral Language
    Since Node.js supports JavaScript, the same language can be used in the front end as well as the backend thus breaking down front and back end development boundaries which make the development process more efficient. Considering that JavaScript is a language that is used by a majority of developers it saves them the trouble of translating code and helps in managing development time and cost. With the Node.js, framework developers do not need to translate the logic from their head on the server side framework and also do not have to translate the HTTP data from JSON (the data-interchange format) to the server side objects. Since Node.js is generally understood both by Java and .NET camps it makes it easier for the developers to deploy it on both Unix and Windows infrastructures.Node.js uses Google’s V8 engine on Chrome which is written in C++ and has an exceptional running speed as V8 has the capability to easily compile JavaScript into native machine code without any problem.
  • Scalability
    Node.js effectively solves the concurrency problems that plague developers unilaterally. The problem of concurrency in server-side programming languages often causes poor performance and impacts the throughput and scalability of an application. Developers get an event driven architecture with Node.js and non blocking I/O API which takes care of these issues. Node.js is also built to handle asynchronous I/O’s from the base up. This helps in managing many web development related problems.
    Node.js also splits a single process into multiple processes called workers by adopting the cluster module. This modules allows developers to create ‘child processes’ that can function under the ‘parent process’ and can communicate with the parent Node by sharing the server handles and by using IPC (Inter-process communication).Furthermore, applications in Node.js are easier to scale since the developers write simple code and Node.js simply takes over from there. Instead of using processes and threads, Node.js uses a simple Event Loop and defined call backs where the server automatically enters the loop when the callback definition ends for scaling efficiently. Simply put, Node.js helps applications execute common tasks like reading or writing to the filesystem, reading or writing to the network connections or to the database with ease and speed and makes applications capable of managing a large number of simultaneous connections with high throughput.
  • Built-in support
    Node.js has an inbuilt support for package management using the NPM, the Node Package Manager, a tool that comes as a default with Node.js installations. The NPM module is publicly available, has reusable components and is easily available through an online repository of over 300,000 packages with dependency and version management. This ecosystem encourages sharing and is open to all and gives developers more prospects to create effective solutions by giving them the opportunity to update, share and reuse code with ease.
  • Great for real-time web applications
    Developing real-time web applications such as chats and gaming apps in Node.js is extremely easy. Developers do not need to concern themselves with things such as low-level sockets and other such protocols. It allows the developers to write JavaScript on both the server and client side and facilitates automatic data synchronization by automatically sending data between the client and the server and ensures that data changes on the server are immediately reflected at the required points.Applications in Node.js are composed of small modules that are piped together which ensures that, unlike monolithic applications, these applications do not creak under unseen weight and stress. This also makes adding new functionalities to the application much easier as the changes do not need to be made deep inside the codebase.

    Along with all this, Node.js can also be used as a proxy server if the enterprise does not have the infrastructure for proxy. It also allows for actual data streaming and does not take HTTP requests and responses as isolated events and reduces processing time. Node.js applications are also capable of dealing with high loads. In 2013 Walmart put their entire traffic on Black Friday through Node.js and ensured that their servers did not exceed 1% of server utilization despite having 200, 000, 000 users online.

    Node.js has all the features that make it most appealing to the developer community and also renders it enterprise ready – it’s easy to scale, is secure and easy to learn. It also takes care of low latency issues that plague most tech companies because of the asynchronous input-output operations feature. Adding it all together, it is clear that with Node.js organizations can achieve more as it half the number of developers can be used to build products. It also reduces the number of required servers to service a client and increases app performance by reducing load times by almost 50%. Given the increasing industry confidence on Node.js, it is quite clear that its future is indeed bright.

Software Testing Life Cycle

We all must be aware of the tree’s life cycle where a small seed goes through distinct phases to gradually grow and develop as a large tree.

The similar concept of life-cycle is also followed in the software engineering field, mainly in the development life cycle and the testing life cycle where former perceives the gradual development of the business or functional requirements into a software application and the latter one visualizes the testing of the software application from a scratch to the release of the quality software application. Since, our article is not concerned to development life-cycle, we will discuss about testing life cycle only.

What is Testing Life Cycle?

Development life cycle is followed by the testing life cycle. A testing life cycle comprises of several phases and activities aligned in a sequential manner to initiate, execute and terminate the testing process.

A software testing process could be initiated as soon as the development process begins and may be carried out in parallel to the development activities. It can be understood through V&V development model, where a corresponding test methodology is defined for each development phase.

Now, coming back to the testing life cycle, it mainly consists of following phases in a subsequent manner.

Let’s find out what each phase consists and is responsible for.

1. Requirement Analysis:-

The very first phase of the software testing lifecycle involves the study and analysis of the available requirements and specifications. Both functional and non-functional requirements are being viewed and study from the testing point of view, to find out the testable requirements i.e. those requirements which may produce results on feeding with the input data.

  • On the availability of requirements and specifications.
  • When the application architecture is available.


  • Brainstorming sessions for the requirement analysis and feasibility.
  • Identifying and sorting out the requirement priorities.
  • Creating the requirement traceability matrix (RTM).
  • Identifying the suitable test environment.
  • Identifying the requirements acceptable for the automated testing and the manual testing.


Requirement analysis stage visualizes the combined efforts of QA team, project manager, test manager, system architect, business analyst, client and the major stakeholders so as to have greater understanding of the requirement and subsequently the better outcomes.


  • Testable Requirements.
  • Requirement Traceability Matrix(RTM)
  • Automation feasibility report (if applicable).

2. Test Planning:-

With the information gathered about the requirements in the previous phase, QA team move a step ahead in the direction of planning the testing process. Basically, a strategy or strategies is/are defined and described for the testing process/activities.

When to go for it?

  • On the successful completion of the requirement analysis phase.
  • When the testable, refined and clear requirements got defined and specified, i.e. on the availability of requirement documentation.
  • Good understanding of the product domain.
  • Availability of Automation feasibility report (if any).


  • Scope and objectives are outlined.
  • Deciding the testing types to be performed along with the specific strategy for each of them.
  • Roles and Responsibilities are determined and assigned.
  • Identifying the resources and testing tools required for the testing.
  • Estimating the time and the efforts to carry out the testing activities.
  • Defining and detailing the test environment.
  • Defining the time schedules.
  • Entry, exit criteria along with the suspension and resumption criteria is defined.
  • Planning the training activity and sessions required by the testers(if any).
  • Risk analysis is being done.
  • Change management process is specified and described.


As per the requirement and the availability, QA Manager or QA lead is accountable for planning the testing process.


  • Test Plan documentation
  • Time and effort estimation documentation.

3.Test Case Design & Development:-

The requirements has got analysed and accordingly the QA team comes out with a test plan. Now, it’s time to do some creative work and to give a shape to this test plan in the form of test cases. Based on test plan and detailed requirements, test cases are designed and developed for the purpose of verifying and validating each and every requirements specified in the documentation.


  • Test cases are designed, created, reviewed and approved.
  • Relevant existing test cases are reviewed, updated and approved.
  • Automation scripts (if any) are developed, reviewed and approved.
  • Relevant test data are generated or imported from the development environment.
  • Test conditions along with the input data and expect outcome for each test cases are defined and specified.


Generally, the testers have the job of writing the test cases under the supervision of QA lead or QA manager. However, the testers may be accompanied by the developers in generating the effective automation test scripts.

When to prepare/create test cases?

  • On the availability of software requirement specification (SRS) and business requirement specification (BRS).
  • When the test plan is ready.
  • Automation feasibility report(if any) is available.


  • Test cases including automation scripts.
  • Test Coverage Metrics.
  • Test Data

4.Test Environment Setup:-

The software testing process needs an appropriate platform and environment encompassing the necessary and required hardware and software, to create and replicate the favourable conditions and intended environmental factors to perform actual testing activities i.e. execution of the developed test cases on the software.

  • Test data is set up.
  • Test environment checklist is prepared and the required hardware and software are aggregated.
  • Test server is setup and network settings are configured.
  • Test Environment management and maintenance process is defined and described.
  • Smoke testing of the environment to check is readiness.
  • Testers are being equipped with the bug reporting tools.


QA team under the supervision of QA manager sets up the test environment

When to set up Test Environment?

  • When test data is ready for use.
  • Test Plan documentation is available.
  • Needed resources such as hardware, software, testing tools & framework, server, etc. are available.

However, the test environment set up phase may be carried out concurrently with the test case design & development stage.


  • Test Environment is set up and ready to execute tests.
  • Smoke Test Results.

5.Test Execution:-

With the test cases, test data and the suitable test environment, QA team is now ready to try hands on some actual testing activities. The test execution phase involves the execution of the developed test cases with the help of test data in the set up test environment.

  • Test Cases execution as per the test plan.
  • Comparison of actual results with the expected outcomes.
  • Identifying and detecting defects.
  • Logging the defects and reporting the identified bugs.
  • Mapping defects with the test cases and accordingly updating the requirement traceability matrix.
  • Re-testing, once a defect gets fixed or removed by the development team.
  • Regression testing(if required).
  • Tracking a defect to its closure.


Test Engineers are deployed to carry out the task of test case execution.

When to go for the test execution?

Being equipped with the test strategy, test plans, test cases, test data, properly configured and set up test environment along with some other needy resources, the QA team can kickoff the test execution process.


  • Test Status and results.
  • Bug or Defect Report.
  • Complete and updated Requirement Traceability Matrix (RTM).

6.Test Closure:-

The completion of the test execution phase and delivery of the software product marks the beginning of the test closure phase. This phase perceives the meeting and discussion amongst the QA team members with respect to test execution and its results. Apart from the test results, other testing related parameters are considered and reviewed such as quality achieved, test coverage, test metrics, project cost, adherence to deadlines, etc.


  • Retrospection of the whole testing process.
  • Test Life Cycle exist criteria is evaluated along with some other essential aspects such as test coverage, quality achieved, fulfilment of goals and objectives, critical business goals, etc.
  • Need to change the exit criteria, test strategy, test cases, etc. are discussed.
  • Test Results are analysed and reviewed.
  • All the test deliverables such as test plan, test strategy, test cases, etc. are collected and maintained.
  • Test Closure Report and test metrics is prepared.
  • Defects are arranged severity wise and priority wise.


Generally, the QA lead or the QA Manager is responsible for preparing the test closure report.

When to perform test closure activities?

Generally, the test closure activity begins after the completion of test execution activities and delivery of the software product. However, it is not necessary to carry out the closure task only after the delivery of the software application. It may be performed after closure of the testing activities due to some other reasons such as achievement of targets, cancellation of the project or when the product needs update, etc.


  • Test Closure Report.
  • Test Metrics.
  • Learned process.


In nutshell, it may be concluded that similar to development life cycle, testing life cycle also consists of several phases and each phase counts a large number of activities to strategically and orderly carry out the testing process in an effective and efficient manner and subsequently ensuring maximum productivity and quality achievement.

Keeping Your Testing and Automation Strategy Relevant

“Evolution is the secret for the next step” – Karl Lagerfeld

The need to change and the ability to adapt to change has been the reason why today, we have grown so much. Without going into the details of human evolution, which will not find relevance in this piece, we want to mention the evolution of technology and how phenomenally it has grown over the last few decades. This growth is only because someone, at some time, identified a ‘chance’ of growth…of doing something better. Bringing about all this change was not easy, yet it was essential and imperative in order to stay relevant.

how to keep your test automation strategy relevant

Just like everything else, change is also essential for a Test automation strategy to stay relevant. Considering the dynamic business environment that we have today where technology changes and advancements are the norm, frequent product upgrades and product evolution are inevitable in order to stay relevant and ahead of the curve. Keeping this in mind, it is imperative that in order to ensure a high-performing and flawless product, having a strong testing strategy is a must. Since the consumer does not have the time, energy or bandwidth to deal with a product riddled with bugs that lead to slow performance, testing assumes an even more important role. For testing professionals, thus, this means building a testing suite that can enable this change.

  1. Much like testing an inventory, testing professionals too, have to look at the overall test strategy as well as the detailed test plans and test cases to identify which test plans will remain relevant in the long run and which plans will become obsolete. Taking this big picture view, with an eye out for the details, on a regular basis thus becomes essential to release upgraded products that are bug-free and with development costs under control.
  2. When there is a product upgrade, changing the entire test strategy can become a problem that can snowball into a big expense. Much like product development, having a monolithic test plan with many interconnected parts only slows down the process, since if one test fails the entire testing suite comes to a halt. Having smaller and independent test cases addresses this problem and increases the efficiency of the testing suite.
  3. One more key element is the test data being considered. As the product evolves, the conditions it operates under will change and the testing has to address those changed conditions. Creation of test data to assess which tests can do so, getting frequent data dumps from the production team, assessing how the test can be spread the tests to other environments, using data dumps from production to access relevant test data are some issues that test automation suits should cover to ensure pertinence.
  4. Then the big item – test automation! What must be considered is the level of automation incorporated into the overall testing strategy. There has to be a healthy balance of manual and automated testing. Test cases that have to be repeated continually, cases that are manually time-consuming and need a speed of execution or cases that are difficult to perform manually are ripe for automation.
  5. One way to ensure continued evolution of the automation suite, in line with the evolution of the product it is testing, is to consider the test automation suite too as a product; one that needs frequent iterations and upgrades. Keeping this relevant starts with selecting the right testing tool, designing the framework and features, test bed preparation, scheduling and timeline management and iterating the deliverables of the testing automation are some of the key contributors of a successful test automation strategy that stays relevant. Along with this, testers have to focus on the maintainability of the testing automation suite so that when the product changes, your test automation suite can adapt to that change and deliver what is expected with minimal effort. Just as software needs to be maintained, a test automation suite, too, needs maintenance. Thus, treating your testing assets like any other piece of software becomes a critical contributor to the relevance of the test automation suite. Charting the lifecycle of the testing suite, much like software maintenance, and identifying maintenance needs such as preventive maintenance, corrective maintenance, and adaptive maintenance are important for the longevity of a test automation suite.
  6. Creating test automation suites that anticipate, or allow, changes in the UI also ensures that the test automation suite can work with future versions of the product. To make this happen, some of the things that testers can do are build test suites that divide the test into individual parts, allow keyword-driven testing and support multiple scripting languages amongst others. What testers need to bear in mind at all times is, that for dependable test automation suites assessing the validity of the testing suite with each product iteration reduces the burden of test maintenance. Adding the needed tests and removing the ones that are redundant on a proactive basis after each product release increases the life of the test automation suite.


By building a strong test strategy that can remain relevant for a long time, testers provide developers the confidence to refactor legacy code and build solid and stronger products. Building a strong test strategy or a robust automation suite is a labor of love for testers that needs a lot of thought and nurturing. Once that is achieved, the test automation suite and the tester want nothing more than to live happily ever after.

Made For Each Other – How a Dating Site Leveraged the Power of Test Automation?

Over the years, test automation has become an indispensable part of the product development strategy of most companies. In these days of extreme pressure to go-to-market faster, it seems no test strategy is complete without an automation component. This is the story of one of our key customers, a (or maybe THE) leading dating and matchmaking site in the game today, and how they gained from adopting a sustained, strategic and comprehensive test automation strategy – oh, and of how we helped them get there.

Our story starts when an updated version of the dating site was in the works. A consumer internet site like this operates under some fairly extreme conditions. Getting your product out into the hands of the target users at a pace faster than the competition is vital. Then there is the need to provide an incomparable, error and trouble-free user experience – in this social age, even the smallest problem could cause users to switch to alternatives. So the name of the game is fast, extremely high-quality product development.

When we entered the stage our client was facing a dilemma because of the limited number of QTP licenses they had. The choice was to either take much longer to run their 8000+ scripts with the existing number of licenses or to buy additional licenses. In the first case, they stood to lose a possible market opportunity because of the extra time the release would take, and the second meant a significant expenditure. Neither were very palatable options.

We approached the problem differently. We suggested a shift to Selenium. Being an open source framework for test automation with wide acceptability in the market, this made sense from the cost point of view definitely. Of course, opting for an Open source option like Selenium had is pros and cons with some possible compromise on features, support and possibly a loss of accumulated knowledge. The migration of 8,000 QTP scripts to Selenium, was also a massive task in itself. The major challenge was that there was very limited time to complete the conversion and every day spent on making this effort meant adding costs. The capabilities of the manual testing team, a substantial part of their workforce, the effective and efficient channelization of their efforts and the best utilization of their time over the period of transition was also a big concern.

Our solution was to leverage our own Krypton, a feature-rich, hybrid test automation framework which is a scriptless test-automation tool. A glimpse at some of these features – parallel test execution is possible within Krypton, a sure shot time saver. Krypton also supports all the browsers in the market and it also features keyword driven testing, automated reporting capability, and parallel recovery. Using Selenium as the base and Krypton, the team managed to complete the migration and create the brand-new test automation framework.

Now that this phase of the effort has been put to bed we can look back at the results with pride. For one, the automation framework delivered on its promise of achieving greater code coverage in a relatively short span of time. This means a better-tested product going out into the market faster.

Krypton has been designed in such a way that it was relatively easy for the manual testers to understand. The analysis at the end showed that getting them working with this tool after training was achieved within a very short span of 2-4 weeks, a crucial aspect in the timely delivery of the product.

By using this solution, the company was able to achieve tremendous savings in terms of license costs and the efforts of the manual testing resources – almost $3 million by their own estimates.

Today, the new framework is implemented as the preferred choice for automated testing. It has helped the customer to increase the efficiency, effectiveness, and overall coverage of the testing effort. The fact what we are most thrilled about is the impact we have managed to deliver in such a high-pressure situation. As far as this dating site is concerned, perhaps Krypton was that special one it had been waiting for, all its life and now, as a result – Love is all around!