Automation in Testing over Test Automation

Recent trends in the IT industry are quite indicative of the well known fact that test automation is becoming all too ubiquitous and that the skills which made the testing an investigative and unique profession (also associated with manual testing) are on the wane. However, our aim is not to focus on things which make automated testing gain an edge over manual testing. There is a stark differentiation between Automation in testing and test automation. How the two affect the development of a software product, what are the experiences encountered and the like ….this article aims to answer and highlight some of these queries.

The global big shots in the IT business, reason that the reputation of the company is at stake and hence, they have no alternative, than to blindly go for the automation test suites which will completely nullify the probability of product recalls, repairs and the ensuing erosion of market share and user trust. After all, it conveniently ticks the boxes of 100 % coverage and fast scripting of test cases.

The companies also come up with a genuine economic concern for lack of faith in putting automation into testing. They suppose that even if they somehow pool together the right king of skillful manpower to develop an automation suite, then it will more or less become a product in itself, with all its support and maintenance costs. What would then happen to the Return of Investment, they argue..

What the approach should be for Automation in Testing

  • When strategizing towards automating a test, there are a whole lot of preconceived notions related to additional costs, that we need to unlearn.
  • The whole line of thought process should revolve around questions like “How can i test faster?” How can i extend its reach for better coverage?
  • The functionality should be targeted for testing from the lowest possible level
  • To come up with a test tool that can combat with irregular tweaks in system, part of the approach should be to overcome the boundaries of development and testing
  • Some of the tools/ techniques which can be a friend in our endeavour are: data management, state manipulation, critical bug tracker and log file parsing.
  • Documents like old user manuals can also aid us to a large extent.

A jigsaw puzzle Architecture:

Just as there are disparate parts to a jigsaw puzzle set, all the aspects towards automating a test case need to be considered as individual parts of the same puzzle. For example, a testing code that manages data, a test code that creates browser, a code for managing user interactions on the page. In the end, a code that will handle reporting of results .Put together all these test codes and scaling up this approach can build the test tool we need. While we are at it, we need to carefully analyse the hurdles we encounter and go deeper into them. For instance, when we design a testing tool for API testing is taking too long, we need to delve into the causes which are impeding it’s quick response time.

Where test automation may not be the best fit?

  • It doesn’t make any sense to automate tests that require a single repetitions or a couple of iterations.
  • When the project requirements for an application are very dynamic. Since the design of an automated test suit requires sufficient resources and enormous labour, it goes against sound economic sense to plan one for a platform that requires regular changes and tweaks.
  • Unless the automation in testing is not started with close co-operation with the development team at every stage of the development cycle, it becomes difficult to come up with a complete and effective test tool. In that case why get into it in the first place?
  • Maintenance costs of these suits are quite significant and can be a dampener while gauging the overall economics of the project.
  • It can be time consuming to automate tests and therefore its usage should be limited to high priority ones.

Regression Testing vs. Retesting – Know the Difference

Regression testing vs. Retesting?

Is there a difference at all? Is regression testing a subset of re-testing? Quite a few times, testing teams use these two terms interchangeably. However, there is a vast difference between these two types of testing. Let us have a look –

regression testing vs retesting

Regression Testing


Regression testing is a type of software testing which is carried out to ensure that the defect fixes or enhancements to the application have not affected the other parts of the application.


The purpose of regression testing is to ensure that the new code changes do not adversely affect the existing functionalities of the application.


Regression testing is carried out in one or any of the following circumstances:

  • Whenever there is a modification in the code based on the change in requirements.
  • A new feature is added to the application.
  • Defects are fixed in the application.
  • Major performance issues are fixed in the application.

Regression testing can be carried out in parallel with re-testing. It is, in many cases, seen as generic testing.

Testing Techniques:

Regression Testing can be carried out with one of the following four techniques-

  1. Re-execute all the test cases in the suite.
  2. Select a part of the test cases which need to be executed with every regression cycle and execute only those.
  3. Based on the business impact, feature criticality, and timelines, select the priority test cases and execute only those.
  4. Select a sub-set of test cases based on frequent defects, visible functionality, integration test cases, complexity of test cases, sample of successful and failure test cases and then execute only such sub-set.
Test Cases:
  • Regression test cases are derived from the functional specifications, user manuals, tutorials, and defect reports related to the fixed defects.
  • Regression testing can include the test cases which have passed earlier.
  • Regression testing also checks for unexpected side-effects.
Role of automation:

Automation plays a very crucial role in regression testing because manual testing can be very time consuming and expensive.



Retesting is a type of software testing which is carried out to make sure that the tests cases which failed in the previous execution pass after the defects against those failures are fixed.


The purpose of re-testing is to ensure that the previously identified bugs are fixed.

  • Whenever a defect in the software is fixed, re-testing needs to be carried out. The test cases related to the defect are executed again to confirm that the defect has indeed been fixed. This type of testing is also referred to as confirmation testing.
  • Re-testing has to be carried out prior to regression testing.
  • Re-testing is a planned testing.
Testing Techniques:

Re-testing needs to ensure that the testing is executed in the same way as it was done the first time – using the same environment, inputs, and data.

Test Cases:
  • No separate test cases are prepared for Re-testing. In Re-testing, only the failed test cases are re-executed.
  • Re-testing does not include the test cases which have passed earlier.
  • Re-testing only checks for failed test cases and ensures that originally reported defects are corrected.
Role of automation:

There is no way to automate only the re-testing since it depends on the defects found during the initial testing.

A Cloud Migration Checklist

An increasing number of enterprises today are migrating to the Cloud. A survey conducted by RightScale, a cloud automation vendor, confirmed this trend and revealed that:

  • 93% of respondents reported that they are adopting the cloud.
  • 88% of the respondents reported using the public cloud.
  • 63% of the respondents use the private cloud.
  • 58% of respondents use both private and public cloud.

Migrating to the cloud presents enterprises with some obvious benefits. Increased availability, better performance and clear cost benefits are some of the obvious advantages of moving to the cloud. Research conducted by various agencies such as Gartner, Ovum, Forrester, International Data Corporation (IDC) and others agree – “the global SaaS market is projected to grow from $49B in 2015 to $67B in 2018, attaining a CAGR of 8.14%.” and by 2019 cloud applications will account for worldwide mobile traffic. Goldman Sachs also estimates that the “cloud infrastructure and platform market will grow at a 19.62% CAGR from 2015 to 2018, reaching $43B by 2018”

However, when migrating to the cloud enterprises have to ensure that their initial footprint in the cloud is compatible with the technology stack present in the cloud platform of their choice. They also have to ensure that the platform is able to scale comfortably to suit growing business and user requirements. Thus taking a strategic approach becomes an essential part of cloud migration and consider how the enterprise intends to do business so that this can become an inherent part of the cloud strategy.

Cloud migration is the process in which data, applications or other business elements are moved from onsite computers to a cloud infrastructure or are moved from one cloud infrastructure to another. In this post, we will shine the light on some essential components of a cloud migration checklist. We hope this will help enterprises looking to migrate to the cloud do so seamlessly and help them reap the real benefits of this move.

Network architecture:
To take complete advantage of the cloud, enterprises need to make sure that their network infrastructures are set up for this. Traditional network infrastructures may suffer poor application performance or even expose themselves to security vulnerabilities. Thus before making the move to the cloud enterprises need to make sure that their network is well-designed and cloud-optimized by ensuring routing optimization, reliability, and low latency in WAN performance, and ensuring device support. Taking a holistic approach to the network architecture thus, becomes the foundation of successful cloud migration.

Application architecture:
While moving applications to the cloud might look simple, in reality, this takes a lot of careful planning for great execution. Before migrating applications to the cloud, architects need to evaluate if legacy applications need to be replaced and assess which applications will get the most out of the cloud investment by doing an inventory assessment and then plan the application move. Typically, enterprises should avoid moving systems in large chunks and should ensure that these systems or application first-movers are not the most business critical and tricky. Mission-critical workloads, legacy application, sensitive data might not be the best first movers to a public cloud. Treating the cloud as a logical extension of the current landscape and assessing application dependency thus becomes an essential part of the cloud migration check-list.

Business continuity plan:
Having a business continuity plan should also form an essential plan of the cloud migration journey as vulnerabilities, natural or man-made (think the Japan earthquake or the Amazon outage in 2011) can sometimes disrupt business. Enterprises need to build diversity into the disaster recovery and business continuity systems and should be able to run on a number of different infrastructures. Evaluating options for business continuity and designing systems and configurations that can enable a high level of automation should find a significant spot on a cloud migration check-list.

Evaluating costs:
So, should you opt for a private cloud, public cloud or hybrid cloud? While cost efficiency is a big reason of why enterprises move to the cloud, it is important to remember that financial benefits differ from one application to another. Applications using legacy hardware can be more expensive to run in the cloud. Identifying the technical requirements, gathering performance data, and identifying if there shall be any hidden expenses when migrating to the cloud can help in planning the network and bandwidth costs and for deciding which cloud flavor will best suit the enterprise.

Governance and security:
Since traditional on-premise systems will not work as-is in the cloud, enterprises have to take a look at evaluating their governance approaches. As most of the governance responsibility rests on the cloud providers once the move to the cloud is complete, enterprises need to reshape their governance strategies to rely more on the offerings by the cloud than on their internal security. Assessing the cloud providers’ security certifications thus becomes important. Planning ahead for any fail overs, potential breaches and disaster recovery also becomes a critical part of a cloud migration check-list.

As enterprises assess the benefits and risks of a move to the cloud, it is important to note that cloud migration does not have to be an ‘all or nothing’ proposition. With careful assessment, enterprises can begin with first moving some applications and services to the cloud and continue to operate the rest on-premise. Once all the boxes in this check-list have been crossed then enterprises can avail rapid cloud transformation and embrace the power offered by the cloud.

Role of Test Automation three Testing Areas

As technology becomes more embedded into business processes, testing is becoming an area of strategic importance. While manual testing cannot be completely done away with, in order to ensure the effectiveness, efficiency and wider coverage of software testing, test automation is the way to go. Gartner notes that in order to be agile testing needs to be automated.

  • Unit Testing:
    Unit testing is a very important part of the software testing process since it is the small code fragments that are tested here. It involves checking the pieces of the program code in isolation and independently to ensure that they work correctly. Since unit tests mean checking the source code, the test suite should be a part of the build process and scheduled to run automatically every day so that testing happens early and often. Manually testing these pieces of code can be resource and time intensive and can lead to a number of errors. By automating unit tests, testers can ensure that the source code within the main repository remains healthy and error free and there is a line of defence built against bugs. While it might take 10-30% more time to complete a feature, automating Unit tests ensure that:
    1. Problems are found early in the development cycle.
    2. Ensures that the code works now and in the future.
    3. Helps developers become less worried about changing codes.
    4. Makes the development process more flexible.
    5. Improves the ‘truck factor’ of the project.
    6. Makes development easier by making it predictable and repeatable.
  • Functional Testing:
    Functional Testing, a test technique to cover the functionality of the software and ensure that the output is as per the required specifications. It also has to ensure that test scenarios including boundary cases and failure paths are accounted for. While functional testing is not concerned about the internal details of the application, it tests in great detail what an application ‘does’. During functional testing, developers need to set benchmarks that are developer-independent in order to identify what they have not achieved. So, as soon as a function is developed it needs to be tested thoroughly and this process has to continue until application development has been completed.Since the user ultimately shall be running the application on a system along with other applications and the application has to endure different user loads, the developers also need to make sure that every function of the application is crash resistant. Testing continuity, ensuring application grade tests are as external as possible, white box testing etc. are some of the areas that functional testing has to cover. Doing all of this manually can not only be time consuming but also leave room for error. A functional testing strategy should incorporate:

    1. Test purpose.
    2. Project creation.
    3. Test construction.
    4. Test automation.
    5. Test execution.
    6. Results check.

    Test automation makes it easier to perform powerful and comprehensive functional tests that will help develop a robust product.

  • Performance Testing:
    Performance testing assumes a big role in the testing and QA process as it helps in identifying and mitigating performance issues that can derail the entire application. Performance Testing can sometimes also be associated with stress testing, load or volume testing as this test aims to gauge the system’s ability to handle varying degrees of system transactions and concurrent users along with assessing its speed, stability, and scalability. Automating test process during performance testing is essential as the goals of a performance test comprise of several factors which include service level agreements and the volumetric requirements of a business. Testers need to test system transactions and user transactions concurrently, measure and reference system response time against the Non-Functional requirements, and also determine the effectiveness of a network, computer, program, device or software. During performance testing, the testers need to measure the response time or the number of MIPS (millions of instructions per second) at which a system functions. They also need to evaluate that the qualitative attributes of scalability, reliability, and interoperability to ensure the product meets with the specifications required. Understanding the content to assess problem areas, building test assets, running tests and analyzing the data to understand performance bottlenecks are some of the things that testers need to do.
    Since performance testing is intensive and exhaustive, testers should leverage test automation once the test strategy has been defined.

Taking a strategic approach to testing automation and viewing it as a part of the development process makes sure you have a product that is flawless and delivers on all the desired metrics. This ultimately leads to higher quality software, greater customer satisfaction, and higher ROI.

Choosing the Right Test Automation Outsourcing Partner

The bitterness of poor quality remains long after the sweetness of meeting the schedule has been forgotten.

The importance of software testing cannot be reinforced more. While testing is an essential part of software development, it is not the core competency of most of the companies which are into the development of the software. It involves a considerably wide range of activities such as test strategy definition, test case creation, test plan development, automation strategy, and of course, execution. Outsourcing the testing to the niche companies which have strong expertise in testing provide multiple benefits such as – faster time to market, improved product quality, and reduced testing costs. It, therefore, does not come as a surprise to know that the worldwide software testing outsourcing market is expected to grow from $30 Billion in 2010 to $50 Billion in 2020.

Choosing the Right Test Automation Outsourcing Partner

Test automation is a further niche area in the testing space where very few companies seem to have mastered the skills and expertise. Choosing a right test automation partner involves a little more careful effort. Before you finalize your test automation outsourcing partner, here are few things which you need to carefully evaluate:-

  • Business Understanding:
    Yes, testing automation involves technology but it is more of a business decision. Whether to go for Automaton, what should be the goal of automation, how does the strategy align with the business goals –  one needs to get answers to these questions while formulating the test automation strategy. Your outsourcing partners should be able to work with you to help you achieve your business goals and not just provide you “testing experts who are ready to work for 10 hours a day”.
  • Project Understanding:
    “Judge a man by his questions rather than his answers.”– Voltaire
    Every test automation project is different and it requires special consideration of various aspects such as project context, the current state of the application, future roadmap, business goals, the current state of automation etc. If a vendor comes back to you with a standard strategy, then you should be skeptical about such vendor. The right test automation outsourcing partner will ask you all the right questions to understand your project before suggesting a strategy to you.
  • Project Management and Processes:
    This might be the standard requirement for all the outsourcing projects but it is a crucial requirement for test automation outsourcing. Test automation requires proper planning and a very streamlined execution for it to be successful. When you are looking for a vendor, look for someone which has well-established project management tools and processes. The teams should be trained on those. You will need periodic reviews, regular status updates, and acceptance cycles – the vendor needs to have all the required setup in place to facilitate this.
  • Respect for Intellectual Property:
    Due diligence and respect for intellectual property is quite important for your testing outsourcing initiatives because you might share some testing artifacts, test cases, and sometimes, even the source code with the outsourced vendor. You must ensure that the vendor not only understands the importance of intellectual property protection but also has strict guidelines in place for adherence.
  • Experience:
    Nobody likes being a Guinea Pig. Evaluate the vendor for its experience of working on a variety of test automation projects and its experience of working with multi-cultural teams. We highly recommend asking for client references and case studies. Go for a vendor who has won the trust and praise of the well-known names in the industry. If you feel it necessary, go ahead and speak to their customers to understand their experience on delivery timelines, output quality, technology competency, costs etc.
  • Testing Experts:
    As mentioned earlier, test automation is a niche area and unless testing is your core competency, there is a little chance that you will be able to do a good job at that. Test automation involves definition of a strong strategy, business understanding, selection of right tools and technology, having the right team in place, expertise in test case creation and execution abilities. When you look for a test automation outsourcing partner, ensure that testing is the core competency of the company you choose- it cannot be “one of the many things they do”. Having a focus helps in the development of right processes, training, and team selection.
  • Scaling Up and Down:
    Instead of depending on one or two persons for the execution of test automation, you need a team which is well-trained on all the aspects of execution. Outsourcing needs to offer you the flexibility to scale up or scale down the team size at a short notice without a strong dependence on any particular people in the team.
  • Knowledge and Understanding of Tools and Technologies:
    Test automation requires knowledge and understanding of a variety of tools and technologies. Your test automation partner should be capable of selecting the right technology which is apt for “your requirements”. Just because they have expertise in one technology does not make that technology right for you. You must evaluate the expertise of the outsourcing vendor with various tools and technologies.
  • Know What NOT to Automate:
    More than “what to automate”, the real skill lies in deciding “what not to automate”. Look for a partner who can help you make these decisions. Sometimes, conducting certain tests manually makes more business sense. A knowledgeable outsourcing partner should help you make these decisions.

We strongly believe that the test automation outsourcing vendor cannot work in isolation. It needs to work with you as a partner to your company and needs to be invested in the project.

We hope these guidelines help you make the right decision in the selection of the test automation partner for your project.

Can test automation replace human testers?

The fact that Software testing is integral to the development and overall success of any product in the IT industry, puts Software development managers worldwide in an all too familiar dilemma. What testing strategy they should pursue? Do they hire the services of a professional testing team for their product verification? Or would they be better served using an automated tool. The debate on the use of Manual testing vs. Automated testing has been raging on for quite some time now. Before we judge the utility of either, it wouldn’t do any harm to briefly enumerate the pros and cons for both.

Automated testing

Test automation employs specialised software tools which run a set of repetitive tests with a predefined set of instructions, to compare a program’s expected and actual responses. If there is a perfect alignment between the two, then it indicates the bug free quality of the product and its readiness for shipping. If not, then the software needs de-bugging and a re-run of tests until all the glitches are rectified. Let’s take a look at some of the advantages of this approach:


 Runs quickly off the mark:
One of the biggest factors which give an edge to Automated testing over manual testing is of course, the speed. Best part is the re-usability of these tests, which is a much needed relief for those running regressions on a frequently changing code.
Automation testing can be done on different machines and operating system platforms on a concurrent basis.
It is very effective for Build Verification and Testing.
 Project size:
Automated testing is apt for large scale projects.


  •  Automation testing is not helpful with UI testing
  •  High initial cost of tools.
  •  When it comes to considerations requiring a human touch such as image contrast or text font size, automated testing cannot be trusted with the most accurate of solutions.

Top 10 Mobile app testing mistakes you must avoid

With India becoming home to the second largest user base of smart phones, the growth of mobile apps has shown an exponential growth over the past few years. Since the task of mobile app testing is as essential as the development of the app itself, it becomes fruitful to list down a few gaffes, which are routine with mobile app testing.

1. The looks are important, but then so are the uses:

Mobile app testers often tend to focus more on the UI/UX aspect of the app and forget about the basic functionality for which the app was developed in the first place! An app is only as good as its use. Agreed an eye catching user interface always attracts more users, but if it fails in providing solutions, then it’s nothing more than a dud. Therefore a balance perspective with equal focus on UI and functionality is the need of the hour.

2. Jumping to test without knowing the Sophistication involved:

More of an attitudinal thing than a flaw, it’s common to find testers run through the process of testing without arming themselves with a thorough understanding of the basic working logic in the app. A couple of tests on the UI, some feature tests here and there and ta da the report is ready for submission. Important thing before testing is to have requisite knowledge of the chief functionality, user end requirements, a vision for changes expected at the end of an update, the business requirements from the app, etc. This will eventually help in developing a wide coverage for all the intricacies in need of testing.

3. Web and mobile testing are as different as chalk and cheese:

To someone who is familiar with the world of web page testing, the approach to mobile app testing may sound similar. However, the two things are a world apart. While an average web browser may require an update once a year at the most, a mobile application requires a visit to the app store virtually every month. Therefore, a mobile app tester requires to incorporate an approach which will validate the functionality of an app in sync with its scheduled updates.

4. Online/offline plus networking issues:

While it is crucial to test the proper working of the app when the user is online and offline, a vital issue often overlooked, is its ability to work across different bandwidths. This becomes relevant for users residing in areas having a history of intermittent service provision of net/ wireless capabilities.

5. Utility of Crash logs:

Crashes are major irritants for users and one of the biggest factors contributing to a lower rating of the app. Hence, it pays to maintain a detailed report of all the crash occurrences, so that it becomes convenient for the tester to prioritise all the major bug issues and prevent them happening post lunch.

6. We can test everything:

It goes beyond the realm of possibility to test, say 100 case scenarios for an app, over 10 different devices, operating under 10 diverse working environments. The problem is compounded when the date for the launch of the app is ever so near. Common sense advocates prioritising a few critical test case scenarios based on recent market trends for devices which are most in vogue, studying user behaviours and making use of google analytics.

7. Inadequate reporting of bugs:

Due to miscommunication, inadequate knowledge or time constraints related to launch deadlines, the testers are not able to send a complete report of all the repetitive and critical bus which hampers an app’s working and consequently the developer team is unable to resolve all the issues cropped up at the time of testing. As a result, the app is released into the market replete with glitches.

8. Complexity in the name of sophistication:

Many users find it difficult to navigate their way in an app because of a congested or sophisticated user interface. Therefore, it pays to have an app that is easy on the eye for the customers and moreover, the test cases involved for such interfaces is always easy and less time consuming.

9. One tester can do it all:

It is always beneficial to have professionals having experience in working on different environments on your team. By tapping into their knowledge, a greater number of potential flaws in the app can be detected at half the time, before the pending launch.

10. Importance of customer feedback:

You have tested the app and it has been launched and your work is over, right?, not quite. The tester must make it a point to gaze through the user reviews for the products similar to the ones he tested. These are readily available at app stores such as Google play store, Mobogenie, etc.

The ubiquity of a smartphone is testament to the fact, our lives will now be increasingly be controlled by mobile apps. The importance of mobile testing can thus never be overemphasized.

Does Test Automation Have a Role in the MVP Way of Life?

By Rajiv Jain (CEO, ThinkSys)

Us veterans of the software product development game may be forgiven a bit of confusion these days. From the old days of the waterfall model of development, we are now faced with choosing between Agile development methodologies, DevOps and Continuous Delivery or perhaps rushing to churn out a Minimum Viable Product. Of these, the last, the MVP approach, is particularly favoured by startups or enterprises who seek to emulate the nimbleness of the startup. In a nutshell, the MVP is a kind of early Alpha of the product that is done fast and released early with a view to gathering customer feedback and either gain early market traction or fail fast enough to allow recovery. Steve Blank explains it well, “You’re selling the vision and delivering the minimum feature set to visionaries, not everyone.” The aim is to release, iterate and release again in short cycles. I could not find any reliable stats but anecdotal evidence would suggest that approach is rapidly gaining ground in product development – frankly, I’m not surprised considering how it gives maximum bang for the effort and money invested.
Does Test Automation Have a Role in the MVP Way of Life?
Among the MVP approaches, there are those that seem to focus on the “minimum” and are thus concerned only with seeking market validation of an idea through landing pages or crowdfunding campaigns. Ignoring those let’s focus our attention on those that focus on the “viable” and release actual products into the market. These products are generally single-feature or window-dressed versions or limited-feature versions of the vision. As such there is software design, architecture and development effort involved and of course where there is development testing has to follow. An interesting question to ponder is does test automation have a role to play in this kind of development?

But before that, one, only slightly, philosophical question is about the MVP itself. In many ways, the MVP itself is a kind of trial balloon, a test of an idea. The limited objective of the MVP is to seek customer validation and only if it passes that initial test is it on to the next stage. This suggests some strong factors in favour of test automation and a couple against it too.

While early customers do expect some rough edges to the MVP they are unlikely to forget or forgive poor quality and such early impressions could kill the product before it really lives. This means testing and QA cannot be given the short shrift even in the MVP. This then brings into focus the speed aspect. The product has to be released quickly, to start with as well in succeeding iterations as customer feedback piles up. Speed has always been one of the chief reasons to pitch for Test Automation. A natural fit it would seem!

Well, not so fast. There are a couple of other important factors that may make you change your mind. For one, the MVP approach does not lend itself well to long-term planning. This approach, by definition, does not provide any great visibility into the final shape the product is likely to take or even into upcoming versions. That’s not great news for those looking to build an effective test automation strategy given the lag time between defining what should be automated and deploying a well put-together automation suite. In the time, it takes you to develop an automation suite the test cases you are automating for may no longer exist! The other issue is applicability The shape of the product changes quickly from version to version so it would seem that many of the test conditions may not occur more than once. Clearly there’s not much value to automating test cases that will be used only a handful of times. So the prospects of test automation in the MVP scenario don’t appear that bright.

That being said though I see one possible connection, and it could be a key one. If you look at the MVP approach as a series of empirical experiments intended to define the final shape of the product, then this can become a wonderful early input into the automation strategy. In this approach, the product plans and, in turn, the test automation strategy can be much more “frozen” since the likely changes and iterations can be identified and accounted for very early in the process. This allows the test automation team more elbow room – they can plan better, design better and presumably execute better on those plans. This could be a big boost to the automation efforts.

So, my own feeling is that the process of building an MVP is perhaps too scrappy to allow for decent degrees of automation but that being said the end product that the approach itself seeks to build could benefit tremendously from a better-planned test automation approach. Let me turn to the leading light of the Lean Startup and MVP movement, Eric Ries for support. He said, “All innovation begins with vision. It’s what happens next that is critical.” Fair enough!

Should Developers Test Their Own Code?

I don’t care if it works on your machine! We are not shipping your machine!” – Vidiu Platon

I can almost see the testing manager talking (negotiating?) with the developer while going over the bug report and making this, rather exasperated, statement. Those of us who have been in software development long enough are quite likely to have encountered such intense “developer v/s tester” situations somewhere along the way. Of late the roles, or should I say battle lines, have blurred somewhat – don’t believe me? Think about how often you now come across the term Software Design Engineer in Test (SDET) – much more frequently than was the case even a couple of years ago right? In the language of car companies, the SDET is a “crossover” – someone who spans the software development and testing worlds and has bits, pieces and features of both in his job description. The SDET is a visible symbol of a discussion we are having quite often now – should developers test? Also by extension, should developers test their own code?
Should Developers Test Their Own Code?
At one level, especially for smaller software development teams, start-ups or even larger companies where software is not the core function, the attraction of having a smaller self-contained team of developers who do their own testing and validation is obvious. Costs are contained, as is the management overhead and the demands on their, more limited, internal technical bandwidth are less onerous. At the other end of the spectrum are larger software development efforts where more development-friendly test functions like test automation demand the presence of software developers on the testing team. We have always made the case that in the case of test automation the test strategy has to be much more closely coupled with the development strategy and architecture – again a function where developers could play a key role.

That being said, I believe that testing is a specialized task in itself that needs a focused testing team to action effectively. There are some key reasons why I think developers would not be quite as well suited to the task of testing their own code.

  1. Mindset:
    Most people will agree with the statement that the job of a tester is to break the software. A quality tester will try every use case and condition possible to try and make the software fail. They look for complex situations, combinations of situations, repeated applications and even heavier than expected load conditions to make the software break. In many ways, this is a completely opposite mindset to that with which the developer approaches his task – essentially the developer is always trying to make it work and that may not be the best approach for testing.
  2. Starting right:
    It’s not unusual to encounter development efforts that have been built on an inaccurate understanding of the initial requirements. A developer testing the code would generally be less inclined to check if the foundation itself is incorrect. Essentially the “bug” is the place where the design started.
  3. Proprietary interest
    It’s a rare developer who doesn’t get attached to his (or her) code. It’s something they have sweated over and as a result, to some extent, they would start testing from the position that the code works. This is likely to lead to short cuts, assumptions about simple or “trivial” things and a possible tendency to skip things that they “have fixed during coding”. Obviously, these are the kind of things that turn around and bite back later in the cycle.
  4. Big picture view:
    Most developers will have a reasonable view of the code they are working on – that unit or that piece of the puzzle. Even if that works fine in itself no software works in isolation. A Tester will usually be able to turn a wider scope and test – look at how the code works when fully compiled? Or how it works in a simulated, or real, user environment.
  5. Tricks of the trade:
    This is, of course, apparent. Testers have much more experience with the act of testing, the tools and techniques, how to log and report bugs and even common faults and the reasons why they occur. A developer would have to reinvent that particular wheel to become as efficient at the role.

So what role can developers play in the testing process? Well for starters it is apparent that they would be extremely well suited to unit testing and, at least, validation testing of their code. That apart there seems to be a good case for pairing developers and testers – the model works well in the Agile context due to the short sprints and multiple iterations. This, though, is the subject of another post. In closing let me share what Brian W. Kernighan said, “Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.” I’m not saying I agree – just that he’s the computer scientist, not me.

Software Testing Trends That Will Rock 2016

Nelson Hall predicts that by 2017, the software testing market size will be $34 Billion. The rapid increase in newer technologies, mobile, and Internet of Things is having a significant impact on the testing industry as well. The testing market is also undergoing a transformation to meet the increasing demands of the digital transformation trends. Testing, today, has moved away from product-centric and has become more customer-centric.
software trends 2016
Let us have a look at some of the latest trends in software testing which, we think, will have a significant impact on testing in near future –

  1. Internet of Things(IoT):
    Connectivity has gone much beyond computers, smartphones, and tablets. Today, the most common day-to-day devices are connected to each other and with various sensors to generate a huge amount of data. With greater connectivity, there are challenges of accuracy and sanity of data being collected. IoT has introduced a significant impact on various types of testing – Because of connectivity with a larger variety of devices, increase in the number of scenarios, use of different operating systems, IoT has enforced the use of automation in testing to ensure wider code coverage. Usability testing also plays a critical role because the IoT devices typically have a minimalistic design and a wider array of buttons and controls. Performance testing is taken to a new level with IoT because the performance of any operation depends heavily on the Internet connectivity and bandwidth. With information being transmitted over the network on a continuous basis, security testing becomes critical with IoT applications to avoid data breaches and malicious attacks.
  2. Mobile Testing:
    With mobile becoming the frontier in digital transformation, mobile app testing has become a critical factor in the digital strategy of enterprises. Today, with more and more users storing critical and sensitive information on their mobile devices and using mobiles to connect with numerous applications, mobile security has got a renewed focus. A comprehensive security and penetration testing of mobile apps has become critical. With the wider adoption of mCommerce and mobile payments, QA professionals have got a newer set of challenges to test in terms of security, usability, and performance of mobile payments. Voice command operated applications like Siri and Google now have opened a whole new area of testing for the QA professionals. Testing these applications for various voice commands in different voices and accents has become a daunting task. With the proliferation of Internet of Things and exponentially increasing combinations of mobile devices, operating systems etc., organizations have started to adopt mobile cloud environments to test their applications to reduce the costs and time.
  3. Agile Development and Continuous Delivery:
    Agile development and continuous delivery allow organizations to push out releases on a more frequent basis. With the increase in the number of devices and platforms, the chances of bugs compromising the end user experience have increased multi-fold. In order to avoid the scenario where testing is an after-thought, organizations are focusing on integrating testing into the development process through the use of a unified testing platform. The objective is to run quality tests in parallel to development and make testing catch up with agile and DevOps. There might be increasing demand for testers to have more development skills. Automation is sure to play a critical role in this.
  4. Renewed Focus on Security Testing:
    A recent study by HP found that 70 percent of devices in the Internet of Things are vulnerable to security problems such as missing data encryption, compromised password requirements, and access to the user interface without passwords. No wonder, MarketsandMarkets predicts that the global security testing market is expected to grow from $2.47 billion in 2014 to $4.96 billion by 2019. With new technology, wider adoption of mobile devices and wearables, and Internet of Things, it is important to focus on building secure software which connects these devices.
  5. Focus on automation in testing over test automation:
    Richard Bradshaw, at the Agile Testing Days 2015, talked about how using the term “test automation” restricts the exploitation of real benefits of automation. According to Richard, automated tests cannot think, they only check for things – and therefore, it should be called checking and not testing. Bradshaw is strictly against keeping the goals like ‘fewer testers’ or ‘100 percent automation’ in testing. He mentions that the role of Automation is to support the testers and not to replace them. He insists that instead of saying Test Automation we should refer to it as Automation in Testing – this mind-shift can help in considering several other ways in which automation can help in testing. We have personally listened to quite a few videos and podcasts by Richard and his talks do open up the mind. Do look those up on online.

Our Favorite Test Automation Tools and Technologies

A quick search for “Test Automation Tools” on Google returns 8,21,00,000 results. With new tools and technologies mushrooming every single day, it is quite obvious to get confused or feel overwhelmed when it comes to selecting the right tool or technology for your needs.
No, this blog is not about recommending any particular tool or technology. Neither does it provide any comparison of tools. There is never a one-size-fits-all condition. The choice of any particular tool depends a lot of business requirement, goals, and test automation objectives. Additionally, the success of test automation projects does not depend only on the tools or technologies. There are a lot of other factors too.
In this blog, we have enlisted some of the popular test automation tools and technologies. We have been using these automation tools since quite some time and have seen great results while automating softwares-

favorite test automation tools

  1. QTP [Now known as Unified Functional Testing (UFT)]-
    A product by Hewlett Packard (HP), QuickTest Professional aka QTP allows testers in automation of functional testing using Visual Basic Scripting (VBScript) as the language.

    • QTP is extremely easy to use with easy options for navigation, results validations, and report generation. Even the testers with no programming knowledge can use QTP.
    • HP offers a full-fledged support to QTP users through email, phone, and an active online community.
    • Getting started with QTP is really easy because of the many in-built features, functionalities, and configuration options.
    • It provides support for functional test automation and regression test automation through advanced solutions.

    QTP is popular amongst the testers because unlike some other tools which require you to code using Object Oriented Programming languages, QTP uses Visual Basic Scripting (VBScript) language which is easier to learn and code.

  2. Selenium :-
    Selenium is probably the most popular test automation tool.

    • Choices, Choices, and Choices – Selenium offers unlimited choices to the testers. It allows the creation of test scripts using any IDE such as Netbeans, Eclipse or Visual Studio, it provides support to a variety of Operating Systems like Windows, Linux and Macintosh and it also offers the testers the flexibility to choose any programing language like Java, C#, Ruby, Python, Perl or PHP!
    • With support for a wide range of popular browsers, including IE, Google Chrome, Firefox, Safari, and Opera, Selenium has gained popularity amongst the testing engineers.
    • Through a very large, vibrant, and active user community, Selenium offers detailed documentation and support.

    Available for free as Open Source, Selenium frees up the organizations from the worries of licensing costs and budgets.

  3. Telerik Test Studio :-Touted as one of the easiest software testing tools, Telerik Test Studio offers ‘Navigate, point and click’ functionality making it easy to generate any kind of test cases including functional, performance and load tests.
    • It is an all-in-one testing software for functional, load, performance and mobile app testing.
    • For data-driven testing, it supports Excel, XML, CSV and Databases including MS SQL Server, Oracle or MySQL.

    With support for in-depth functional testing for web apps, desktop applications, mobile apps, HTML5, AJAX, Silverlight, and WPF apps, Telerik has gained popularity amongst the testing community.

  4. TestComplete :-
    A functional automated testing platform by SmartBear Software, TestComplete allows QA engineers to create automated tests for softwares(Desktop applications, web apps, and also mobile applications). It supports keyword-driven operations for tests recording, scripting or for manually creating the automated playback and error logging.

    • It works very well with .Net and Java applications, websites, and also ActiveX in webpages.
    • It interfaces well with MSBuild, Team System, JIRA, HP Quality Center, and also run nUnit/jUnit scripts.
    • There are a lot of tutorials and general support available.
    • The error detection offered by TestComplete is quite precise and the reproduction of recorded scripts is very fast and quick.

    Apart from support for many languages such as VB. NET, C#, JavaScript, Delphi, C++, etc., TestComplete also supports various UI controls such as Flex, Flash, Sencha ExtJS, Silverlight, and jQuery.

  5. Protractor :-
    With the rising popularity of AngularJS applications, the need for a tool for testing AngularJS-based applications has increased and Protractor, an open source end-to-end testing framework, is gaining popularity there. Built on top of Selenium WebDriver, it uses Node.js framework. It can be installed as a standalone test runner or can also be embedded in tests as a library.

    • It supports a variety of behavior-driven development test frameworks including Jasmine, Mocha, and Cucumber. These tools provide additional syntax and reporting tools which are useful for better test writing and management.
    • With automatic waiting, developers don’t need to worry about adding sleep and wait commands manually – it optimizes sleep and wait times and speeds up the testing.
    • It allows both unit and functional tests. It also runs the tests against application in a real browser.

    With AngularJS-specific locator strategies, Protractor provides out-of-the-box testing for AngularJS-specific elements, making it a great tool for testing AngularJS applications.

  6. Krypton :-
    Termed as a ‘new age Automation Framework’ built on top of Selenium, Krypton by ThinkSys is being used by many Fortune 1000 companies for testing their web and mobile applications. It is an excellent automation solution for testing websites, web-based applications, mobile websites and mobile native apps and helps in improving time to market as well as ease of maintenance.

    • Automation test cases can be written as a set of commands. Krypton “Execution engine” automatically reads the test cases and executes these commands.
    • Testers can write their Test Cases in Excel Spread Sheets in an ‘easy to understand’ JavaScript which are reusable skills.
    • It has an in-built ability to perform backend validations.
    • It solves for many web browser and mobile device and browser idiosyncrasies.
    • By creating reusable objects, it allows for easy maintainability.

    Krypton helps the manual testers become experts in automation in 2-4 weeks without the need of knowing programming.

  7. Appium :-
    A discussion about test automation will be incomplete without the mention of tools for automation of mobile app testing. Appium is one of the popular mobile test automation technologies. It is an open source test automation tool by Sauce Labs which helps in the automation of native and hybrid apps.

    • It supports execution on iPhone devices as well as iOS simulators.
    • Along with the support for multiple JAVA and .NET IDEs, it also supports open source IDEs.
    • Since it uses Selenium as the backend, testers can avail the Selenium functionality and use that knowledge base for testing mobile apps.
    • It is completely cross-platform and supports Android as well as iOS operating systems without having to write separate test scripts for each platform.
    • Thanks to its use of JSON Wire protocol, it supports a wide variety of programming languages.

    Appium allows testers to write the test codes using a variety of languages such as RoR (Ruby on Rails), C# or Java without the need of modifying the apps just for the automation. purposes.


8 Reasons Why Your Web Apps Need to Use AngularJS



Developers love AngularJS. They just really really love it. Don’t believe us? Look at the GitHub activity. AngularJS probably has the highest number of contributions than any other competing JavaScript framework. Check out Google searches or even StackOverflow mentions – AngularJS is all over.

Job trends on, a developer jobs site, also show a strong bias towards AngularJS than any other framework.
angular job trends graph
So what are the reasons behind this popularity? Why is this the self-proclaimed “superheroic JavaScript framework” gaining so much attention and traction? Why is the popularity of AnugularJS exploding? I have also heard that hundreds of tickets for ng-conf, one of the first AngularJS conferences, were sold out in just a few minutes!

Let’s take a look at what makes AngularJS so popular –

1. The Google Power :-
The very fact that AngularJS is built and maintained by a dedicated and talented team of Google Engineers, assures that you are working with reliable and efficient code which can offer you the required scalability for your project. Along with the solid foundation, the Google team brings in the required knowledge and support. So along with the vast open community, you have the Google engineers available to answer your questions and help you use AngularJS optimally.

2. Faster Page Loading :-
AngularJS is massively popular amongst single page sites or web applications. One of the popular reasons behind this could be the faster page loading. By saving the templates (which is essentially the HTML content) in the cache, AngularJS cuts the server reload time and removes the need to hit the server each time to retrieve the template. This makes the page loading faster.

Developers can architect the front-end application in such a way that the coupling on the back-end is minimized, which further improves the page loading performance.

3. Rapid Development :-
AngularJS on front-end along with Node.js in the backend can help the developers in rapid development by automating the repetitive tasks. This frees up the developers time for the actual development and building of business logic. With the use of data binding and Dependency Injection of AngularJS, developers can eliminate much of the code that has to be written. The two-way data binding reduces the amount of code which has to be written to maintain cohesive agreement between the model and view.

Some of the tools which help in faster development are –

  • ng-annotate – This helps in maintaining Angular dependency injection while minifying the JS files.
  • JSLint – Helps in validating the JS Code.
  • Sass/Less compiler – Helps in CSS compilation.
  •  Bower – It is a very good Package Manager for AngularJS third party components (For example: Nuget and NPM, a NodeJS package manager)

4. Flexibility in Development :-
Most of the other frameworks require you to first split your app into MVC components and then bind those components using the code. This increases the amount of coding. In case of AnugularJS, all you need to do is just split your app into MVC components – the rest is taken care by AnugularJS itself. It manages the components on its own and acts as the pipeline to connect them. Since the connection is handled by AngularJS, developers cannot write incorrect code that breaks abstractions just to make the components fit easier.

5. Collaboration :-
For projects where there are multiple developers working on the project, there are often issues about managing dependencies. AngularJS makes it relatively easy to manage the dependencies. It allows you to break down various actions into their own services and sub-controllers. Developers can independently code and test without messing with each other’s code. This is a bliss in the case of larger projects. It automatically introduces a nice workflow and processes which allow frictionless collaboration.

6. Mobility:-
AngularJS is extremely useful in building hybrid mobile applications. Before AngularJS, developers needed to use JQuery mobile framework for building hybrid mobile applications. However, JQuery mobile framework makes the code complex and it is also not very efficient for multi-page applications. AngularJS makes the code less complex and helps in creating highly efficient mobile apps. Some of the advantages of using AngularJS in mobile development are –

  • Reduced DOM manipulation
  • In-built routes provider
  • Seamless gesture handling
  • In-built transition animations
  • Performance improvement through the use of saving of templates in cache
  • Access to local storage

7. Testability:-
AngularJS is known as unit testing ready. It has been written keeping testability in mind. AngularJS links the whole application together using Dependency Injection or DI. The DI manages the controllers and scopes of the application.

Since all the information to the controllers is passed through DI, the unit tests can be performed by injecting mock data into the controller and measuring the output and behavior.

AngularJS also offers a mock HTTP provider for injecting fake server responses into controllers for unit testing. This is anytime faster than the traditional way of web application testing where individual test pages had to be created to invoke one component and check the interactions with it.

8. Less Coding:-
AngularJS requires considerably less coding as compared to other frameworks. Because of less coding, it becomes easier for the developers to keep a systematic track of the functions.

Here are some of the cases where AngularJS requires less code –

  • Views are defined using HTML to make them more concise.
  • Developers do not need to write their own pipeline – it is offered readymade through Dependency Injection.
  • With no need of getter/setter functions, the Data model is simple.
  • Developers don’t need to provide data manually to the view – it is facilitated by the data-binding feature.
  • Directives are written separately from the application code – this allows parallel development without any worries of integration issues.
  • Without changing the controllers, data on the view level can be manipulated using filters.

AngularJS encourages object oriented design thinking on the client-side which makes the application more maintainable. And the icing on the cake is that one can be proficient in AnugularJS in a reasonably less time!

The Mobile Choices You Have To Make

By Rajiv Jain (CEO, ThinkSys)

“The future of mobile is the future of online. It is how people access online content now.” – David Murphy, Founder and Editor of Mobile Marketing Daily

Earlier this year a major shift in the balance of power took place – almost unnoticed. Statista reports that now 52.7 of all internet access is now from the mobile. They further say that by 2017 over 63% of all mobile users worldwide would be accessing the internet from their mobiles. It seems clear that for those businesses that are web based the battleground is shifting from the large screen to the small screen, from the desktop to the phone or the tablet. Many have foreseen this and put in place robust mobile strategies and the rest are probably thinking it’s time to do so. In either case, though, there are some difficult choices to be made along the way. Let me take this chance to try to bring some order to the big decisions you will have to take initially.

Q1. Mobile App or Mobile enabled site?

Do you need to build an app and get it added to the App store of your choice or is it enough to have a site that renders well on the mobile? In Google’s dispensation of the day, a mobile friendly site is anyway mandatory to avoid site ranking penalties. That apart if your site is more likely to be used for information dissemination than for active transactions or actions then a mobile friendly website is likely the way to go. Among the other considerations is how frequently is an individual user likely to have to access the site – if the frequency is generally likely to be high, say public transport route and schedule info of social networking, then an app that is downloaded and installed on the phone may work better.

Q2. If you choose a site then what next?

What’s the design approach to pick? There are 3 that are the most common now Responsive, Adaptive and Blended Web Design. Without getting too much into gory details here are the broad differences:

  1. Responsive is a client-side approach. That’s to say the technology “resides” on the device on which the site is consumed so the experience is very seamless as far as the size and resolution capabilities of the device go. The downside is that traditionally this is tougher, more effort & skill intensive and takes longer to pull off well.
  2. The Adaptive approach, on the other hand, focuses the technology on the server side and adapts to whatever device the “connection” comes from by serving them the version of the content most appropriate to them. While this is usually easier to get off the ground the ongoing coordination and maintenance are a far greater coordination and execution challenge.
  3. A convenient, recent, middle ground has been arrived at with the Blended approach. This has been called Responsive with Service Side components – so the site being served may be the same, but there may be elements like the headers that may be specific to the device. The approach is still settling in but seems promising.

Q3. If App then Native Or Mobile web?

Essentially the thinking was that at one time if you wanted to reach out to as large a cross section of users as possible across devices, operating systems and so on you had to follow the Mobile Web approach utilising the powers of languages like HTML5.

The native approach tied itself much more closely to the operating system (& hence the device family) – if you wanted to target customers who were likely to be iPhone and iPad fans and wanted to use the capabilities of those devices much more fully then you picked iOS as your operating system of choice. The app would be built in accordance with the App Store specs and that was your little (or big) universe. If you thought your customer base was in the wider Samsung, LG, Motorola universe then you picked Android as your platform and that then decided all further choices.

Hybrid platforms like PhoneGap have been around for a while and these seem to be gaining ground. This an argument where the contours are changing even as we speak – maybe in a few months we will talk only about a Hybrid approach driven by people like ReactJS.

The prediction is that by 2018 half of the world’s mobile users will have smartphones. If your business has to be ready for that day you have some choices to make now. You will likely have to start with the answers to these questions.

Key Considerations In Localization Testing

The spectacular growth of the software industry and globalization has made it easy for software vendors to target the non-English speaking geographies, reach out to a wider audience, and spread their footprint. With wider coverage, software product companies are increasing their chances of capturing a larger market share. Such fantastic plans also come with their own set of challenges. Launching the product in multiple geographies means that the product needs to work flawlessly in various regions from the day of the launch.
Localization testing is the process which enables testing of the software product for adoption and use by the users using different languages and from different regions. It also means testing and ensuring that the product is suitable for use in the particular language as per the local culture and look-and-feel.

Key consideration in Localization testing

Only great translation does not make the product local-ready. In addition to perfect translation, it requires attention to the locale details such as date and time formats, numerical, currency, keyboard usage, symbols, icons, suitable graphics, and other such legal requirements. The primary focus of localization testing is on testing of the features, ensuring accurate rendering of the content, making sure that the user interface is fine, cultural awareness, and country-specific changes such as currency, taxation etc. and data storage. Localization testing is always tricky because while the translators have a great mastery over language, they are not necessarily product experts or testing experts. At the same time, the testers are not language experts.

In this post, let us have a look at some of the key considerations in localization testing.

  1. Testing Scope:

    It is important that the complete scope of testing is rightly identified – this includes the main focus, target languages, required platforms, and more importantly, what is not to be tested. The test case development and testing execution depends on the testing scope and, therefore, it is important to have finality on this.

  2. Test Case Development:

    Test case development needs to be done based on the features to be tested as a part of location testing. The test cases should typically include testing of installation/ uninstallation (for desktop-based applications), user interface testing (checking for missing icons, broken text, missing text etc.), and documentation consistency (including online help, manuals, multimedia etc.). The test cases should also include checking of issues which could arise in the product functionality because of localization. For example: the currency symbol placement is different for different currencies, the date and time formats are different, the use of fonts vary as per the language, the use of colors and images change as per the regions, or the keyword usage patterns change as per the regions. The localization testing needs to test for all such aspects so that the users get a feel that the software has been created for them to address their specific needs.

  3. Test Case Management and Defect Reporting:

    To ensure that each team testing for different languages does not reinvent the wheel, it is important the test cases and testing data are managed through a centralized platform.
    While reporting the defects, the testing team should categorize the bugs as localization bugs or global bugs. Localization bugs are the errors which occur during or because of localization process whereas; the global bugs are reproducible on multiple languages and need to be resolved in the core code.

  4. Role of Automation:

    Test automation can be tricky for localization testing and, therefore, it is not always feasible to consider it. However, for legacy software where there is a lot of legacy feature testing required for each release, automation works well from the perspective of cost and time saving. Keyword driven framework, also called as a table-driven framework, is one of the popular methods for automating localization testing. While these frameworks take longer to build, they are automation tool and programming language independent, easy to read and maintain, and scalable.

  5. Mobile App Localization Testing:

    The localization testing of mobile apps is more complex than that of standard web applications. Web applications typically work through standard browsers and use keyboard or mouse as input methods. Whereas; for mobile apps, there are multiple operating systems, screen sizes and different input methods like keyboard and touch. Because of the limited screen size, the mobile apps need to make use of the best font in different languages so that the text does not get truncated or misaligned. Most of the languages require around 30% more space than English, and, therefore, user interface testing is very critical. The rules for spelling, upper and lower case conversions are also different based on locale and culture and the app needs to provide a support for those.
    Because of the vast number of mobile players in the market, the test bed creation itself is a huge task. For example: In Switzerland, primarily French, German and Italian languages are spoken. The three most widely used mobile operating systems are Android, iOS, and blackberry. Swisscom, Sunrise, and Orange are three main service providers. Considering all these parameters, the localization testing needs to be done for 27 combinations for this one country alone!


As we see, while translation is one of the important aspects of localization, localization testing goes much beyond that. It requires a well-thought out strategy and timely and flawless execution to ensure speedy time-to-market at reduced costs and no defects.

Test Automation May Not Be Right For You

The wonderful book, “Implementing Automated Software Testing” authored by Elfriede Dustin, Thom Garrett, and Bernie Gauf, mentions a worldwide survey they conducted on the subject. The online survey got over 700 responses and among a host of interesting results they included in their book was a startling finding. They reported that 72% of the survey respondents believed that automation was useful and even that their management agreed but despite that they either had not implemented it at all or had had only limited success.

The book came out in 2009 and how things seem to have changed since then. Anecdotal evidence, at the very least, would seem to show that most software development efforts today consider automation initiatives somewhere along the way. Much has been written about why one should choose to automate testing – there are great technical as well as commercial arguments. That being said, though, as a testing and test automation company every so often we come across a situation when we recommend to the client that test automation may not be the best bet for them or for that specific development effort. Let’s look at what these situations are where test automation may not be the best fit?

  1. Very less or almost no regression testing: Don’t be surprised – there are a lot of situations where future iterations add on to what has gone before without really touching the old code base in any significant way. For example, remember the waterfall model – it’s still very pertinent in many situations. There are many small, self-contained software projects or projects where the requirements are extremely clearly defined where this model plays well. Here since the development for each phase is completed and then accepted after testing, conventional wisdom is that the regression effort is reduced. In such situations, there is not much value in automating the testing since you are unlikely to gain any significant advantage of time once you factor in the effort of developing the test suite.
  2. Tests that will be run only once: In some ways, this is an extension of the earlier thought. Like everything else, there are choices to be made while automating your testing. If there are tests that are likely to be run on only 1 iteration or very few times in the overall effort then save your time, energy and money – don’t automate them. Instead, prioritize those test cases that are likely to come up time and again in the development cycle.
  3. GUI is not yet frozen or there are frequently functional changes: In many ways, this is the opposite of the first condition. Here the requirements are very dynamic. The fact is that building the test automation suite is an effort and once the suite is built making tweaks to it to allow for changes that have crept into the project scope is neither easy nor advisable. Our advice is always to get the test automation team involved at an early enough stage in the project to be able to design, plan for and build an effective test automation suite – if such planning is not possible then chances are high that the test automation suite will be incomplete or ineffective. In that case, why do it at all?
  4. No skilled resources: We know, this is obvious. That being said you would be surprised how many test teams struggle to provision the right kind of people on test automation teams. Chances are for a test automation effort to be successful, developers will have to join the team – in the end, your automation suite is also a product and hence developing it is like any other product development effort. And it’s not just developing it either, consider that this “product” will need support and maintenance over its lifecycle. If you cannot find the resources with the right skills for this then you may be better off not automating.
  5. Seeking 100% Automation: There has been enough said about this. If you are seeking to automate your testing in quest of the mythical “100% Automated” one-horned beast, then just don’t. It’s not possible, not desirable and just plain wrong to start with inflated expectations. Set out on the automation path with tempered expectations or not at all.

In closing let us just say there is a time and a place for automated testing – just not every time and place. Brad Siims, co-founder of SandVine, explains the value of automated testing elegantly, “With manual testing you can’t reproduce problems as readily and the breadth of test-case coverage isn’t as high. With more automation, you can focus your QA team on making a difference, versus manual re-cabling to run nightly tests.” Do you agree?