DevOps -ThinkSys Inc

Application Development with Microservices in the DevOps Age

Does anyone even remember when companies developed an entire product, tested it, fixed it, and then shipped it? The entire process would take months, even years, before a functioning product made it to the customer. Before the product hit the market, neither did the potential customers know what it held for them and neither did the product owners know if it would hit or miss the mark.

Today, product users expect to be a part of the development process. They want to contribute their insights to develop a product that matches their ongoing needs. The need is for continuous innovation and improvements. The need is for DevOps!

DevOps combines technology and cultural philosophies to deliver products and services quickly. It is a continuous process of developing, testing, deploying, failing, and fixing applications to achieve market-fit. Jez Humble, one of the leading voices of DevOps sums it up “DevOps is not a goal, but a never-ending process of continual improvement.”

Today, DevOps is not just for a handful of large enterprises. According to Statista, the number of companies adopting DevOps went up by 17% in 2018.

A quick look at what has made DevOps popular?

Apart from the continuous innovations and improvements, DevOps also helps in:

Improving customer satisfaction: With a DevOps mindset, companies use advanced methods to identify issues and fix them real-time before the customer is impacted. There is also scope to improve the product on-the-go driven by frequent suggestions and feedback from customers. Continuous improvement in quality leads to customer delight. Take Rabobank of Netherlands, for example. This large financial institution has over 60,000 employees and hundreds of customer-facing applications. As the deployments were manual, the failure rate was over 20%, and they received many complaints about delays. When they moved to DevOps, they were able to deploy applications 30x more frequently with a lead time that was 8,000 times faster than their peers.
Change in organizational culture: DevOps has played a significant role in breaking silos and boosting the collaborative culture in companies. In an agile environment, working in silos can slow down the process of developing, testing, and releasing the product. A DevOps team will be able to collaborate better and ramp up the process of developing, testing, and troubleshooting the product.
A decrease in failure rates: According to the State of DevOps report, high-performing DevOps organizations have seen a reduction of failure rates of 3x, thanks to their ability to find and fix errors early in the cycle.
Higher productivity: DevOps organizations can deploy products 200x more frequently than a non-DevOps organization, leading to happier and highly motivated teams. Take Microsoft’s Bing, for example. It has moved developers to a DevOps environment with the idea of continuous delivery and innovation deeply ingrained within their processes. The result? Bing deploys thousands of services 20 times a week and pushes out 4000 individual changes every week. The continuous effort by the team to deliver has made Bing the second largest search engine in the world.

While adopting a DevOps culture is essential for a company to thrive, it is also crucial that they have the right architecture and systems in place to complement their principle of continuous delivery and innovation. That’s where microservices is now playing a massive role.

Micro-services and Their Role in DevOps Organization:

For a long time, companies relied on a monolithic architecture to build their application. As monolithic applications are built as a single unit, even a small change in a single element made it necessary to build a completely new version of the application.

With more and more companies moving towards DevOps, such a monolithic architecture makes it difficult to implement changes rapidly. The need for greater agility gave rise to a new type of architecture -enter microservices.

With Microservices, an application is built on small, independent components that are independently deployable. Although independent, these components communicate with each other via RESTful APIs. So, even if a single piece of code has to be changed in a single element, the developer does not have to build a new version of the whole product. They can simply make the changes to the individual components without affecting the entire application, making the deployment efficient and faster.

For companies that have adopted the DevOps culture, developing applications with microservices has several benefits that include:

Easy rectification of errors: When a component fails the test or requires changes, it is easy to isolate and fix. This makes it easier for companies to fix errors quickly without affecting the users of other services.
Better collaboration: Unlike a monolithic architecture where the different teams focus only on specific functions such as UX, UI, server, etc, a microservices architecture encourages a cross-functional way of working.
Decentralized governance: Monolithic architecture uses a centralized database, while microservices use a decentralized method of governance, wherein each service manages its database. This makes it easier for developers to produce tools that also can be used by others to solve specific issues.

A key trend accelerating the adoption of Microservices in such scenarios is Containerization. Containerization allows code for specific elements to be carved out, packaged with all the relevant dependencies, and then run on any infrastructure. These applications can be deployed faster and can be made secure. The applications are extremely portable and adaptable to run on different environments.

Companies like Amazon and Netflix have shifted to microservices to scale their business and improve customer satisfaction.

Product companies aiming to become customer-centric and delight with continuous improvement in the product may find it essential to adopt a DevOps mindset married to a transition to the microservices architecture.

Of course, it will take some time to transition product development. Teething problems are bound to arise, including duplication of efforts due to the distributed deployment system. However, given the larger picture and the potential benefits, it’s a wise move for product companies to make.

September 18, 2019 Rajiv Jain Development, DevOps, Microservices No comments yet

How Offshore Development has Changed With DevOps?

Offshore software development has never been easy. Neither has DevOps. Although both offer a distinct set of advantages to organizations, trying to do them together could be challenging. In addition to creating a culture of collaboration, new tools have to be adopted. Yet, many large global organizations have successfully built DevOps capabilities across time zones, while meeting requirements 24×7 – within time and budget.

Here’s how Offshore Development has Changed with DevOps:

The Improvement in Product Quality: Quality management has always been a basic requirement of software development, and also a popular way to control development costs. But with offshore development, quality management gained a reputation for being rigid and imbalanced. Offshore teams had a tough time balancing quality and costs. The perception grew that they could only focus on one aspect while overlooking the other. However, DevOps brings in a way for offshore development teams to drive quality and costs simultaneously. Since there is more collaboration between teams, bugs are identified quickly – which improves quality, and there is less rework – which reduces the associated costs.
The Stress on Culture: Offshore development teams have often focused on the tools and technologies needed to drive outcomes. However, with the advent of DevOps, there is a ton of business culture aspects to consider. When DevOps comes into the picture, it’s not just about tooling; teams have to work together and collaborate to drive the intended DevOps outcomes. Rather than looking at culture as a nice-to-have feature, offshore development teams have started to look at it as a core competency that lays the foundation of an efficient software development practice.
Accelerated time-to-Market: Since the dawn of offshore development, teams have been following the sun; once early analysis and design are complete, documentation is sent to remote developers to start coding and testing. However, what DevOps does, is turn all of this on its head; by seeking greater collaboration between teams, it helps them release software in bite-sized sprints – so teams can get more frequent visibility and feedback. Such an approach builds faster feedback loops, accelerates the velocity at which a company can test hypotheses about what the client wants – without wasted time and effort – and brings products to market sooner.
The Elimination of Hand-offs: Offshore development has also always been about hand-offs. When one person (or team) is done with a piece of work, a key milestone is achieved, and he/she then notifies the other to start working. However, what DevOps does is just the exact opposite. It enables different teams to work on aspects of software development in tandem, while greatly reducing the number of handoffs or delays. Teams do not have to waste time waiting for a “go-ahead” to start working; instead, they drive continuous collaboration through the entire development life cycle, keep track of tasks across coding, unit testing, build scripts, configuration scripts and avoid passing work back and forth.
The Growth of Analytical Dashboards: For offshore teams having a tough time getting visibility into project status, DevOps drives the use of analytical dashboards. These dashboards often serve the purpose of providing a single source of truth across the complete organization, while giving real-time updates on project status, issues, challenges, and improvement opportunities. Teams that leverage these tools find themselves resolving issues faster while making the entire process of offshore development far more effective.
Handling out-of-Scope Requests: Offshore teams have always found it difficult to handle out-of-scope requests and cater to emergency patch-up works which come out of their schedule. This is mainly due to the differences in time zone. However, with DevOps, the project’s scope is clearly defined through several iterations of communication between the internal team and the offshore team. Any out-of-scope request can be accommodated, based on the availability of resources, as can urgent jobs which need immediate attention.

Improve Software Development Outcomes:

When the world embraced the offshore development model, the productivity gains and cost savings stimulated technological innovation for years to come. While offshoring helped businesses achieve their market and customer goals – quickly and more efficiently, it also paved the way for the adoption of methodologies and approaches to produce software more efficiently and effectively.

DevOps is one such transformation, that is helping offshore teams break departmental siloes, and drive a cultural shift towards efficient software delivery. The changes range from dramatically improving software quality to accelerating time-to-market, eliminating wasteful hand-offs, to offering real-time visibility into product status while seamlessly handling out-of-scope requests. The impact of DevOps on offshoring has been phenomenal, and the approach will continue to boost offshore development outcomes for years to come.

September 4, 2019 ThinkSys DevOps No comments yet

The 5 Point Guide For A Successful DevOps Strategy

As the requirement for high-quality software in short time frames and restricted budgets increases, developers are looking for approaches that make building software a lot faster and more efficient. DevOps greatly helps in improving the software product delivery process; by bridging the gap between the development and operations teams, DevOps facilitates greater communication and collaboration, and improves service delivery, while reducing errors and improving quality. According to the State of Agile report, 58% of organizations embrace DevOps to accelerate delivery speed.

Tools for a successful DevOps Strategy

DevOps creates a stable operating environment and enables rapid software delivery through quick development cycles – all while optimizing resources and costs. However, before you embark on the DevOps journey, it is important to understand that since DevOps integrates people, processes, and tools together, more than tools and technology, it requires a focus on people and organizational change. Begin by driving an enterprise-wide movement – right from the top-level management down to the entry-level staff – and ensure everyone is informed of the value DevOps brings to the organization before integrating them together into cross-functional teams.

Next, selecting the right tools is critical to the success of your DevOps strategy; make sure the tools you select work with the cloud, support network, and IT resources and comply with the necessary security and governance requirements. Here’s your 5-point guide for developing a successful DevOps strategy and the tools you would need to drive sufficient value:

Understand your Requirements: Although this would seem a logical first step, many organizations often make the DevOps plunge in haste, without sufficient planning. Start by understanding the solution patterns of the applications you plan to build. Consider all important aspects of software development including security, performance, testing, and monitoring — basically all of the core details. Use tools like Pencil, a robust prototyping platform, to gather requirements and create mockups. With hundreds of built-in shape collections, you can simplify drawing operations and enable easy GUI prototyping.
Define your DevOps Process: Implementing a DevOps strategy might be the ideal thing to do, but understanding what processes you want to employ and what end result you are looking to achieve is equally important. Since DevOps processes differ from organization to organization, it is important to understand which traditional approaches to development and operations to let go of as you move to DevOps. Tools like GitHub can enable you to improve development efficiency and enjoy flexible deployment options, centralized permissions, innumerable integrations and more. GitHub allows you to host and review code, manage projects, and build quality software – moving ideas forward and learning all along the way.
Fuel Collaboration: Collaboration is a key element of any DevOps strategy. It is only through continuous collaboration that you can develop and review code and stay abreast with all the happenings. With frequent and efficient collaboration, you can efficiently share workloads, enable frequent reviews, be informed of every update, resolve simple conflicts with ease, and improve the quality of your code. Collaboration tools like Jira and Asana enable you to plan and manage tasks with your team across the software development lifecycle. While Jira allows team members to effectively plan and distribute tasks, prioritize and discuss team’s work, and build and release great software together, Asana allows project leaders to assign responsibilities throughout the project; you can prioritize tasks, assign timelines, view individual dashboards and communicate on project goals.
Enable Automated Testing: When developing a DevOps strategy, it is important to enable automated testing. Automated test scripts speed up the process of testing, and also improve the quality of your software by testing it thoroughly at each stage. By leveraging real-world data, they reflect production-level loads and identify issues in time. DevOps-friendly tools like Selenium are ideal for enabling automated testing. Since Selenium supports multiple operating systems and browsers, you can write test scripts in various languages including Java, Python, Ruby and more and can also extend test capability using additional test libraries.
Continuously Monitor Performance: To get the most out of your DevOps strategy, measuring and monitoring performance is key. Given the fact that there will be hundreds of services and processes running in your DevOps environment, all of which cannot be monitored, the identification of the key metrics you want to track is vital. Tools like Jenkins can be used to continuously monitor your development cycles, deployment accuracy, system vulnerabilities, server health, and application performance. By quickly identifying problems, it enables you to integrate project changes more easily and deliver a functional product more quickly.

Improve Service Delivery

Implementing a DevOps strategy is not just about building high-quality software faster; it’s about driving a cultural shift across the organization to improve development processes and make it more efficient. Making the most of a switch to DevOps requires you to start with a new outlook, along with the use of new tools and new processes. By using the right tools at every stage, you can accelerate the product development process, meet time-to-market deadlines, and begin your journey towards improved service delivery and optimized costs.

December 24, 2018 ThinkSys DevOps 2 comments

Watch Out for these DevOps Mistakes

The past few years have witnessed the meteoric rise of DevOps in the software development landscape. The conversation is now shifting from “What is DevOps” to “How can I adopt DevOps”. That said, the Puppet’s State of DevOps Report stated that high performing DevOps teams could deploy code 100 times faster, fail three times less and recover 24 times faster than the low performing teams. This suggests that DevOps, like with every other change in the organization, can be beneficial only when done right. In the haste to jump on the DevOps bandwagon, organizations can forget that DevOps is not merely a practice but is a culture change – a culture that breeds success based on collaboration. While DevOps is about collaboration between teams, continuous development, testing and deployment, key mistakes can lead to DevOps failure. Here’s a look at some common DevOps mistakes and how to avoid them.

Oversimplification:
DevOps is a complex methodology. In order to implement DevOps, organizations often go on a DevOps Engineer hiring spree or create a new, often isolated, DevOps department to manage the DevOps framework and strategy. This unnecessarily adds new processes, often lengthy and complicated. Instead of creating a separate DevOps department, organizations must focus on optimizing their processes to create operational products leveraging the right set of resources. For successful DevOps implementation, organizations must manage the DevOps frameworks, leveraging operational experts and other resources that will manage DevOps related tasks such as resource management, budgeting, goals and progress tracking.
DevOps demands a cultural overhaul and organizations should consider a phased and measured transition to DevOps implementation by training and educating employees on these new processes and have the right frameworks in place to enable careful collaboration.
Rigid DevOps processes:
While compliance with core DevOps tenets is essential for DevOps success, organizations have to proactively make intelligent adjustments in response to enterprise demands. Organizations thus have to ensure that while the main DevOps pillars remain stable during DevOps implementation, they make the internal adjustments that are needed in internal benchmarking of the expected outcomes. Instrumenting codebases in a granular manner and making them more partitioned gives more flexibility and gives DevOps teams the power to backtrack and identify the root cause of diversion in the event of failed outcomes. However, all adjustments have to be made while remaining within the boundaries defined by DevOps.
Not using purposeful automation:
DevOps needs organizations to adopt purposeful automation – automation that is not done in silos like change management or incident management. For DevOps, you must adopt automation across the complete development lifecycle. This includes continuous delivery, continuous integration, and deployment for velocity and quality outcomes. Purposeful end-to-end automation is essential for DevOps success. Hence organizations must look at complete automation of the CI and CD pipeline. At the same time, organizations need to keep their eyes open to identify opportunities for automation across processes and functions. This helps to reduce the need for manual handoffs for difficult integrations that need additional management and also in multiple format deployments.
Favoring feature-based development over trunk-based development:
Both feature-based development and trunk-based development are collaborative workflows. However, feature-based development, a development style that provides individual features their isolated sandboxes, adds to DevOps complexity. As DevOps automates many aspects of the code between development and production environments, keeping different conceptual flavors around the codebase makes DevOps more complex. Trunk-based development, on the other hand, allows developers to work in a coherent and single version of the codebase and alleviates this problem by giving developers the capability to manage features through selective deployments instead of through version control.
Poor test environments:
For DevOps success, organizations have to keep the test and production environments separate from one another. However, test environments must resemble the production infrastructure as close as possible. DevOps means that testing starts early in the development process. This means ensuring that test environments are set up in different hosting and provider accounts than what you use in production. Testing teams also have to simulate the production environment as closely as possible as applications perform differently on local machines and during production.
Incorrect architecture evaluation:
DevOps needs the right architectural support. The idea of DevOps is to reduce the time spent on deploying applications. Even when automated, if deployment takes longer than usual there is no value in the automation. Thus, DevOps teams have to pay close attention to the architecture. Ensure that the architecture is loosely coupled to give developers the freedom and flexibility to deploy parts of the system independently so that the system does not break.
Incorrect incident management:
Even in the event of an imperfect process, DevOps teams must have robust incident management processes in place. Incident management has to be a proactive and ongoing process. This means that having a documented incident management process is imperative to define incident responses. For example, a total downtime event will have a different response workflow in comparison to a minor latency blip. The failure to do so can lead to missed timelines and avoidable project delays.
Incorrect metrics to measure project success:
DevOps brings the promise of faster delivery. However, if that acceleration comes at the cost of quality then the DevOps program is a failure. Organizations looking at deploying DevOps thus must use the right metrics to understand progress and project success. Therefore, it is essential to consider metrics that align velocity with throughput success. Focusing on the right metrics is also important to drive intelligent automation decisions.

To drive, develop, and sustain DevOps success, organizations must focus on not just driving collaboration across teams but also on shifting the teams’ mindset culturally. With a learning mindset, failure is leveraged as an opportunity to learn and further evolve the processes to ensure DevOps success.

September 4, 2018 ThinkSys DevOps No comments yet

Understanding The Terminology – CI and CD in DevOps

The path to building cutting-edge software solutions is often paved with several obstacles. Disjointed functioning of various development teams often results in long release cycles. This not only results in a poor quality product but also adds to the overall cost of development. For organizations looking to set themselves apart from the competition, it has become essential to embrace the world of DevOps and enable frequent delivery of good-quality software.

The Growth of DevOps

Conventional software development and delivery methods are rapidly becoming obsolete. Since the software development process is a long and complex one, the process requires teams to collaborate and innovate with each passing day. Models evolved -first it was Waterfall, then Agile, and now it’s DevOps – in order to meet the dynamic demands of the industry, and growing expectations of the tech-savvy user, the software development landscape is undergoing constant change. Today, DevOps is being seen as the most efficient method for software development. According to the recently released 2017 State of DevOps Report, high performing organizations that effectively utilize DevOps principles achieve 46x more frequent software deployments than their competitors, 96x faster recovery from failures, and 440x faster lead time for changes. There seems little room for doubt any longer about the impact of DevOps.

DevOps aims at integrating the development and operations teams to enable rapid software delivery. By fuelling better communications and collaboration, it helps to shorten development cycles, increase deployment frequency, and meet business needs in the best possible manner. Using DevOps, software organizations can reduce development complexity, detect and resolve issues faster, and continuously deliver high-quality, innovative software. The two pillars of successful DevOps practice are continuous integration and continuous delivery. So, what are these terms? What do they mean? And how do they help in meeting the growing demands of the software product industry? Let’s find out!

Continuous Integration

Definition: Continuous Integration (CI) aims at integrating the work products of individual developers into a central repository early and frequently. When done several times a day, CI ensures early detection of integration bugs. This, in turn, results in better collaboration between teams, and eventually a better-quality product.

Goal: The goal of CI is to make the process of integration a simple, easily-repeatable, and everyday development task to reduce overall build costs and reveal defects early in the cycle. It gets developers to carry out integration sooner and more frequently, rather than at one shot in the end. Since in practice, a developer will often discover integration challenges between new and existing code only at the time of integration, if done early and often, conflicts will be easier to identify and less costly to solve.

Process: With CI, developers frequently integrate their code into a common repository. Rather than building features in isolation and submitting each of them at the end of the cycle, they continuously build software work products several times on any given day. Every time the code is inputted, the system starts the compilation process, runs unit tests and other quality-related checks as needed.

Dependencies: CI relies heavily on test suites and an automated test execution. When done correctly, it enables developers to perform frequent and iterative builds, and deal with bugs early in the lifecycle.

Continuous Delivery

Definition: Continuous Delivery (CD) aims to automate the software delivery process to enable easy and assured deployments into production —at any time. By using an automatic or manual trigger, CD ensures the frequent release of bug-free software into the production environment and hence into the hands of the customers.

Goal: The main goal of CD is to produce software in short cycles so that new features and changes can be quickly, safely, and reliably released at any time. Since CD involves automating each of the steps for build delivery, it minimizes the friction points that are inherent in the deployment or release processes and ensures safe code release can be done at any moment.

Process: CD executes a progressive set of test suites against every build and alerts the development team in case of a failure, which then rectifies it. In situations where there are no issues, CD conducts tests in a sequential manner. The end result is a build that is deployable and verifiable in an actual production environment.

Dependencies: Since CD aims at building, testing, and releasing software quickly and frequently, it depends on an automated system that helps the development team to automate the testing and deployment processes. This is to ensure the code is always in a deployable state.

CI/CD for Continued Success

Software development involves a high degree of complexity that requires teams to embrace new and modern development methodologies in order to meet the needs of business and end-users alike. DevOps focuses on the continuous delivery of software through the adoption of agile, lean practices. The pillars of DevOps, CI, and CD, improve collaboration between operations and development teams and enable the delivery of high-quality software for continued success. RightScale estimates that over 84% of organizations have adopted some aspect of DevOps principles -it’s time you do too. As DevOps pundit Jez Humble rightly says, “DevOps is not a goal, but a never-ending process of continual improvement”.

June 1, 2018 ThinkSys DevOps No comments yet

Agile? DevOps? What’s The Difference And Do You Have To Choose Between Them?

Any roles involved in a project that do not directly contribute toward the goal of putting valuable software in the hands of users as quickly as possible should be carefully considered.” – Stein Inge Morisbak

Does anyone remember the days when the Waterfall model was still around and widely adopted by the enterprises? Over the years most developers have stories of how they realized that it wasn’t giving the best results, that it was slow and inflexible as it followed a sequential process. Fast forward a few years and the principles of Kanban and scrum methodology organically evolved and gave rise to the Agile approach to software development –and we were all on board in a flash. Suddenly, software development teams were able to shift from longer development cycles to shorter sprints, fast releases, and multiple iterations.

But the evolution was not over, as we now know. As Agile shone a spotlight on releasing fast and often, enterprises started loving the opportunity to be more flexible and to speedily incorporate the feedback of their customers. However, this also revealed some drawbacks with the Agile approach. Though the development cycle was faster, there was a lack of collaboration between the developers and the operations team and this was adversely impacting the release and the customer experience.

This gave rise to the new methodology of DevOps which focused on better communication among development, testing, business, and the operations team to provide faster and more efficient development.

So now software development organizations face a choice –should they be Agile? Or do DevOps? Or perhaps somehow both? Let’s look at both approaches more closely, starting with filling in the essential backstory.

The Agile Approach Explained

Software Development approaches like the Waterfall model took several months for completion, where the customers would not be able to see the product until the end of the development cycle. On the other hand, the Agile approach is broken down into sprints or iterations which are shorter in duration during which certain predetermined features can be developed and delivered. There are multiple iterations and after every iteration, the software team can deliver a working product. The features and enhancements are planned and delivered for every succeeding iteration after discussions (negotiations?) between the business and the development teams.
In other words, Agile is focused on iterative development, where the requirements and solutions are developed because of collaboration between cross-functional and self-organizing software teams.

What is DevOps?

This is the age of Cloud and SaaS products. That being the case, DevOps can be defined as a set of practices enabling automation of processes between the software development and the IT teams for building, testing, and deploying the software in a faster and more efficient manner. DevOps is based on cross-functional collaboration and involves automation and monitoring right from the integration, testing, releasing, and deployment along with the management of infrastructure.

In short, DevOps helps in improving collaboration and productivity by integrating the developers and the operations team. Typically, DevOps calls for an integrated team comprising developers, system administrators, and testers. Often, Testers turned into DevOps engineers are assigned the end-to-end responsibility of managing the application software. This may involve everything from gathering requirements to development, deployment, and gathering user feedback to implementing the final changes.

How do they compare (or contrast)?

Creating and deployment of software:
Agile is purely a software development process. That means, the development of software is an inherent part of the agile methodology. Whereas Devops can deploy software which may have being developed using other methodologies, based on either Agile or non-agile approaches.
Planning and documentation:
The Agile method is based on developing new versions and updates during regular sprints (a time frame decided by the team members). Besides, daily informal meetings are key to the Agile approach, where team members are encouraged to share progress, set goals, and ask for assistance if required. To that extent, the emphasis on documentation is less.

On the other hand, DevOps teams may not have daily or regular meetings but plenty of documentation is required for proper communication across the teams for effective deployment of the software.

Scheduling activities and team size:
Agile is based on working in short and pre-agreed sprints. Traditionally sprints can last for a week to 1 month or so at the extreme. The team sizes are also relatively smaller as they can work faster with fewer individuals working on the effort.
DevOps can comprise of several teams using different models such as Kanban, Waterfall model, or scrum where all of them are required to come together for discussing regarding software deployment. These teams could be larger and are by design much more cross-functional.
Speed and risk:
Agile releases, while frequent, are significantly less than what DevOps teams aim for. There are DevOps products out there that release versions with new features multiple times in an HOUR! The application framework and structure in Agile approach needs to be solid to incorporate the rapid changes. As the iterative process involves regular changes to the architecture, it’s necessary to be aware of every change related to the risks to ensure quick and speedy delivery. This is true of DevOps also, but the risk of breaking previous iterations is far greater in DevOps as the releases are much more frequent and follow much faster on the heels of one another than in the Agile approach.

Conclusion

DevOps is a reimagining of the way in which the software needs to be configured and deployed. It adds a new dimension to the sharp end of the value chain of software development i.e the delivery to the customers. There is some talk about that that DevOps will replace Agile, but our view is that DevOps complements Agile by streamlining deployment to enable faster, more effective, and super-efficient delivery to the end users. That’s a worthy goal –so why choose between the two!

February 28, 2018 ThinkSys DevOps, Quality Assurance Testing No comments yet

Achieving Assured Quality in DevOps With Continuous Testing

DevOps has finally ushered in the era of greater collaboration between teams. Organizations today realize that they can no longer work in siloes. To achieve the required speed of delivery, all invested in the software delivery process, the developers, the operations, business teams, and the QA and testing teams have to function as one consolidated and harmonious unit. DevOps provides organizations this new IT model and enables teams to become cross-functional and innovation focused. The conviction that DevOps helps organizations respond and adapt to market changes faster, shrinks product delivery timelines, and helps to deliver high-quality software products is reflected in the DevOps adoption figures. According to the Puppet State of DevOps Report, in 2016, 76% of the survey respondents adopted DevOps practices, up from 66% in 2015.

One of the hallmarks of the DevOps methodology is an increased emphasis on testing. The approach has shifted from the traditional method of adding incremental tests for each functionality at the end of each development cycle. The accepted way now is to take a top-down approach to mitigate both functional and non-functional requirements. To achieve this DevOps demands a greater testing emphasis on test coverage and automation. Testing in DevOps also has to start early in the development process to enable the DevOps methodology of Continuous Integration and Continuous Delivery.

The Role of Testing in Continuous Delivery and Continuous Integration:

In order to deliver on the quality needs, DevOps demands that testing is integrated into the software development and delivery process and acts as a key driver of DevOps initiatives. Here, individual developers work to create code for features or for performance improvements and then have to integrate it with the unchanged team code. A unit test has to follow this exercise to ensure that the team code is functioning as desired. Once this process is complete, this consolidated code is delivered to the common integration area where all the working code components are assembled for Continuous Integration. Continuous Integration ensures that the code in production is well integrated at all levels and is functioning without error and is delivering on the desired functionalities.

Once this stage is complete, the code is delivered to the QA team along with the complete test data to start the Continuous Delivery stage. Here the QA runs its own suites of performance and functional tests on the complete application in its own production-like environment. DevOps demands that Continuous Integration should lead to Continuous Delivery in a steady and seamless manner so that the final code is always ready for testing. The need is to ensure that the application reaches the right environment continuously and can be tested continuously.

Using the staging environment, the Operations teams too have to run their own series of tests such as system stability tests, acceptance tests, and smoke tests, before the application is delivered to the production environment. All test data and scripts for previously conducted application and performance tests have to be provided to the operations teams so that ops can run its own tests comprehensively and conveniently. Only when this process is complete, the application is delivered to production. In Production, the operations team has to monitor that the application performance is optimal and the environment is stable by employing tools that enable end-to-end Continuous Monitoring.

If we look at the DevOps process closely we can see that while the aim is faster code delivery, the focus is even more on developing error free, ready for integration and delivery code by ensuring that the code is presented in the right state and to the right environment every time. DevOps identifies that the only way to achieve this is by having a laser sharp focus on testing along with making it an integrated part of the development methodology. In a DevOps environment, testing early, fast and often becomes the enabler of fast releases. This means that any failure in the development process is identified immediately and prompt corrective action can be taken by the invested stakeholders. Teams can fail fast and also recover quickly – and that is how to ensure Quality in DevOps.

August 22, 2017 ThinkSys DevOps, Quality Assurance Testing 2 comments

The Big Challenges in Automating Your Testing for DevOps

To stay ahead of the market organizations have to deliver a high-quality product in the least possible time. For this, organizations have had to fundamentally change their development methodologies as well as their testing practices. These shifts have prompted all the stakeholders of product development to work more closely and in tandem with one another. DevOps is one such development methodology that takes a more holistic approach to software development by bringing software developers, testers, and operations together to improve collaboration and to deliver a quality product at light speed.

Clearly, the role of QA and testing have been redefined in the DevOps environment. DevOps is heavily focused on the ‘fail fast, fail often’ mandate propelled by the ‘test first’ concept. Testing, thus, becomes continuous and exhaustive and hence demands greater levels of automation. But just how easy is it to automate testing in DevOps?

DevOps makes testers an important part of the development team to develop new features, implement changes and enhancements, along with testing the changes made in the production software. While on the outset this arrangement looks fairly simple to achieve, there can be some challenges that first need to be addressed to automate testing in a DevOps environment. In fact, Quali’s 2016 survey on the challenges of implementing DevOps states that 13% of those surveyed feel that implementing test automation poses a barrier to successful DevOps implementation. In this blog, we take a look at some changes that create challenges in automating testing for DevOps.

The New-age Testing Team
The DevOps environment needs testing teams to change pragmatically to accommodate accelerated testing – not always easy to achieve. These teams, instead of being in the back end now have to co-exist with the other development stakeholders in DevOps. Along with being focused on the end-user, testing teams in DevOps also have to be aware of the business goals and objectives and have to have the ability to understand how each requirement impacts another and be in a position to identify and iterate cross-project dependencies. So along with being able to understand user stories and define acceptance criteria’s, they also need to have better communication, and analytical and collaboration skills. This allows them to clarify intent and also provide sound advice on taking calculated risks.
The Process Change
DevOps demands greater integration of development and testing teams. This also means that the testing and QA team has to work closely with product owners and business experts and also understand the working of the business systems being tested. Even testing teams need to develop a Product Lifecycle Management mindset by first unlearning the standard SDLC process. DevOps testing teams also need to assign an architect to select the right testing tools, determine best practices for continuous integration and integrate the test automation suite with the build deployment tool for centralized execution and reporting. There, thus, has to be a ‘one team’ mentality across the invested teams – a significant change in the “way we work”.
The Pace of Change
DevOps also focuses heavily on the speed of development and deployment. This places a lot of emphasis on increasing test coverage, iterating detailed traceability requirements and ensuring that the team does not miss testing of critical functions in the light of rapidly changing requirements. Test plans in DevOps thus need to be more fluid and have to be carefully prioritized to adapt to these uncertainties that arise from changing requirements and tight timelines. Test Automation also takes time to develop. At the blistering pace set by the DevOps team how is the automation to be completed?
Unified Reporting and Collaboration
Test automation in DevOps demands consolidated timely reports to provide actionable insights to foster collaboration in cross-functional teams. Testing teams also need to ensure that they introduce intelligence into the existing test automation set up. This is to proactively address scalability challenges that may slow down testing speed. Analytics and intelligence can also play a key role in implementing intelligent regression models and establishing automation priorities. This is essential to test what is needed, and only what is needed, in the interest of time. Ensuring easy maintainability of the automation architecture has always been a priority but it may now become necessary to have a central repository of code-to-test cases for easier test case traceability. Prevailing test practices are not necessarily tuned to this level of reporting and analysis and this is a significant challenge to overcome.
Testing Tools Selection and Management.
Traditional testing tools may be misfits in a DevOps environment. Some testing tools can be used only once the software is built, thus failing the whole purpose of DevOps. Some testing tools can only be employed once the system has evolved and is more settled. DevOps testing teams thus, need to use those tools that help them explore the software still being built. They must test in a manner that is unscripted and fluid.

The test automation tools DevOps needs can link user stories to test cases, provide a holistic requirement view, keep a record of testing results and test data, have REST API’s, help manage test cycles and create and execute test cases real-time, and provide detailed reporting and analytics.

Testing teams in a DevOps environment are critically important. They need to work with an enhanced degree of speed and transparency and they must root out all inefficiencies that impede the automation process. Automation is key to their success but as we have outlined, there are some significant challenges to overcome in getting Automation right in DevOps. Stay tuned for future posts where we reveal just how these challenges can be addressed in the DevOps environment.

July 20, 2017 ThinkSys DevOps 5 comments

What should startups look for while choosing their technology stack?

Look at any business today and you will find a compelling dependence on technology. Today, technology also forms the core of any successful startup. When it comes to startups, it has sometimes been seen that while entrepreneurs focus on building the front end of their business, the job of choosing the right product technology stack features low on the priority list…almost as an afterthought.

The right choice of technology stack for product development contributes greatly to the efficiency and smooth running of a start-up. Ensuring that the right technologies are being leveraged ensures that you release on time. At the same time, given the overwhelming number of technology options, this can be a tough decision to make as well.

Many non-technical founders tend to depend on developer opinion when choosing a technology stack. This sometimes can backfire as developers can be biased towards particular technologies depending on their proficiency and comfort level. They also might assess the technology based on its technical merits rather than on the business requirements. Technology options need to be evaluated more objectively and here we take a look at some business considerations that need to be made before choosing a technology stack for building the product that will define your startup.

Usability

One of the primary considerations before making a technology selection is to first identify how and for what the technology will be used. The usage aspect heavily influences a technology decision as a technology that works perfectly for developing an eCommerce website might not necessarily be best suited for an enterprise mobile application. ‘Purpose’, thus, ranks the highest when selecting a technology. The technology stack has to be such that it fulfills the requirement demands and helps in establishing the business.

UI and UX Focus

The consumer of today goes by the principle of ‘don’t make me think’. Having high-end user experiences thus becomes of paramount importance. Simple, intuitive and intelligent user interfaces that facilitate a seamless user experience are a must. Technology choices have to be made such that they act as enablers of usability and allow the application users to be consistently productive in their work.

Talent Availability
You might want to choose the next hot technology on the block but if you cannot find the talent to work with this technology then all you’ll be stuck! This, for startups, can be a big financial drain. For example, finding developer talent to create a chat server with Erlang may prove harder than finding developers proficient in Java, Ruby or Python. Leveraging mainstream technologies that are open source and opting for a development methodology such as Agile or DevOps with a heavy testing focus is a good idea. This will give your startup the advantage of getting to market faster, rapidly shipping code and getting the desired features to the users at the earliest.

Technology Maturity
Startups need to look at the maturity of the technology before selecting a particular technology to ensure that the technology is built to last. Programming languages such as Ruby are relatively recent but have gone through several iterations and has now achieved language maturity. Mature technologies also give startups the benefit of a mature tools ecosystem that allows bug tracking, code analysis, facilitate continuous development and continuous integration etc. all of which make development faster and easier.

When looking at technology maturity, it is also essential to assess how easily you can build and share solutions built on the technology stack. Leveraging a technology that has great third party packages or ready to use, community generated code or a complete suite of easy to use and build solutions, or automated testing capabilities helps in not only attracting more developers but also helps in making development quicker and convenient.

Technology Dependencies

All it takes is one weak link to bring down a large fortress. Take the case of the Heartbleed bug which was caused because of the OpenSSL component in the library. When this bug was introduced, every technology that leveraged this widely used cryptographic library was affected. This just goes to show that when making a technology choice you have to ensure that the primary and secondary technologies are robust and secure and that their dependencies can be managed easily. So if for example, you are looking at Ruby on Rails, you should know that Rails (the Framework) is the secondary technology since it relies on Ruby (the primary technology) and that Ruby will also have its own set of dependencies. To leverage the two well you need to know the risks of both.

Scalability and Accessibility
Technology choices should support the demands of a growing business. The technology that a startup chooses thus has to allow for adding more users over time, add new functionalities or services, allow iterations, and enable integration with higher technologies. These days, looking at technologies that support a Service Oriented Architecture or SOA, gives more scope for extensibility to a startup by accommodating changes and iterations according to the needs or the market or product evolution demands.
Along with this, startups also have to ensure that the technology choice that they make allows for greater accessibility and security to allow business users access the product or service anytime anywhere.

Community Support
Community support might not rank highly in the startup technology choices priority list, but it probably should. Why? Simply because, as a startup, you can do with all the help that you can get. Along with this, a strong developer network and back-end support emerge as crucial resources when you are exploring the technology to either solve a problem or add new functionalities.

When evaluating technology options, startups also need to consider the maintenance needs of the technology, its compatibility capabilities, and security levels. Choosing the right technology is an imperative for the success of any startup. Startup entrepreneurs thus need to tick the right boxes when it comes to making the technology choice if they want to enable their startup to maximize their chances of success.

May 31, 2017 ThinkSys DevOps, Startups No comments yet

Testing in the DevOps World-Test Automation the Sequel

“The most powerful tool we have as developers is automation.” – Scott Hanselman

In a previous blog about some stats that told the developing software testing story, we had identified the impact on testing strategies due to the growing adoption of DevOps as a key trend. Our CEO has also written in greater detail on how testing is changing due to DevOps. Looking back though it’s clear this story is still developing and that something more remains to be said. In other words, it deserves a sequel and the central role in this continuing story is reserved for Test Automation.

You may ask, why does Test Automation deserve this star billing? Well, consider the DevOps way for a bit. This is a world with several, almost continuous iterative releases, each within days, even minutes, of each other, all being pushed out to the final production environment into the demanding hands of paying customers. So many releases, so little testing time and so much pressure to deliver quality – has there ever been a more, theoretically, perfect case for automated testing? Let’s hope that puts the “Why” question to bed – now let’s move on to the “How” and “What”.

First, a look at the “How”. As was already apparent in Agile, with so many iterative releases following so close on the heels of each other, it is absolutely impossible to build your automation in parallel with the product-under-test. Thus, with DevOps, it becomes critically important to involve the test automation team at an early enough stage of the product planning to be able to anticipate, as much as possible, the direction the product is likely to take and automate early. This is also the time to plan for the automation carefully. Factors to consider include what conditions are most likely to remain reasonably unchanged and which are likely to undergo frequent changes? How reusable can you make the components of the automation framework? This is also a good time to define the objectives you are looking to achieve with the automation – Faster deployment? Better code quality? Greater confidence based on better regression tests? Essentially, start with the end in mind and measure as you go along to know if you are on the right track.

So, on to the “What”. There is both the opportunity for creating a comprehensive test automation framework and the threat that some of it could be rendered irrelevant by the pace of change in the product.
That said, though, there is value to considering automating the Unit Tests as there is a reasonable chance that several specific components will remain relatively stable over the course of the many iterations.
The greatest value could well be in automating the regression testing – the maximum bang for the buck considering the sheer number of releases. Many DevOps models allow code to be delivered late in the cycle and fixes can be applied right until the time that specific release goes live. Automating the regressions will allow you to test the entire code after each such addition and this makes it far more likely that a high-quality, bug-free product goes out to the end customer.
Among the central value propositions of the DevOps, way is continuous deployment on the operational infrastructure. Continuous deployment means continuous integration of code and of the code into the operational infrastructure. This is where automation can play a key role. An interesting approach being followed by many is to run integration testing separately and parallel from unit testing and sometimes even before unit testing. This approach believes that the integration testing is not impacted by the business logic and is only concerned with whether the product works on the deployment infrastructure. Automation helps test this quickly and relatively comprehensively.
There is also great value on automating the testing for the build – deployment stage starting at the test environment itself. The objective is to run the tests in the test or development environment and ensure smooth deployment in the production environment.

A quote we like about DevOps goes, “DevOps is not a goal, but a never-ending process of continual improvement”. While agreeing with Jez Humble, who said this, perhaps we would like to add that this continual improvement is driven by continual testing, which is turn is based on a solid test automation platform. What do you think?

January 23, 2017 ThinkSys DevOps, Quality Assurance Testing 29 comments

Role of DevOps in QA

DevOps, a compendium of development and operations is an organisational strategy, focusing on a close collaboration and communication of the software developer team with the other professionals belonging to the testing and releasing teams. The planning employs automated processes in a symbiotic environment which ultimately results in building, testing and releasing the software with clockwork and guaranteed reliability.

How does QA benefit from DevOps:

Some ten years back, QA was seen as a group disparate from the Developers’ teams, with different skill sets and responsibilities and management. Fast forward into the DevOps age, and things are quite different today. Here’s how we look at QA through the glasses of Devops…

1. Automated Deployment:

The conventional approach of a software “release” is now passé with devops facilitating the delivery of the product on a monthly, weekly and even hourly basis into the market through automated processes. This has been made possible through a continuous cycle of improvement where the developers, testers and operations people all working in sync and moving in the same direction.

2. Environment is now a part of the product:

Traditionally here’s how the flowchart used to be like….you create a software, get it verified in a testing environment of the QA team and when the litmus test is over, so to speak, unleash it into the big bad world of the user. If anything then went wrong, it was the problem of the operations teams. Not any more….As is evident from the success of Google’s Unbounce , the QA verifies the environment with their chief enabling infrastructure being the code itself. At the occurrence of any change/problem, the QA team initiates the requisite deploys, examines that the intended change, functions as expected and move over to the latest deployed code with the option of a roll back if needed.

3. Prevention is better than discovery:

In a devops environment, the priority for QA is prevention of faults and not just finding them. As opposed to let’s say ten-fifteen years ago, the QA teams of today have the luxury of pushing the code on when it’s fully functional and rolling it back when things go awry. This has positive ramifications in that, the QA team can continuously track the quality of the product. Thus the QA team has a profound influence on the development and operational phases of the software.

4. Less of Human error:

Devops enables more of automated testing in QA thus reducing the glitches due to fatigue associated with manual testing. This also ensures 100% code coverage and quick scripting of test cases.

5. Greater teamwork and rapport:

At the individual level, the testers and the operations team get a chance to be on the same page and level as the developers. This improves coordination ticking the boxes for higher market outreach and efficiency gains

May 12, 2016 ThinkSys DevOps, Quality Assurance Testing No comments yet

The Role of Software Testing in a DevOps World

By Rajiv Jain (CEO, ThinkSys)

Mark Berry said, “Programmers don’t burn out on hard work, they burn out on change-with-the-wind directives and not ‘shipping’”. I don’t know about the first but the desire to “ship” seems to be a powerful motivation for the push towards the adoption of DevOps practices among software development teams and companies everywhere. That and the business benefits obviously. I remember a Puppet Labs survey from a couple of years ago that showed that of the organizations surveyed, those that adopted DevOps shipped code up to 30 times more frequently with lead times of as little as a few minutes. They also experienced only half as many failures and had the ability to recover from those failures up to 12 times faster. These are not just numbers, I recall reading about one of the early DevOps adopters, Etsy. Apparently they update their site every 20 minutes without any service disruption to over 20 million users – that’s a phenomenal competitive edge to possess. Small wonder then that the DevOps tide is rising.

Across most SaaS-based product development efforts, the approach seems to be to bring Development and Operations together in, a largely, automation-driven effort to keep pushing releases out, more or less continuously. Given the sheer number, it’s perhaps not even fair to term these as releases any longer! Coming from a company that has a significant Software Testing practice though an interesting facet of this partnership between Development and Operations occasionally occurs to me. It seems to me that a third, equally interested party, Testing and QA, may not be getting the attention it deserves in this discussion. In fact, the case has sometimes been made that the greater degree of automation and the souped-up release cycles imply a reduced role for software testing. I couldn’t disagree more.

Let’s go back to the Puppet Labs survey I quoted earlier – notice that one of the key benefits stated was the reduced rate of failures and the ability to recover from those failures quickly. Clearly the aim is not to ship code, it’s to ship market-ready product on a “Continuous” basis. To me it seems that would be impossible to achieve without an even greater emphasis on testing. Testing, while Agile had brought out the need to involve testing strategy at a very early stage of the product planning. How else to keep up with the short release cycles? DevOps with its Continuous Integration and Continuous Delivery is, in this context, like Agile on steroids and the need, thus, is even greater to get testing involved early in the product definition and planning stage. In many ways, the approach now has to be to plan to prevent defects rather than detect them.

So, in DevOps testing comes in early, plans for the way the product is going to pan out and prepares itself accordingly so that as code starts rolling out it gets tested. This is a significant change, at least of mindset. Earlier the development used to get done and the testing would start – today these seem to have to go, more or less, in parallel. Many have called this Continuous Testing – a perfectly valid term.
Another significant change that this “Continuous” model is engineering is the integration of functions. DevOps teams now tend to contain people from Development, Operations and, increasingly, Testing. Without that level of integration, getting quality code out in such short time frames would be impossible. This integration is also throwing up new roles for everyone, including testing. Carl Schmidt, CTO of Unbounce explains it well, “I’m of the mindset that any change at all (software or systems configuration) can flow through one unified pipeline that ends with QA verification. In a more traditional organization, QA is often seen as being gatekeepers between environments. However, in a DevOps-infused culture, QA can now verify the environments themselves, because now infrastructure is code.”

That statement points to a third significant change for testing in the DevOps way – what to test? Now that the Operations, i.e the nuts and bolts that get the SaaS product out to the millions of subscribers, is a part of the effort of getting the product out is it not necessary to test that too? There are also some subtle changes of emphasis that emerge here – functional testing is always important but now there is a premium placed on testing for load conditions and for performance. The environment is also, as much a part of the SaaS product now as the code. There is a role here that testing can excel in – clearly they know more than most about the difficulties of taking the application code, deploying it in test environments, testing it and then moving it out to production. The process may have become shorter and sharper but the skills are the same.

I like a quote, sadly unattributed, that I read somewhere, it goes, “Software testers do not make software, they only make them better.” In the context of DevOps it would seem that finally they are getting an opportunity to not only make software but also to make it better!

March 30, 2016 ThinkSys DevOps, Quality Assurance Testing 1 comment

Currently browsing: DevOps