Skip to main content

When you think of a high-performance organization, have you ever wondered what best practices they follow that enables them to achieve that stature? Do they lean towards agile or DevOps approaches?

To achieve their goals, these organizations often rely on a mix of agile tools for managing iterative development and adaptive planning, alongside DevOps tools that streamline automation, continuous integration, and deployment.

The objective of a development and operations process is to help the company fulfill its requirements, help the team members collaborate and break silos to unblock issues and finally complete the projects/features they are working on, so they can increase their daily or monthly active users – or in the case of enterprise applications - increase the functionalities of their applications.

When teams work in silos, it results in gaps in communication which in turn results in chaos. Whereas when teams work in tandem, they are more effective. 

For the scope of this article, I will cover agile and DevOps approaches, examples of when to adopt specific methodologies, and how testing is impacted in each of these scenarios.

Agile Software Development Practices

Compared to the waterfall development process, agile focuses on one of the core principles, which involves satisfying customers early on in the process and through continuous delivery. Continuous delivery is possible only when we involve the quality team early in the process and collaborate with them frequently.

Agile software development process.

Currently, some of the smaller to mid-size companies are in a state where they are following neither agile approach nor waterfall entirely but a mix of both. These so-called agile team members focus so much of their energy on granular processes like 30-minute standups and 60-minute retrospective meetings, forgetting about the return on investment that they bring to the table, which ends up in a situation where a lot of bugs are found in production.

Discover how to deliver better software and systems in rapidly scaling environments.

Discover how to deliver better software and systems in rapidly scaling environments.

  • By submitting this form you agree to receive our newsletter and occasional emails related to the CTO. You can unsubscribe at anytime. For more details, review our Privacy Policy. We're protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
  • This field is for validation purposes and should be left unchanged.

Has the industry really transformed from a waterfall approach? 

Have you ever been part of an organization (which you thought followed agile software development processes) only to realize that the quality team is involved in testing only after development is completed?

In this example, they still called it agile because the development team members worked on multiple branches (or features) at a given time in an iterative manner. But when they had to get their features tested, they waited until all development was complete—which is the core of waterfall methodology. However, since companies want to move faster so they can expand their customer base, most of the time, teams end up accumulating technical debt.

The best way to solve this problem is to convert their organization to become completely agile and work hand-in-hand with the quality team iteratively. This involves working with all stakeholders to ensure they understand the consequences of not making the switch and how it may affect the end-user experience. 

Agile Frameworks and its Variants

For the scope of this article, I'll discuss two of the most popular agile frameworks:

  1. Scrum
  2. Kanban

Scrum is an agile methodology used in software development based on iterative and incremental processes. It consists of adhering to time-boxed development called sprints. The duration of sprints varies according to each organization, ranging from weekly to monthly to quarterly.

There’s a sprint planning session, followed by the sprint where implementation and testing happen, including daily standups, and finally ends with a retrospective session.

Scrum methodology.

Scrum is usually implemented in product development teams where the team members have to timebox their work to meet the customer deadlines. It mostly applies to B2C applications because if you wait too long, your feature is no longer new, as someone could have implemented it already!

In B2B applications, which are mostly enterprise companies, a modified version of agile methodology called SaFe - Scaled Agile framework is commonly used. 

Most of the projects at enterprise companies fall into one of the following categories:

  1. Team level
  2. Program level
  3. Portfolio level

Team-level projects are feature-based development that involves each team member to own their project and processes. These teams usually use Scrum or Kanban depending on the nature of their work.

If the teams fall under research & development organizations or test automation, then it is imperative that they adhere to Kanban, which is less rigid, and the goal is more towards completing their current task than jumping onto the next shiny object at hand! If the teams are part of the product development organization, then they tend to use scrum processes.

Program-level projects involve multiple teams working towards a specific goal, such as AWS migration. Even though this project can fall into the portfolio level-based project, I would argue that it wouldn’t necessarily affect HR or accounting teams.

In this case, the project AWS migration could last months due to unforeseen issues, so the focus is on completing the AWS migration successfully while breaking it into justifiable sprints. 

Portfolio-level projects involve different organizations within the company; an example would be JIRA implementation. Each organization would require JIRA workflows to be implemented differently based on their needs. Human resources teams don’t require testing; the focus is mainly on serving specific requests to employees, and once they help them, the task is marked as “complete.”

Agile methodologies should be adopted by organizations that are known to undergo changes constantly. But these changes still have to go through a round of testing and bug fixing, and testing isn’t necessarily automated, leading to increased process times.

Finally, when the features are ready to be deployed, they are handed off to the build/ops team. So, testing happens iteratively when compared to the waterfall approach but doesn’t shift far left as the focus isn’t on continuous testing and delivery. Hence the need for the DevOps approach!

DevOps Approach

The DevOps approach focuses on aspects beyond development and testing and evangelizes a completely automated CI/CD pipeline along with monitoring capabilities. While Agile embraces changes, the DevOps approach focuses on continuous testing and delivery to ensure frequent releases to the end-users are successful.

The goal of the IT Operations team is to scale the software development process to accelerate the writing and updating of the code responsible for creating new applications and services and updating features within the IT team.

Even in terms of team setup, these days, the quality and IT operations teams are part of the same organization so they can work hand in hand. In fact, a new role called TestOps has been introduced to focus specifically on setting up the CI/CD pipelines where automated tests are run against pull requests, giving immediate feedback to the development teams.

Automated testing is essential to scale testing efforts and its value is lost if it is not part of continuous testing and continuous deployment release cycle. So, the TestOps personnel always have their hands full with automated test infrastructure maintenance.

This ensures that the operations team members are focused on delivering a world-class infrastructure for software development without getting distracted by testing infrastructure.

The DevOps approach.

There are a lot of tools and DevOps practices in place these days that streamline the overall process.

  • A Terraform state file refers to a set of infrastructures that are defined and managed as a single unit—this is how various testing and development environments are defined and managed. Terraform helps configure servers, but we need an infrastructure to run these servers. 
  • AWS provides the infrastructure via EC2 instances, which are currently the most cost-effective way to configure and run these servers. To take a step further, if your tech stack has a lot of microservices, with docker compose, you can define and run multiple container docker environments.
  • Ansible automates the process of configuring machines to run any processes or servers. 
  • Kubernetes helps to manage a cluster of these EC2 instances as a pod, and schedules containers to run on this cluster based on the available compute resources. 

Should You Use Agile or DevOps?

While the Agile software development vs. DevOps approach is a constant debate within organizations, the best way to look at it is based on asking ourselves the following questions:

  • What is the adaptability of our organization when it comes to new technologies? 
  • Are we able to oppose our competitors in terms of the quality of our products? 
  • Are customers likely to use our products as it meets their expectations? 

If the answers to the above questions involve some form of negation, then it is time to focus more on a combination of both approaches and customize it to suit our needs.

The Ideal Software Development Methodology

The key difference between agile and DevOps-based software development processes is that the former focuses more on accommodating constant changes, whereas the latter focuses on continuous testing, delivery, and deployment. Collaboration between the development and operations teams is crucial so that the dev team is not blocked.

So, in my opinion, an ideal software development process should entail the following:

  • Handling customer/user personas and incorporating customer feedback
  • Focusing on best practices to prevent technical debt from accumulating
  • Continuous improvement, continuous testing, continuous integration, continuous delivery, continuous deployment and monitoring

Here’s what the process would look like:

Customer-driven AI/ML-based methodology.

Handling different customer personas is not restricted to product management teams. At the end of the day, why are we developing these products? What is the purpose of these products if they aren’t being used by customers? 

Take the classic example of Nokia—when they kept accumulating technical debt for not keeping up with the market trends, namely their competitor Apple’s innovation. They focused on following rigorous sprint cycles only to eventually be forgotten! 

All companies should understand why their features are being developed and what kind of customers they cater to. This will ensure that we are developing user-centric features.

When a new product is released, be it a mobile application or a web application, always ensure there’s a way for customers to leave their feedback. Not all companies follow a strict process of working closely with customer support. 

CI/CD

Finally, advocate for AI/ML-based continuous integration and continuous delivery. When a project fails, it doesn’t necessarily connote the lack of skill set of the team members; it is instead the lack of sufficient testing and failure to find issues earlier during development.

But sometimes, even with adequate testing and automation in place, things go wrong—so if there was a recommendation system in place to identify when to release vs when not to, maybe such catastrophic failures could be averted.

Frequent iterations help the development team understand what’s working and what’s not and aid in thinking about scalability. The caveat is that it's time-consuming! If another recommendation/prediction system could gather data about customers and predict if a specific feature would help gain traction vs. fail even before development begins, it would help streamline the process of building a high-quality product from day 1!

The overall objective is to ensure business impacts multiply faster when these best practices are adopted as part of the development process.

Final Thoughts

In the end, both Agile and DevOps play crucial roles in modern software development, but the right choice depends on your organization’s goals, team structure, and project needs.

Agile focuses on iterative development and flexibility, while DevOps emphasizes continuous delivery, automation, and collaboration between development and operations. Many organizations find success in combining both approaches to maximize efficiency and quality. No matter which path you choose, it’s essential to align your tools and processes to drive innovation and deliver value faster.

For more insights on optimizing your development workflows and staying ahead of industry trends, subscribe to The CTO Club's newsletter.

Niranjani Manoharan

An accomplished software engineering leader in building tools, test infrastructure and improving quality and developer productivity for industry pioneers like Lyft, Pinterest, eBay, Twitter and now at Hippo Insurance.