CI/CD Pipeline Security: Key Risks and Best Practices Explained

CI/CD pipelines are powerful enablers of faster, more frequent software development, from design through deployment. But when they first hit the tech scene, they often had a missing link: security.

That’s a problem, especially given stats like this one: approximately one in five organizations reported a security incident in their CI/CD pipeline in the previous year, according to a recent survey conducted by Techstrong Research.

As CI/CD pipelines have become increasingly common, they’ve also altered the threat landscape for software teams. As the Open Worldwide Application Security Project (OWASP) notes: “Given their importance and popularity, CI/CD pipelines are also an appealing target for malicious hackers, and their security cannot be ignored.”

In this article, we’ll dig into the specific risks – and effective principles and practices for mitigating those risks and ensuring CI/CD pipeline security.

What is a CI/CD Pipeline?

First, let’s make sure we’re on the same page: What is CI/CD? Short for Continuous Integration and Continuous Delivery, CI/CD refers to a set of processes and tools that modern software teams use to design, build, and ultimately deployment – with automation as one of the core enablers of faster, more frequent code delivery and updates.

Here’s how we defined it in our previous article, “CI/CD Pipeline Overview: Why You Should Know About It”: “A CI/CD pipeline is a transparent, automated, and reliable software development and delivery process.”

The goal is usually not just to ship faster and more frequently – it’s also to improve quality, reliability, and – assuming you’re giving it the proper priority – security.

Common CI/CD Security Risks: 5 Focus Areas

The roots of many CI/CD security risks should sound at least somewhat familiar to a tech pro. Things like access control and over-privileging, insufficient monitoring and logging, ungoverned dependencies and software supply chains all contribute to increased risks when not properly managed. 

OWASP has published a list of Top 10 CI/CD Security Risks worth checking out as a benchmark. It’s an offshoot of the organization’s popular OWASP Top 10 Web Application Security Risks, which is a widely accepted industry baseline for web application security.

CI/CD pipeline risks can be both introduced and exacerbated by automation. Many of the risks are introduced automatically – or deployed to production automatically – in CI/CD pipelines. Somewhat ironically, automating the security of those pipelines with Infrastructure as Code (IaC) tools or other automation can likewise introduce or compound risks if you’re not paying attention, notes Derek Ashmore, Application Transformation Principal at Asperitas.

Here are five key risk areas to consider when securing your CI/CD pipelines, according to Ashmore:

• Access Control: “Access control is often over-provisioned out of convenience and doesn't adhere to the principles of least privilege and dual control,” Ashmore says. In general, the principle of least privilege should prevail here – don’t give people or machines access to data or systems that they don’t actually need to do their jobs.
• Unvalidated Dependencies: Today’s software – and as a result, CI/CD pipelines – often relies on third-party libraries and other external dependencies, and those components sometimes have vulnerabilities. “Always ensure external dependencies are validated and have an auditable chain of custody,” Ashmore says.
• Auditing and Logging: Make sure you have comprehensive auditing and logging implemented. A lack of visibility and auditability can lead to increased security exposures that you’ll otherwise be unaware of.
• Privilege Escalation: Automation – of the sort intrinsic to CI/CD and CI/CD security – can sometimes be developed in silos that lead to unintended gaps where privileges can be elevated by running an unintended automation sequence. “To mitigate this risk, ensure least privilege is implemented and test cases are implemented for automation work streams and individual tasks or pipelines,” Ashmore says.
• Inadequate Automated Testing: Automated testing is key not only for strong CI/CD pipeline security – think things like automated vulnerability scans for container images, for example – but also a healthy pipeline overall. “Pipeline defects can create unwanted security vulnerabilities and negatively impact application development teams by preventing them from doing their assigned work,” Ashmore says.

10 Top CI/CD Tools!

Here's my pick of the 10 best software from the 10 tools reviewed.

1. 1. Travis CI — Best for on-premise deployments
2. 2. Azure DevOps — Best for Azure development
3. 3. Terraform — Best repeatable code
4. 4. GitHub Actions — Best for small teams
5. 5. Jenkins — Best for scaling companies
6. 6. OpenShift Pipelines — Best open-source option
7. 7. Spinnaker — Best for custom integrations
8. 8. CircleCI — Best for enterprise development
9. 9. Google Cloud Build — Best collaborative features
10. 10. GitLab CI/CD — Best maturity feedback

Show More (5)

15 Best Practices for Stronger CI/CD Security

Once you’ve got a solid understanding of the risks, it’s time to turn to solutions. The good news: there are lots of them, in part because the industry at large has realized the importance of integrating security as a first-class citizen in CI/CD. 

Ashmore from Asperitas gave us a rundown of 15 commonly accepted best practices and tactics for bolstering CI/CD pipeline security. Without further ado:

1. Use Secure Version Control Systems: Version control systems like Git capture change history that might be needed to investigate security breaches or other issues. “Ideally, nothing should be changed manually,” Ashmore says. Check out our comprehensive list of source control tools for more options: “20 Best Version Control Tools Reviewed For 2024.”
2. Enforce Least Privilege: Make sure that roles and permissions granted to both human users and to things like infrastructure resources are minimal and follow the principle of least privilege principle. If they don’t need it, don’t grant it.
3. Use Secrets Management: “Avoid hardcoding secrets – passwords, tokens, API keys – in IaC templates,” says Ashmore. Instead, use secrets management tools like HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault. Check out our deep dive on secrets management tools for even more options: “Digital Safekeepers: The 24 Best Secrets Management Tools Of 2024.”
4. Perform Regular, Automated Security Scanning: Use tools that automatically scan for security vulnerabilities, and do so at the earliest possible stages of your CI/CD pipeline.
5. Follow the Principle of Immutability: Minimizing manual changes, especially to infrastructure environments, is a good strategy for minimizing risks. “Infrastructure should be immutable; once provisioned, it should not be altered manually,” Ashmore says. “Instead, changes should be made by updating the IaC code and redeploying. Removing the ability to make manual changes is the safest path.”
6. Use Role-Based Access Control (RBAC): “Control access to your IaC tools and environments using RBAC,” Ashmore says. “Only authorized users should be able to deploy or modify infrastructure.” Kubernetes is another important example here, especially since it’s a key part of many CI/CD pipelines. Don’t just rely on out-of-the-box configurations.
7. Apply Idempotency: Now we’re getting fancy: idempotency is the concept of being able to run certain operations – or code, in the case of software engineering – multiple times without changing the outcome. “Design IaC [and CI/CD pipelines] in an idempotent way, meaning that running the code multiple times should not change the state of the infrastructure unless explicitly required,” Ashmore says.
8. Use Parameterization: Ashmore recommends using variables and parameterization in IaC templates to avoid hardcoding environment-specific values, such as regions, instance sizes, and IP addresses.
9. Implement Monitoring and Logging: These are needed to detect unauthorized or unusual activities, and to troubleshoot root causes when incidents occur. “Services like AWS CloudTrail or Azure Monitor can help,” Ashmore says. Here are two more resources: “The 25 Best Log Monitoring Software Picks” and “Guide To The 26 Best Infrastructure Monitoring Tools Of 2024.”
10. Use Infrastructure Testing Tools: “Incorporate infrastructure testing tools such as Test Kitchen, Terratest, or InSpec to validate the correctness of your IaC before deployment,” Ashmore says.
11. Implement Infrastructure Drift Detection: Use tools to detect infrastructure drift – i.e., changes that occur outside the CI/CD pipeline – and revert unauthorized changes when detected, Ashmore advises.
12. Follow Compliance and Security Standards: Ensure your CI/CD pipeline passes muster with industry security standards and compliance frameworks, such as CIS benchmarks, NIST, or GDPR. The OWASP Top 10 list we shared above is another example.
13. Create Reusable and Modular IaC Code: “Break down IaC code into reusable and modular components, such as Terraform modules or AWS CloudFormation stacks,” Ashmore says.
14. Document Your Pipeline: Ashmore recommends creating and maintaining thorough documentation of your IaC policies, templates, and security procedures, as well as other components of your CI/CD pipeline.
15. Regularly Update Tools: “Keep your IaC tools (such as Terraform, Ansible, CloudFormation) and any related dependencies updated to the latest versions,” Ashmore says. The same advice applies widely to other tooling and dependencies in your pipeline: outdated versions are more likely to have known (and possibly unknown) risks.

Need expert help selecting the right tool?

With one-on-one help, we guide you to your top software options. Narrow down your software search & make a confident choice.

Get free tool advice

Join for More CI/CD Insights

CI/CD pipelines have become mainstays of modern software development. That means CI/CD security is every bit as paramount. Before you go, sign up for our newsletter to get the latest insights from top thinkers in the software industry.