Open Policy Agent Moved to CNCF Incubating Status

When the project was started three years ago, we predicted that the increasing complexity of cloud-native systems (their inherent distributed and automated nature), combined with developers’ need for speed would eventually become unsustainable. No longer would policies written in wikis, PDFs, and emails be effective at protecting the business from financial, security, compliance, and operational risks. In the cloud native world there are simply too many teams delivering too many architecturally diverse services running on platforms with too much automation for people to keep up with the pace. The only viable solution we could see was taking those policies out of wikis and PDFs and putting them into software that would be able to monitor and enforce those policies at the pace of the cloud-native world.

We envisioned developers and administrators writing policies—invariants, more accurately—that describe how infrastructure and applications should be configured and behave, e.g., which applications should run on which compute nodes, how those applications should communicate with other applications, and which kinds of storage they should use. Those invariants would be enforced by a range of systems: orchestrators, provisioners, CICD pipelines, and even the applications themselves. Those same invariants could also be used passively to monitor applications and infrastructure so that admins and/or machines would know precisely where the problems are, as they emerge, and when appropriate, to raise alerts about critical issues observed in live systems.

Turning that vision into reality required addressing two core challenges. First, we simply couldn’t find a declarative language suitable to express these invariants. While there were a plethora of decIarative policy languages (both in academia and in industry), the rich, structured data of modern APIs (think JSON and YAML) made it difficult to express the logical conditions in a concise, readable, and reusable manner. Second, cloud software systems have strict availability requirements (i.e. SLAs) that extend to policy decisions and thereby necessitate a local agent that makes policy decisions. It was these two observations around the complexity of structured data, and localized policy decision-making that became the two founding cornerstones of Open Policy Agent and Rego, the policy language it provides.

As we talked to more people, it became evident we were not alone in these two observations. In fact, we heard more and more use cases across the stack—from the lowest layers of infrastructure to the highest layers of applications. As the cloud-native stack started to mature and evolve, we saw large sections of the community using OPA to build authorization solutions for Kubernetes and for custom applications, which are still today the two leading OPA use cases.

Right from the beginning it was clear for us that OPA and Rego would have to be community-driven efforts. They would thrive only if they had an active, passionate community built around them, feeding in language ideas, improving tooling, and building integrations with external systems. The CNCF was a perfect fit for the goals we had for OPA, and we knew they’d provide a great home for the project. So in March 2018 we were thrilled to donate OPA to the CNCF.

The diversity and energy of the OPA community is something we are particularly proud of. Looking back, it was late 2017 when the first major user of OPA (Netflix) talked publicly about their OPA use cases. Since then OPA has attracted production users from a wide range of companies: from Fortune Global 50 to those with less than 50 people. It has always been a community effort, and the breadth of OPA talks and discussions at KubeCon North America 2018 highlight the energy this community has moving forward.

Steadily growing adoption has made OPA more mature, both in terms of the code and the community around it. Rego has been proven in production in some of the largest cloud-native deployments in the world, and we continue to solve new and evolving customer problems both on the open source and commercial sides of the project. OPA moving to Incubation is well timed in this regard: the OPA community is now ready to welcome more contributors and help newcomers understand how to get the most out of what OPA has to offer.

It’s been an amazing journey so far, and it’s far from over. As Vikings would say: Skål (Cheers) to the OPA community!