Protecting sensitive data in microservice architectures with consistent policies

Auhtor: Ryan Aebi

In modern microservice and zero-trust architectures, access decisions are made on the go on each component independently. This presents a very significant challenge:

• Access authorization to sensitive data can be implemented quite differently by product teams in their respective microservices.
• Changes in access authorizations, in turn, need to be adjusted in each microservice.

The Open Policy Agent (OPA) serves as a policy engine for this purpose:

• Decouple the Policy Decision Point (PDP) from the Policy
• Enforcement Point (PEP) Enforce authorization in microservice and devops driven architectures as a Policy Decision Point
• Manage access authorizations centrally.

Authorization in Microservice Architectures

How authorization decisions work formally

Several (logical) components are involved in an authorization decision for an API call.These components can be roughly described by the following diagram and text:

• The Policy Enforcement Point (PEP) is the component that implements the decision for an API request (e.g. Allow / deny).
• The decision as to whether access is permitted is made by the Policy Decision Point (PDP).
• In order for the PDP to make this decision, it accesses the Policy Information Point (PIP); this manages the policies that are used by the PDP as the basis for the decision.
• The policies in the PIP can be viewed, administered and edited by the so-called Policy Administration Point (PAP).

Cloud-native development and micro services

New trends deal with Zero-Trust [2], there is no central security anymore which controls everything.
The goal here is to look at each API call / request separately and thus perform authorization in each component and therefore in each access. 
This requires that each component implements a PEP and can independently allow or deny a corresponding API call.

Authorization implementation in each service

The fact that each component must itself check whether an API call should be allowed or not means that each component must also be capable of making this decision. 
As a micro-service developer, this means that in addition to the PEP task ("I must be able to allow or block access"), a PDP ("I must be able to decide whether access is allowed") must also be implemented. This should be done within the same microservices.
As a rule, this leads to each microservice implementing its PDP differently and, in particular, using different rules and policies as a basis for decision-making. In other words, there is no central PIP, policies are distributed everywhere and are formalized differently.
Besides a massive governance overhead to keep the situation under control, this also means that every informal policy change leads to a code change: 

• new build / release
• new regression tests
• new deployments (downtime / waiting window).

In the worst case it could look like shown below and this for each microservice.
 

In addition, there is the issue of service level/user level authorization. This is about which microservice is allowed to consume which API (what-can-do-what) and what a user is allowed to consume from a service/application (who-can-do-what). All these decisions need to be implemented per micro-service, more precisely per API. 

Are you ready for this architectural decision, the additional costs as well as the resulting risk?

How can governance overhead be avoided?

In many ways, it makes sense that in a zero-trust driven architecture, each component implements a PEP. Both the backend, the microservice, and even the database must be self-accountable to enforce a policy decision.
Imagine the following scenario:

• A new employee starts in your company as a microservice developer.
• For this purpose, the person needs access to the internal tools (e.g. Jira, Confluence, Gitlab, etc.) (user-level authorization) as well as further permissions on the microservice (service-level authorization) for the further development of his microservice.
• So who assesses which microservices need to be customized and how?

The governance overhead is not in the PEP, but in the PDP, PIP and PAP. A possible solution to this is provided by the Open Policy Agent (OPA) [3].

Open Policy Agent

The OPA project was launched by the Cloud Native Computing Foundation (CNCF) in April 2018 in the CNCF Sandbox and one year later in the incubation phase [11].

Finally, in February 2021, the OPA achieved 'graduated' status, a reserved term given by the CNCF to projects that have achieved a certain level of maturity in several areas [11].

Decoupling Policy Governance & Policy Decision from Services

The OPA allows policies to be managed as code. Thus, a Git repository (Github, Gitlab, Bitbucket, etc.) can be used to manage policies: 

• versionable 
• traceable 
• centrally manage

This means that each microservice no longer has to manage and edit its policies itself. Developers are even empowered to build their continuous deployment pipeline / rolling releases, while they can simply consume the OPA / PDP as a service to build / deploy and runtime. In this respect, the tasks of PIP (policies are managed centrally in a repo) and PAP (policies can be edited centrally in the repo) are eliminated for the microservices.
While the policies are written in Rego, the OPA handles the publication as a REST service. The OPA processes the request based on the stored Rego policy and then decides accordingly whether access should be allowed or not. The OPA thus assumes the role of the policy decision point (PDP) by providing information on whether a request is allowed. The enforcement of the policy decision (PEP) remains with the microservice.

More benefits with OPA: Kubernetes & Terraform

One of the nicest features of OPA and Rego is that they are not limited to microservices. Rego can be used to model basically anything. For example, in a Kubernetes environment, it can be controlled that [6]:

• only images from predefined repos may be used
• only images with certain labels may be used
• only predefined ports may be used
• TLS is used.

Analogously, in a terraform environment, it can be enforced:

• That only certain services may be used in a Terraform plan (service whitelisting) [7]
• How many instances of a service may be added/deleted or modified [8].
   

Conclusion

Less risk for data leakage

In many ways, it makes sense in modern microservice/devops architectures to implement layered security by having each component implement its own Policy Enforcement Point (PEP). The Open Policy Agent supports this by providing PDP, PIP and PAP via central policies.

Governance centrally manageable

Since the Open Policy Agent obtains its policies through a source code repository, there is a single source of truth. There is no delta because microservices implement their policies themselves, instead they can be viewed and edited centrally. This makes it possible to see at any time which policy was deployed/active, but also which changes were made last.

In summary, the advantages are:

• Policies can be versioned.
• Policy changes are documented and can be centrally integrated into a release process.
• Policy/security audits can be performed centrally.

Applicable for Microservices and Kubernetes

The Open Policy Agent can be used wherever decisions need to be made based on a policy. The policy language Rego is universally defined and can therefore be used everywhere.

Transparent decisions through Policies as Code

A central enabling of OPA is that policies can be managed centrally in a source code repository. In this way, policies are managed in a versionable and traceable manner. In combination with a clean deployment pipeline, it is possible to check at any time which policy was deployed to which OPA, how it looked configurationally, and thus also record policy decisions. This interplay of code-based policies and traceability through deployments is ideal for any audits.

Sources:

1. Speed in der Entwicklung Dank Cloud-native
2. So schützt Google Cloud ihre Kunden und deren Daten - 5 Massnahmen
3. https://www.openpolicyagent.org/ 
4. https://www.openpolicyagent.org/docs/latest/
5. https://play.openpolicyagent.org/ 
6. https://thenewstack.io/open-policy-agent-the-top-5-kubernetes-admission-control-policies/
7. https://scalr.com/blog/opa-series-part-1-open-policy-agent-and-terraform/
8. https://www.openpolicyagent.org/docs/latest/terraform/ 
9. https://www.openpolicyagent.org/docs/latest/external-data/ 
10. https://github.com/open-policy-agent/contrib/tree/main/kafka_authorizer 
11. https://www.cncf.io/announcements/2021/02/04/cloud-native-computing-foundation-announces-open-policy-agent-graduation/