Argo CD and Crossplane for Managing the Whole Application Stack

At Akuity, we are collaborating with Upbound to streamline our infrastructure using Argo CD and Crossplane. This blog post will explore how Argo CD users can implement Crossplane to manage infrastructure resources beyond Kubernetes to simplify infrastructure management and create a clear sense of the dependencies for an application.

Introduction

Argo CD is a prevalent tool for implementing GitOps, the practice of declaratively storing the desired state with immutable versions and using tooling to reconcile the live state, in Kubernetes. Argo CD enables GitOps by implementing the Application resource, which provides a declarative approach to managing config management tooling for Kubernetes resources (e.g., Helm, Kustomize) and a controller for handling the reconciliation loop for syncing the resources into the cluster (i.e., updating the live state to match the desired state).

A basic example of an Application looks like this:

This 'guestbook' Application will create a Deployment and a Service in Kubernetes based on the manifests in the `guestbook` folder of the `akuity/argo-cd-crossplane` repository.

What is not represented by the manifests in the repository is that the fictitious guestbook application relies on a bucket for storing images uploaded by guests. But since the bucket isn’t a Kubernetes resource, Argo CD can’t manage it without the help of another tool.

Before a Universal Control Plane

The existing process to create the storage bucket is to log in to the GCP console and create a new bucket, specifying its name, location, storage class, access controls, configure versioning, and the lifecycle rules. Once the bucket is created, you need to configure the application to use it.

This whole process is done once, with none of it represented declaratively. So the next time the application is deployed to a new environment, another bucket must be created manually. Hopefully, the same person is creating it, or the documentation is good. Otherwise, you will waste time figuring out what settings are needed for the bucket (probably by comparing it to the existing one).

The downside of this approach is that it requires the application developers to have direct access to GCP or rely on an operator to create it, and it’s a manual imperative process prone to human error. For example, the bucket access could be easily misconfigured, compromising the bucket's contents. Additionally, if the bucket configuration needs to be updated, the change must be made manually, potentially leading to inconsistencies between the desired and actual state.

Introducing Crossplane

Crossplane is an open-source Kubernetes-native framework for building cloud-native control planes using a consistent API without writing code. Crossplane facilitates creating resources on various providers, including GCP, AWS, and Azure, using configurations defined as Kubernetes resources, making it easier to manage infrastructure using familiar Kubernetes tooling.

Upbound created Crossplane to democratize control plane technology for all and continues to drive it forward as maintainers and contributors. Upbound also offers commercial products and support for those organizations that want to take Crossplane even further.

Using Crossplane to Manage the Storage Bucket

To use Crossplane to manage the storage bucket, you must install the GCP provider (`provider-gcp`), which requires authentication to a service account with permission to manage the resources on the provider. This can be done with a JSON key from a service account in GCP or using Workload Identity to authenticate based on a service account in Kubernetes.

Creating the Crossplane GCP provider in Kubernetes will also introduce all of the Custom Resource Definitions (CRDs) representing resources on GCP. Including one for managing storage buckets, `Bucket`. The CRD defines the structure of the resource and the parameters that can be configured, such as the bucket name, location, and access credentials.

A basic example looks like this:

Adding the Storage Bucket to the Application

To add the storage bucket to the Kubernetes resources managed by the Application, you need to add the `Bucket` manifest to the `guestbook` folder of the `akuity/argo-cd-crossplane` repository. Argo CD will then deploy the new `Bucket` custom resource (CR) alongside the existing Kubernetes resources, ensuring that the live state of the cluster matches the state described in the Git repository. Once Argo CD creates the CR in the cluster, Crossplane will use the provider to create the corresponding resource on GCP using all of the configurations from the `Bucket` CR.

This approach provides several benefits, including:

• creating visibility into how the bucket is configured,
• reducing the likelihood of human error, and
• making it easier to manage infrastructure alongside application resources using Kubernetes tooling.

Conclusion

Using Argo CD and Crossplane together provides a powerful toolset for managing infrastructure and application dependencies in Kubernetes environments. Defining infrastructure as Kubernetes resources makes it easier to manage the entire stack using GitOps practices and Kubernetes tooling.

Resources that were once a distant and implied dependency of the application can now be managed declaratively alongside the Kubernetes-native resources. The complete configuration for the Bucket is transparent and easily reproducible. Creating a simple mechanism for deploying new instances of the application in any Kubernetes environment with Crossplane and Argo CD set up.