Control Plane Revolution

Control planes are a hot topic of conversation within the cloud native community. Increasingly, SREs describe the way they build platforms as being architected around a control plane. These platforms come in many shapes and sizes. Some are full-blown Internal Developer Platforms used across the entire organization. Other kinds of platforms simply orchestrate core services in a way that ensures high uptime. And still other platforms seek to better integrate with the organization’s cloud native tooling by removing a legacy Infrastructure as Code dependency.

Upbound has been at the forefront of this control plane revolution since inventing the Crossplane project in 2018. The key insight behind the creation of Crossplane was that the Kubernetes architecture was more broadly applicable than just container orchestration. In the end, Kubernetes pioneered an entire operating model of resource management, one which was highly relevant (if correctly applied) to SREs tasked with building platforms and managing infrastructure.

Extending the Kubernetes Operating Model

Kubernetes revolutionized cloud computing by applying an “Observe, Analyze, Act” model for resource management to container orchestration use cases. Massive amounts of complexity are required within a Kubernetes cluster to make the idea of this seemingly simple model a reality. At a high level, Kubernetes looks at what the desired state of things needs to be, assesses what’s different in reality, and automatically makes configuration changes which eliminate any differences between desired and actual state.

Until Kubernetes officially supported Custom Resource Definitions (CRDs) in Kubernetes version 1.7, the Kubernetes operating model only applied to bits inside of the cluster. CRDs allowed for Kubernetes users to extend their cluster’s API with custom types, freeing the Kubernetes operating model to be applied to things other than pods, containers, deployments, and so on.

When Upbound launched Crossplane in 2018, many Kubernetes users had never heard of a CRD before. Crossplane was one of the first cloud native projects to take the idea of Custom Resource Definitions to their logical conclusion. By relying on a Kubernetes controller and CRD for every cloud resource, Kubernetes users are able to orchestrate resources in the public clouds using the exact same tools, products, and methodologies they used for applications.

This insight has allowed Crossplane users to rapidly converge both application and infrastructure workflows around the Kubernetes API, and standardize on cloud native tooling which has been built up around it. Crossplane users are some of the most advanced Kubernetes users on the planet. A typical user will tell us how they’ve integrated their control planes with cloud native continuous delivery systems like Argo or Flux while simultaneously collecting platform-level metrics with modern services such as Datadog or New Relic.

From IaC to Control Plane Architectures

The journey towards convergence has partially been motivated by Crossplane users realizing they’ve been using Infrastructure as Code tools such as Terraform as a substitute for a proper control plane. Countless users have told us stories about how their CD pipelines diff the output of `tf plan` commands, and how they’re constantly struggling to manage TFSTATE between clients as they attempt to deploy infrastructure changes using GitOps.

Control planes offer an alternative architecture that completely eliminates these issues. Because Crossplane leverages the Kubernetes operating model, desired state and actual state are always kept in sync, eliminating the need to have a plan command or write state to a file entirely. Additionally, day two operations become streamlined with configuration drift automatically being reconciled by Crossplane whenever an accidental change is detected.

Lastly, platforms built around a control plane can provide consumers of these platforms with self-service super powers. On top of the CRDs that Crossplane installs into a cluster for infrastructure management, Crossplane enables platform builders to present a custom cloud API to the developers using those platforms. For example, Crossplane is commonly used to power Kubernetes-as-a-Service platforms inside large enterprises. The platform teams running these services are trying to solve the problem of enabling any developer to get a Kubernetes cluster when they need one and making sure those clusters are properly configured.

Before Crossplane, each cluster would be a one-off deployment for developers, typically with weeks of requirements gathering and custom configuration done by the platform team. With Crossplane, platform teams compose a custom cloud API consisting of two parts. One part defines the infrastructure configuration details - things like VPCs, subnets, and permissions. The second part is a definition for a self-service API designed to be used by the developers using the platform. Developers consume this API in order to deploy a new Kubernetes cluster without having to know anything about the details of the cloud environment or cluster configuration.

Democratizing Control Plane Technology with Crossplane

Engineers who’ve built hyperscale cloud computing services like AWS, Azure, and GCP have been building control planes for almost 15 years. Each of the services in the hyperscalers are designed around a unified control plane that powers the entire cloud service, while presenting an easy-to-use API to customers. For example, while an AWS S3 end-user may consume a bucket via the popular boto3 SDK, behind the scenes is an incredibly complex service with a control plane orchestrating everything from data replication to supply chain logistics for new hard drives coming online.

For almost 15 years, how to build and use a control plane has essentially been proprietary knowledge. The consequence is that well-architected services and systems were the exception rather than the norm. Platform engineers simply didn’t have a corpus of work to build on or a community where best practices could be shared and components reused.

By open sourcing Crossplane and donating it to the CNCF, Upbound democratized control plane technology for the masses. Now anyone who knows how to build a microservice using Kubernetes can deploy a control plane for their organization. There’s also a vibrant and thriving community being built around Crossplane, too. Every day, new platform engineers are joining the Crossplane Slack to learn from one another, share best practices, and collaborate on the project.

The community’s growth and collaboration has increased so dramatically that Upbound launched Upbound Marketplace, an open hub for the Crossplane community to discover and share plugins for Crossplane.

Production with Upbound

Upbound created the Crossplane project to take the cloud native space to a new level. As the inventors of this technology, Upbound is uniquely positioned with a wealth of knowledge and understanding to effectively harness Crossplane’s capabilities into a complete solution that fits your specific needs.

If you are interested in leveraging the power of control planes without needing to worry about operational concerns or managing the underlying infrastructure, check out my webinar where I talk more in depth about the control plane revolution with my collaege Sumbry.