Community Day Recap: How the Kubernetes Resource Model Enables Configuration as Data

Kubernetes co-founder Brian Grant explains what Kubernetes as a Framework for Control Planes is all about.

Kubernetes can and is being used as framework for control planes, thanks in part to the Kubernetes Resource Model (KRM)

Brian Grant, Distinguished Software Engineer at Google is one of the founders of Kubernetes and is credited with creating KRM. In a session at the Crossplane Community Day, Grant explained what the Kubernetes Resource Model is and how it enables what he referred to as, Configuration as Data.

"Anybody who's done it or even seen it, knows that infrastructure configuration is hard for a number of reasons," Grant said. “APIs are verbose but they're also inconsistent and not necessarily designed to be declarative, they're designed to be programming interfaces.”

Infrastructure as Code vs Configuration as Data

Grant explained that Infrastructure as Code is a fairly common way to manage configuration and it's considered by many to be a best practice.

Infrastructure as Code actually refers to two ideas that are often mixed together.

The first idea is to manage configuration using the same tools and practices as code, such as text, version control and CI/CD systems. The second is to represent infrastructure configuration as code or in a code-like format.

"Infrastructure as code tools aim to simplify the programming of infrastructure APIs, however I've seen thousands of configuration files for hundreds of different tools using dozens of different configuration formats and virtually none are as simple as users would like," Grant said. “An alternative approach is to represent configuration as code, which I call configuration as data.”

According to Grant, Infrastructure as Code tools generally assume they solely control the desired state of resources under management. Additionally, all configuration changes generally need to be made using the templates of those tools. The templates don't generally work well with other tools and the only way to modify them is for a human to edit them. The templates are designed to be an interface for humans not for automation.
 

The Kubernetes Approach

The Kubernetes approach to configuration as code is centered around its declarative APIs and control plane.

Grant explained that at a high level the Kubernetes control plane can be thought of as a document store, and asynchronous controllers can operate on it. The API service support REST-like APIs that primarily support Create/Read/Update/Delete (CRUD) operations on resources. All the persistent resources are stored in the etcd key value store and all other system components including the scheduler are controllers. Both the end user clients and controllers implement the same Kubernetes business logic and interact with the same APIs.

"I coined the term Kubernetes Resource Model (KRM) because i thought the term API overemphasized operations," Grant said. “In the declarative model the resource content is primary, rather than the operations.”

Grant noted that API extensions such as Custom Resource Definitions (CRDs) enable a number of type specific control plane behaviors including authorization policy and schema discovery.

"It is these API extension mechanisms that make the Kubernetes control plane general purpose," Grant said. “It can be used to control pretty much anything and we can see from the public cloud that there is no bound to the number and variety of possible services.”

What the Kubernetes Resource Model Looks Like

A key aspect of the Kubernetes Resource Model is its uniform declarative metadata such as names and labels. Grant emphasized that along with consistent operations, consistent metadata enables bulk operations and client-side control. KRM also encourages declarative references and sets.

However Grant noted that one thing that's currently missing from the model is the ability to automatically find cross references.

Desired State

KRM references also include the desired state. Grant observed that there is no hard line between human specified intent and automatically set attributes, and as such both are included in the desired state.

Something that's currently missing from KRM is consistent status conditions which Grant commented is an area where Crossplane has developed its own convention. He added that the Kubernetes SIG CLI has also developed a similar specification which could perhaps be converged one day with the Crossplane effort.

Kubernetes Configuration Principles

Grant emphasized that unlike most configuration tools the approach established by Kubernetes with the kubectl apply command doesn't assume exclusive control. As such, non-conflicting changes to the desired state can be merged so that human-driven changes and automation can interoperate.

The decoupling of the tooling and provider layers in Kubernetes has really enabled both to flourish, according to Grant. Instead of implementing support for every individual resource type or API in every tool, support for KRM can be added to tools generically which drastically reduces the integration effort.

"The Kubernetes control plane is increasingly being used as a declarative platform to manage arbitrary infrastructure, resources and services, not just Kubernetes workloads," Grant said. "The systems it manages don't have to run on Kubernetes at all as Kubernetes provides an open consistent declarative universal control plane that can drive heterogeneous systems."

The second Crossplane Community Day was held December 15, 2020 and you can watch the complete replay on Youtube, as well as read the full wrap-up report. Ready to dive deeper? Join the Crossplane community on Slack, follow on Twitter or get started on GitHub.