Crossplane Testing: Rendering and Validating Compositions

This is the first of a series on testing tools and patterns in Crossplane.

Crossplane is a Universal Control Plane that can manage almost any resource, like AWS S3 buckets or Azure Databases. Building upon Kubernetes Custom Resource Definitions (CRDs) and continually-reconciling controllers, Crossplane allows Platform Engineers to provide a cloud-like experience to end users.

While Crossplane provides an excellent foundation, Engineers still need to define APIs and configure cloud resources. Catching errors early is critically important in reducing outages and misconfigurations. This blog will cover testing during the infrastructure development cycle.

A Review of Crossplane Concepts

Crossplane has a modular control plane architecture that has multiple extensibility points and supports day 2 operations.

Providers extend Crossplane by providing schema definitions and a controller for cloud-provider APIs. For example, the AWS Provider can manage almost 1,000 separate AWS Managed Resources.

Compositions take individual Managed Resources and combine them into higher-level abstractions. An example is the EKS reference configuration, which defines a custom API and provisions IAM Roles, Nodepools, OIDC, and other AWS resources to create a fully-defined EKS cluster.

Because the real world can be complex, Composition Functions allow engineers to define resources using any programming language, like KCL or render state using text-templating engines. When rendering a Composition, Functions are run in a series of pipeline steps, with each step defining or modifying the desired state of the resources we want to create.

A Composition can contain dependencies, conditions, and loops and generate dozens or hundreds of resources, creating the risk of introducing errors and misconfiguration.

Composition developer must ensure:

• The Composition code is valid and error-free
• The desired state generated from the Composition matches expectations
• The desired resources generated from the Composition are valid

Ideally, we’d be able to validate the Composition during development and get feedback immediately. In the next sections we’ll take a look at crossplane render and crossplane validate, utilities that emerged from Crossplane’s Developer Experience special interest group. 

Rendering Functions with the Crossplane CLI

Crossplane CLI has the ability to render the output of Crossplane functions, allowing you to check and validate the desired state generated by the Composition before applying it to your Cluster.

In Crossplane 1.17, coming out in August 2024, render is graduating from beta. If you have an earlier version of the Crossplane CLI, use the crossplane beta render command for all of these examples.

Render requires three files: a resource to test, the Composition file, and a file that contains the function Docker images for crossplane render to run. We are going to test creating XTenant types:

Code for the basic example can be found here.

Using the command line, we can run:

Running this command gives us the output of the XTenant kind and all the Kubernetes Objects that provider-kubernetes uses to create the namespaces.

While this is a promising start, one problem is the status of the XTenant is not set, since we are only creating a desired state. In the next section we’ll discuss how to simulate existing resources.

More Complex Rendering Patterns

What if we want to make our test environment simulate the real world, where we have resources already provisioned?

Using the CLI, we can simulate the output of a live environment by providing more data to the render command.  Observed resources allow developers to add existing resources to the context of a function.

Code for observed resources is located here.

To create observed resources, one can save the YAML output of the resource into a text file.

It’s important when doing this to make sure each observed resource has the right crossplane.io/composition-resource-name annotation to match it to the resource being generated in our composition.

Now that we have stored resources in the observed directory. We update our render command to include the Observed resources:

Finally, when we run crossplane render, the status of the XTenant is now populated with the data from the observed resources.

Context

To share data that is not desired state, function pipelines can contain optional context. The most usual Composition pattern is a pipeline step reads the context.

Extra Resources were introduced in Crossplane 1.15, and are a powerful feature allowing a function to search for any other Crossplane resource on a Cluster and store the data in the Function’s context. This data can be any Crossplane object from Managed Resources, other Composites. Function-environment-configs looks up Crossplane EnvironmentConfigs and loads them into the pipeline’s context.

In order to create context in render, create a directory like extra-resources, and add theYAML manifests of the resources you are trying to simulate.

The example code is located here. 

Run the following command:

With –include-context, we can check the output to ensure that the Pipeline is selecting the correct resource. There is a dev and production EnvironmentConfig in the extra-resources directory. What happens in the Context when you change xr.yaml from dev to prod?  

You may be wondering what is the difference between Observed and Context: Observed data refers to resources that are in the composition as part of the desired state. Extra resources are defined outside of the composition.

Now that we can render our Composition with Resource and External data into our function, how do we validate that the resources being generated match the schemas of the resources? 

Validating Compositions

The Crossplane CLI comes with a validate command that is currently in beta status that takes a file or the output of render and checks that it matches the schema for the object.

Example code is located here.

Running the following command will render a Composition and then pass the output to validate. The validate command will look into the schemas directory for packages or YAML manifests to validate the output of render.

The validate command can extract Kubernetes schema definitions from the provider packages, in our example, our schema directory contains a configuration.yaml package that has dependencies: