Unified Testing with Upbound: Smarter, Faster, More Efficient Crossplane Testing

UpTest was built on Chainsaw, adding specific features for testing providers and configurations. You can read more about it in our previous blog post.

While functional, Chainsaw/UpTest had limitations for composition testing. For example, it required live Kubernetes resources, limited support for conditional logic, no understanding of composition rendering, and was an entirely isolated workflow from the composition development workflow.

The New Unified Approach

Our new testing experience addresses these limitations with a comprehensive, integrated approach. Here's a detailed look at each component. First, install v0.38 (or greater) of the Up CLI:

1. Up Composition Render

Due to popular demand, we brought the crossplane render command into our Up CLI tooling. You can use the up composition render command to print your desired composed resources for review.

To use the up composition render command, you need to pass in a Composite Resource (XR) file alongside your composition. These XRs use the Composition template to create new managed resources. An XR uses the same parameters as your example claim, but specifies the XStorageBucket type and specifies the target cluster. This local test will ensure that the build, configuration, and orchestration for your composition runs as expected before you deploy it to a development control plane.

2. Composition Tests

The heart of the new experience is the CompositionTest API. The CompositionTest API is built on top of the up composition render command. Each test models a single composition controller loop, making testing more streamlined for reading and debugging. You can author your tests in the preferred language of your choice between KCL, Python or YAML. To get started, run the following commands to install the latest version of the Up CLI and generate your first composition test. 

A test file should have been generated in the newly created tests folder.
 

Now, let’s walk through a real-world example testing a network configuration. Inside your newly generated test file, you can paste in the test below.

While the render command was effective for straightforward use cases, it fell short in handling more complex scenarios. With this API, you will be able to dynamically manage dependencies and conditionals between resources, validate resource statuses and state transitions during testing and even support mock states. The test above points to our composition, XR, XRD, and asserts various values for all of the resources we want to be testing. This is a Unit Test framework for compositions, built straight into the Up CLI.

After authoring your tests, you can now run them all via the following commands. You should get an output of passed tests, failed tests, and warnings as well.
 

3. Enhanced End-to-End Testing (E2E) Testing

In addition to the CompositionTest API, the new E2ETest API provides a more robust way to test entire systems. No need to have UpTest as a separate workflow - everything can be done in one place.
 

The command above will generate an E2E test file for you. There, you can author your own E2E test. An example is provided for you below.

Similar to CompositionTests above, you can also write your E2E tests in your preferred language of choice. In the test above, it automatically generates a Control Plane with Crossplane installed, with a configurable TTL and a set timeout. In addition, you can pass in any additional resources and provide a test manifest for an XR that the test can wait for. Once the test ends, cleanup happens automatically - the control plane winds down and destroys all resources for you.

You can run your E2E using the same command above with the –e2e flag.
 

Development Control Plane dedicated to the E2E test spins up:

In due time, resources become available and healthy:

4. CI/CD Integration

The beauty of our new testing workflow is how seamlessly it integrates into your CI pipelines. We have provided a Github Action for you so that you can trigger composition tests or e2e tests whenever changes to your compositions are merged into your Github repositories.