Crossplane in a Box: Your Toolkit for Fast Prototyping and Testing

This article will be helpful for anyone interested in setting up a local Crossplane dev/test environment in a reproducible and easy way. Source code for this blog is available in a companion repository.

There are several reasons why local Crossplane environment is useful:

• fast prototyping of the infrastructure APIs development
• testing infrastructure changes and configuration
• testing new Crossplane versions
• testing new Crossplane CLI versions

Before diving any further let’s see how the dev/test environment would look like:

There are 3 phases that we would like to automate as much as possible:

1. Infrastructure setup
2. Iterative development and testing (for example developing compositions or composition functions)
3. Optionally collaborate with others

This setup saves a lot of time, and because it’s fairly reproducible, it can be shared with team members and used as a standard template. Let's learn how to create it.

Choosing local Kubernetes cluster

First things first, in order to run Crossplane we need a Kubernetes cluster.

There are a lot of choices as it comes to running a local Kubernetes instance, but we are going to focus on KIND.

KIND is very easy to maintain and fast to set up. Second best option is k3s.

• Minikube
• MicroK8s
• → Kind Kubernetes IN Docker
• Vagrant and VirtualBox
• Docker Desktop
• Rancher Desktop
• k3s also with k3d wrapper

Declarative Approach All the Way?

Before we dive into the setup, let's talk about two approaches: declarative versus imperative. Both approaches makes sense depending on the circumstances. Declarative approach provides more stability and predictability, while imperative approach is suitable for quick prototyping and provides more control over the setup.

Whenever it comes to something vs something else, it's really about when it makes more sense to use something and when something else.
 

Given this approach, we will use both approaches depending on the requirements and what we want to achieve.

Orchestrating commands

Setting up a local or test Crossplane environment means orchestrating a bunch of commands, including scripts, YAML files, Helm charts, etc.

In the imperative paradigm, this is typically done via a Makefile or bash scripts. The problem with make is that it is designed as a tool to build C source code, it can run commands but that's not its purpose. This means that when using Makefile we take on the whole unnecessary baggage of the build part.

Separate bash scripts are a bit better but after a while they became too verbose and hard to maintain.

There is a tool that combines best of both worlds; just is similar to make, but focused on commands orchestration.

Declarative approach is still our friend

Justfile contains all imperative logic needed to quickly create and destroy our local cluster. It exposes various knobs for us to interact with it.

Installing a Helm chart, operator or simple YAML file can be done declaratively using a GitOps process. This can be accomplished with Flux or ArgoCD. For a quick, local setup ArgoCD is a bit more user friendly due to its robust web client. Here we are utilizing app of apps pattern to bootstrap additional apps from a single source. This article describes the pattern very well.

We have just scratched the surface of ArgoCD or GitOps. You can read more about GitOps in here and here.
 

Setup

Follow along with the companion repo.

The setup is tested on macOS and Linux. It should work on Windows with WSL2 as well.
 

Prerequisites

• To learn more about setting up reproducible shell environments, watch Viktor’s video about using devbox to do just that. You can use it or setup the tools the old-fashion way.

Other than kind, we need a few additional CLI tools:

just command runner

For macOS users, you can use Homebrew to install just:

For Linux users, refer to the just repository for installation instructions.

envsubst

There are several ways of templating YAML. We can wrap it in a Helm chart, use ytt, jsonnet, yq, kustomize or many others. Those are all valid approaches, but for local environment, there is a simpler method.
 

We will use envsubst instead. It is a part of the GNU gettext utilities and should be already installed on your system. This tool enables us to patch environment variables on the fly.
 

On macOS, you can install gettext (which includes envsubst) using Homebrew:

On Linux, envsubst is usually included with the gettext package. Use your distribution's package manager to install it:

For Debian-based distributions (e.g., Ubuntu):

For RHEL-based distributions (e.g., CentOS):

kubectl

Follow the official Kubernetes documentation to install kubectl for your operating system.

With all the prerequisites installed, we can now proceed to setting up our local Crossplane environment.

How To

Here are simple steps to get started:

1. Fork the repository https://github.com/Piotr1215/crossplane-box and clone it into your local environment
2. Export variable with your GitHub user name export GITHUB_USER=your-github-username
3. Run just setup to create prerequisites

Running the last command will do the following:

• replace my GitHub user name with yours in the bootstrap.yaml and apps/application_crossplane_resources.yaml
• create a kind cluster named control-planewith ArgoCD and Crossplane installed
• copy ArgoCD server secret to clipboard and launch default browser with ArgoCD login page

username: admin password: should be in your clipboard so just paste it in the password text box. In case this didn't work, you can run kubectl -n argocd get secret argocd-initial-admin-secret -o jsonpath="{.data.password}" | base64 -d to get the password.
 

ℹ️ Since this setup is meant to be quick and easy, we are not setting up a HTTPS certificate for the web UI and in order to proceed click “Advanced” → “Proceed to localhost (unsafe)”.. If you’re interested in a more secure setup with HTTPS for local development, you might explore using mkcert. It requires a bit more work, as everyone needs to run mkcert on their machine to create and trust their certificates

Typing just<TAB> will show all available just recipes. Here is the list:

Now we ready to start iterating quickly on our local Crossplane setup, by adding new content to the yaml directory and pushing changes to the repository. It's also possible to create a new ArgoCD app in the app folder and point it to a different repository or sub-folder.

Install more content

ArgoCD's bootstrap app observes the apps directory for any changes once deployed.

Notice that the apps directory already contains an app that points to the yaml folder with Crossplane functions and providers. Adding to this directory and pushing the changes to the repository will automatically deploy the new content to the cluster.

When you add new content to the yaml directory, you can deploy it to the cluster by running the following command:

Destroy the cluster