Deploy Secure EKS Clusters and Databases with Upbound and Crossplane on AWS

In today's fast-paced tech environment, the ability to rapidly provision and manage cloud infrastructure is a competitive advantage. For organizations leveraging AWS, creating internal cloud platforms with a self-service API can significantly enhance productivity and agility. The Upbound Platform Reference for AWS is a great starting point for building these internal platforms, offering your development teams a robust and user-friendly way to manage AWS resources. By using this reference platform, you can streamline your cloud operations, empower your developers with self-service capabilities, and ensure consistent, scalable infrastructure management across your organization.

In this blog, we'll explore the architecture of the Upbound Platform Reference for AWS and how it can streamline your cloud operations.

The GitHub repository for the AWS Reference Platform contains a reference AWS Platform Configuration for Crossplane. It's a great starting point for building internal cloud platforms with AWS and offering a self-service API to your internal development teams.

This platform offers APIs for setting up fully configured EKS clusters with secure networking, stateful cloud services (RDS) that can securely connect to the EKS clusters, an Observability Stack, and a GitOps System. All these components are built using cloud service tools from the Official Upbound AWS Provider. App deployments can securely access the necessary infrastructure through secrets distributed directly to the app namespace.

Overview

This reference platform outlines a specialized API for generating an EKS cluster (XCluster) that incorporates XRs from the specified configurations:

• upbound-configuration-app
• upbound-configuration-aws-database
• upbound-configuration-aws-eks
• upbound-configuration-aws-network
• upbound-configuration-gitops-flux
• upbound-configuration-observability-oss

Learn more about Composite Resources in the Crossplane Docs.

Quickstart

Prerequisites

Before we can install the reference platform we should install the up CLI. This is a utility that makes following this quickstart guide easier. Everything described here can also be done in a declarative approach - which we highly recommend for any production type use-case.

To install up run this install script:

See up docs for more install options.

To intstall crossplane CLI follow these Crossplane docs.

We need a running Crossplane control plane to install our instance. We are using Universal Crossplane (UXP). Ensure that your kubectl context points to the correct Kubernetes cluster or create a new kind cluster:

Finally install UXP into the upbound-system namespace:

We will need Usages alpha feature for the correct deployment and eventual de-provisioning of this reference platform.

You can validate the install by inspecting all installed components:

Install the AWS Reference Platform

Now you can install this reference platform. It's packaged as a Crossplane configuration package so there is a single command to install it:

Validate the install by inspecting the provider and configuration packages:

After all Configurations are ready, you can check the status of associated Providers that were pulled as dependencies:

Check the marketplace for the latest version of this platform.

Configure the AWS provider

Before we can use the reference platform we need to configure it with AWS credentials:

See provider-aws docs for more detailed configuration options.

Using the AWS reference platform

🎉 Congratulations. You have just installed your first Crossplane-powered platform!

Application developers can now use the platform to request resources which then will be provisioned in AWS. This would usually be done by bundling a claim as part of the application deployment code, e.g. provisioning it together with the application helm chart. In our example here we simply create the claims directly:

Create a custom defined cluster:

Create a custom defined database:

NOTE: The database abstraction relies on the cluster claim to be ready - it uses the same network to have connectivity with the EKS cluster.

Alternatively, you can use a postgresql claim:

Now deploy the sample application:

Note: application has a strong dependency on mariadb type of the database.

You can verify the status by inspecting the claims, composites and managed resources:

To get nice representation of the Claim deployment status you can use the crossplane beta trace command:

To delete the provisioned resources you would simply delete the claims:

To uninstall the provider & platform configuration:

Customize for your Organization

So far we have used the existing reference platform but haven't made any changes. Let's change this and customize the platform by ensuring the EKS Cluster is deployed to Frankfurt (eu-central-1) and that clusters are limited to 10 nodes.

For the following examples we are using my-org and my-platform:

Pre-Requisites

First you need to create a free Upbound account to push your custom platform. Afterwards you can log in.

Make the changes

To make your changes clone this repository:

You can make arbitrary changes like extending the Composition or changing your custom platform API XRD. For example, you can change your XRD and Composition to serve your company's api group instead of upbound.io . Modify the composition and definition files according to this diff:
 

Now, you can create Configuration package that will encapsulate all your custom modifications.

Build and push your platform

To share your new platform you need to build and distribute this package.

To build the package use the up xpkg build command:

Afterwards you can push it to the marketplace. It will be not automatically listed but the OCI repository will be publicly accessible.

Using your custom platform

Now to use your custom platform, you can pull the Configuration package from your repository:

For the alternative declarative installation approach see the example Configuration manifest. Please update to your org, platform and tag before applying.

🎉 Congratulations. You have just built and installed your first custom Crossplane-powered platform!

Set of encapsulated best practices

You might notice that the reference platform contains a set of highly opinionated practices that we believe are extremely useful in Crossplane Configuration management and maintenance.

Let’s review them one by one.

Configuration of Configurations pattern

Historically we maintained the reference platform as a single monorepo that contained all associated Compositions and XRDs in the same repository.

While getting more experience with Crossplane configuration development and maintenance we came up with the pattern that we can name as Configuration of Configurations.

The approach is pretty straightforward: we split the platform into a distinct set of Crossplane Configurations and collect them into a single higher-level configuration that we call ‘platform’.
The best illustration of this approach is the dependency list of the main `crossplane.yaml`  of the reference platform.

As you can see, it mainly consists of other fine-grained configurations: