Since the release of AWS Lambda in 2014, serverless applications have gained wide adoption. But it can be a challenge to configure and manage the mix of events, code, backends, permissions, and other dependencies that make up a serverless application.

With powerful Crossplane primitives like Managed Resources and Composition we can manage these serverless deployments and provide internal development teams with custom Kubernetes-based API.

This blog will cover two topics: Using Crossplane to provision AWS Lambda functions, and invoking those functions in order to return data to Crossplane so that we can use it to provision other Resources.

The Challenge

The platform engineering team supports developers in many global regions. Each team has a primary and a backup region for their cloud infrastructure.

The proposed solution is to have an AWS Parameter Store (SSM) to store common parameters and have Crossplane provision infrastructure based on each team’s Primary and Backup regions.

Our Lambda Architecture

To start, we’ll review the infrastructure architecture. Our application is made up of several Crossplane Composite Resources. To make Composite Resource available and functional in your custom API need to configure two components:

• An API defined by the platform team called the CompositeResourceDefinition or XRD.
• One or more Compositions that implements the API defined by the XRD by provisioning Managed Resources like S3 Buckets or GKE Clusters.
  For the platform we are building, we have defined a number of XRDs and Compositions.
  
  One of the ways to deploy AWS Lambda functions is by storing them in an S3 Bucket as a .zip file, so the XBucket Composition creates an S3 bucket and associated configuration settings like Encryption and BucketVersioning. This bucket will be used to store the packaged Lambda functions.
  
  The XParameter Composition provides an API for storing values into AWS SSM, and the XFunction composition takes care of provisioning the Lambda function and associated IAM roles (for example, we have to allow our Lambda function to fetch parameters from the SSM Parameter store.

Storing Parameters in AWS SSM

First, let’s look at the XParameter Composition. As part of the Composition, the platform team can define a namespace-scoped Claim called Parameter, which allows teams to request Crossplane infrastructure into a Kubernetes namespace.

Our Lambda function will use the value stored in these Parameters to tell Crossplane where to install the user’s infrastructure.

In our example team-1 and team-2 use parameters to store their Primary and Backup cloud regions.

To create a new SSM Parameter, a team submits a standard Kubernetes YAML manifest to the Cluster to deploy using our Parameter Kind. In this example, team-1 configures their backupRegion to be us-west-1:

On the cluster each team has their Parameters scoped to their namespace:

Creating our Lambda Function

We’ll use Crossplane to provision and manage our Lambda function, using a Composition called XFunctionDeploy. This Composition creates an S3 bucket and uploads our Lambda function’s python code into an Object in the created bucket. BucketVersioning is set so that any new code uploaded to the S3 bucket triggers an update of the Lambda function.

Since compositions can be nested, we use our existing XBucket and XFunction Compositions. In this way we’re creating a reusable library of components.

Giving Our Lambda Function the Right Permissions

Our Lambda function needs to be able to connect to the SSM parameter store in order to query it. In our XFunction Composition create a Role with the following Policy and also attach the AWSLambdaBasicExecutionRole managed policy to this Role.

The Lambda Function

To make things easy to understand, we’ve created a simple Lambda function in Python that uses the boto library to fetch SSM parameters. This function is packaged into a Zip archive and uploaded to the S3 bucket using a BucketObject.

Invoking Our Lambda Function

To trigger our Lambda function, we need to send it a JSON payload with the parameter we are interested in fetching:

In Crossplane we can use an Invocation to trigger the function to run and collect the output. How does the Invocation know which Lambda function to call? One method is to use a Selector to match the Lambda function’s Kubernetes Object on the cluster:

When provisioned by Crossplane, the Invocation will return the output of the Lambda in its status:

Now that we have results, we can use the outputs in other Compositions.

Putting It All Together

Finally we can define a sample Application that will be created in a Composition called XApplication. The Composition consists of two Invocations that call the lambda function and store the output in the compositions status field. Those values are then passed to a VPC Managed resource to create VPCs in both the primary and backup regions.

The results of each function invocation is applied to the XApplication’s status field:

Now that Crossplane has created our VPCs, we can use kubectl’s support of JSONpath to extract the region for each VPC:

Deploying the Platform

This is the first in a series of blog posts highlighting how Crossplane and AWS Lambda work together. This next section will describe how you can install and deploy this configuration into your Kubernetes and AWS environments.

The code and examples can be cloned from https://github.com/upbound/platform-ref-lambda, with installation instructions located in the README file. You’ll need a Kubernetes cluster and AWS credentials to deploy the platform

Final Thoughts

We hope this is a starting point for integrating AWS Lambda with Crossplane. If you’re interested in learning more or have questions, join the Crossplane Slack and ask us there or open an issue on the https://github.com/upbound/platform-ref-lambda repository.