In this post I’m going to show you how to build a multi-region Node.js Lambda API using Serverless Framework and pair it with a serverless multi-region CockroachDB database (full disclosure: I work for Cockroach Labs).

Both CockroachDB Serverless and AWS Lambda operate on a pay-as-you-go basis, making this pairing a cost-effective solution for serving up data to end users, fast — no matter where in the world they may be.

To show you what I mean, here’s the API I built for this post.

• 🚀 API Response Preview: api.mr-paulie.net
• ⚙️GitHub Repository: https://github.com/PaulieScanlon/multi-region-node-lambda-api

Multi-Region API Responses

If you visit the API Response preview link above, you’ll see one of the three screenshots below. Depending on where in the world you are will determine the value of the region.

This screenshot shows what you’ll see if you’re located within Europe.

My VPN is set to the United Kingdom.

This screenshot shows what you’ll see if you’re within Asia.

My VPN is set to Taiwan.

And finally, this screenshot shows what you’ll see if you’re outside of Europe or Asia and acts as the default response.

My VPN is set to the United States.

In each of the screenshots, you’ll notice the region is different. The first part of this blog post deals with how to route requests to your API through an appropriate region using an AWS Route 53 Hosted Zone. The second half deals with configuring CockroachDB to use a regional-by-row topology pattern.

The Anatomy of a Multi-Region API

There are several pieces to this puzzle and I’ll explain each of them in detail. They are as follows:

1. Registered domain name
2. SSL Certificate created in Certificate Manager
3. DNS configuration in Route 53 Hosted Zone
4. API Gateway
5. Node.js Lambda function

Before You Start: API

To deploy a multi-region API to AWS you’ll first need a domain name. This URL will act as the gateway for all requests. Where these requests are routed will be handled by DNS configuration set up using an AWS Route 53 Hosted Zone.

Register a Domain Name

If you don’t already have a domain name, buy one now. You can do this from the AWS console or another service.

If you buy the domain name using AWS, the Name Servers (ns) will be automatically added to the DNS for you. If you do this using another service, you’ll need to add the AWS Name Servers yourself.

Once your domain has been successfully registered in AWS, you should see DNS settings similar to the below:

If the registration is taking a little while to complete, you can circle back to it later and carry on with the next step.

Create a New GitHub Repository

It’s completely up to you if you do this first, or later. Personally, I always like to start with an empty GitHub repository and fill out the default setup options.

• Add a README
• Add a default Node .gitignore
• Add an MIT license

With the empty GitHub repository created, clone it to your local development environment; change directory so you’re in the correct location on disk, then run the following.

This will create a default package.json. (Again, you don’t need to do this, but it’s a good pattern to start a project with sensible consistent defaults).

How to Build a Multi-Region API Using Serverless

The Serverless Framework is mainly to ease the pain of deploying to AWS. Whilst it’s possible to write Lambda functions directly in the AWS console, in practice, you really don’t want to do that. For starters, you’ll likely need version control. Plus, I’d imagine you’d rather write code in your preferred code editor and not a “browser version” of a code editor.

In the case of a multi-region API, using Serverless will allow you to write the code once and then deploy it to multiple regions, rather than having to manually do this yourself using the AWS console.

There are two versions of AWS API Gateway. For this post, I’ll be using v2. You can read more about this in the Serverless Docs: HTTP API (API Gateway v2).

Installing Serverless

To use Serverless you’ll need to have it globally installed.

The Serverless CLI can be used to automatically set up some of the following, but personally, I don’t find it helpful.

serverless.yml

Create a new file at the root of your project and name it serverless.yml. Add the following code. (You might want to change the service name to the name of your project)

Most of this setup should be self-explanatory, but I would like to draw specific attention to a couple of things:

1. useDotEnv:true: To deploy the function to my AWS account I store my AWS credentials in a .env file at the root of my project.
2. environment.DATABASE_URL: As above. The connection string to the CockroachDB database will be stored in a .env file. (We will set up the CockroachDB serverless cluster after a few more steps).
3. cors:true: Without cors set to true you’ll likely experience CORS errors when you attempt to make requests to your API because "Access-Control-Allow-Origin": "*" won’t be present in the headers. Setting this under the provider section applies the settings to all functions.

Default Function

Create a new directory at the root of your project and name it v1. Inside this directory, create a new file called api.js and add the following code:

This function doesn’t really do anything, but it will be the default entry point to your API. I’ve added it to demonstrate the bare minimum required for a Lambda function and to show how to access the AWS_REGION environment variable available. The resulting value of this environment variable will be determined by the region to which it’s deployed.

Environment Variables

I’ve included an .env.example file in the repository. Rename this to .env and add your AWS credentials:

You can find your AWS credentials from the AWS console by visiting the IAM profile section. (There’s more information in the AWS Docs about how to set this up if you haven’t already: Getting Started with IAM).

Multi-Region Lambda Deployment

You can deploy your functions to AWS using the command line, which is fine for single regions. For multi-region deployments, however, I’ve found it easier to add a script to package.json that handles the deployment for multiple regions using a single command.

Add the following to package.json:

The top three scripts use the serverless deploy command and the --region flag to define the region the function should be deployed. For my API I’m deploying to us-east-1, eu-central-1 and ap-southeast-1.

The fourth script, deploy, runs the first three scripts one after the other and can be invoked from your terminal using the following:

If your deployments are successful you should see something similar to the screenshot below in the AWS Lambda section of the AWS Console:

This screenshot is for N. Virginia (us-east-1). Depending on which region you’ve deployed will determine which region you should select from the dropdown.

If I were to select eu-central-1 or ap-southeast-1 instead, I’d see a Lambda function with the same name.

Multi-Region AWS Configuration

In order to route traffic for requests made to the API endpoint from specific regions to a Lambda deployed in the same region, you’ll need to configure a couple of other AWS services. These are:

• SSL Certificate
• API Gateway with custom domain (one per region)
• Route 53 A Record (AWS’s DNS service, one per region)

Here’s how I configured each of the services named above:

SSL Certificates

In the AWS Console search for “Certificate Manager” and select “Request certificate.” For public-facing APIs, you’ll need to “Request a public certificate”.

You’ll need to do this for all regions. I don’t know why, because the SSL certificates are the same across all regions. However, if you don’t request an SSL for each region it won’t be available to the API Gateway when you add a custom domain.

1. For the FQDN (Fully Qualified Domain Name) enter the URL + www.
2. I also add a second FQDN using the wildcard prefix of “*”. Later in the post I’ll explain how I use the domain with a subdomain prefix of “api”. This is possible because the SSL certificate accepts any value in place of the wildcard “*”.
3. I also prefer to validate the SSL by selecting the DNS validation method.

1. Once you request the certificates they will appear as “pending” until validated.
2. As above.
3. By clicking “Create records in Route 53” AWS Will automatically add the CNAME records to your hosted zone.

If all is successful, you will see that two new CNAME records have been added to the DNS config in the Route 53 Hosted Zone: (1) and (2) from the screenshot below.

API Gateway Custom Domain

With the Lambdas deployed and the SSL Certificates validated, now it’s time to set up the API Gateway. The API Gateway is what actually routes traffic to the Lambda functions.

Search for “API Gateway” in the AWS Console

Enter a name for your custom name and click “create.” I added a prefix of “api” to the domain name I registered, e.g. api.mr-paulie.net.

Pay attention to which region you’re currently in. In my case, I have deployed a Lambda to us-east-1 so the API Gateway will also be set up in us-east-1.

You’ll need to do this for each region you’ve deployed your Lambda functions. The following steps will, in this case, apply to, us-east-1, eu-central-1 and ap-southeast-1 in accordance with my deploy scripts.

Custom Domain Configuration

1. Enter the name and include a prefix if you’re using one. (i’ve added “api”)
2. Select the SSL certificate created in the previous step.

Scroll down a little further and you’ll see a button that says “Create domain name”.

If everything worked, you should see something similar to the below. Once the domain name is successfully created, the next step is to configure the API Mappings.

Add API Mappings

API Mappings are required so that the API Gateway knows which Lambda function(s) to invoke when a request is made to the domain name.

Click on API Mappings and use the dropdown inputs to select the API (1) and Stage (2); as shown in the screenshot below.

When you’re done click save.

You’ll need to repeat this step in each region you’ve deployed. In my project I have the same settings for, us-east-1, eu-central-1 and ap-southeast-1.

Check all your dropdown boxes. If there are empty menus, it’ll be because you’ve missed a step.

Geographically Aware DNS A Record Type

Geographically Aware DNS A Record Type is actually the key to the entire API. By adding geographically aware A Records you’ll be able to route traffic to Lambda functions deployed in different regions. E.g. requests that originate in Europe will be routed to a Lambda also deployed in Europe. This will yield much faster response times for end users, because latency is reduced when the request has the fewest miles to travel.

Create Record

From the Hosted Zone DNS, click “Create record”.

Routing Policy

Select “Geolocation” for the routing policy.

Define Geolocation Record

Within the A Record settings you’ll be able to select the alias to the API Gateway and determine the onward journey for requests that originate in a number of AWS regions.

In the screenshot below, I’ve configured the A Record to route traffic that originates in Europe to the API Gateway deployed to eu-central-1 and have given it a name of “Europe load balancer”.

A Record DNS

Once the A Record has been created you should see it appear in the “Hosted Zone” DNS settings.

When you make requests to your API from a location within Europe, you’ll be directed though this A Record and on to the API Gateway and Lambda function that you defined in the Geolocation record. But what about requests originating from other regions?

In the screenshot below, I’ve configured the A Record to route traffic that originates in Asia to the API Gateway deployed to ap-southeast-1 and have given it a name of “Asia load balancer”.

And lastly, I’ve added one more A Record and set the location to “default”. This will route all requests from outside Europe or Asia through the API Gateway and Lambda deployed to us-east-1.

There are a number of permutations to choose from when configuring A Records, depending on where you perceive your users to be. This will likely determine which regions you create A Records for; and similarly, which regions you deploy your Lambda functions too. 

Before You Start: CockroachDB

I’ll be using ccloud CLI (a CockroachDB command line interface) to perform some of the configuration that makes multi-region possible. Go ahead and install that now before continuing: Get Started with the ccloud CLI.

Create CockroachDB Multi-Region Serverless Cluster

Here’s a short video from my colleague Rob Reid that will walk you through the process of creating a multi-region serverless cluster in Cockroach Cloud.

Following Rob’s explanation, here’s the cluster I’ve set up for this blog post.

It’s a Serverless Cluster that uses the AWS provider, and has x3 regions: eu-central-1, us-east-1 and ap-southeast-1. These should look familiar, since they are the same regions I used to deploy the Lambda functions. The primary region is set to us-east-1, as this is the default region from the API / DNS configuration.

CockroachDB Connection String

With the cluster created, you can now connect to it using the ccloud CLI. In Cockroach Cloud, you’ll see a button that says connect. Click it and you’ll see the below screen:

You can change the language option from the default to JavaScript/TypeScript and select node-postgres as the tool. When you’re ready, copy the DATABASE_URL.

You don’t need the “export” part from the code snippet, above. Add the connection string to the .env file you created earlier.

Connect to CockroachDB using ccloud CLI

To connect to your cluster, run the following in your terminal.

If the connection is successful, you should see something similar to the one below.

You can exit the CLI and close the connection at any time by typing exit.

Set up a Regional Table

Run the following in your terminal.

These are the default settings applied when the cluster was created. You can see the regions match the settings I used when I created the cluster.

However, these aren’t quite what’s needed for a multi-region database. To configure CockroachDB to be multi-region the database needs to be altered slightly.

Ensuring you’re still connected to the cluster, run the following in your terminal.

Now you can create a new table and configure it to be REGIONAL BY ROW.

If you run the following your terminal…

…you should see, under the locality heading, REGIONAL BY ROW. This confirms the database and table have been correctly configured for multi-region usage.

To see the columns for the table run the following:

Which should show you this:

The region is of particular importance and here’s why. When you post data to this database you’ll use the AWS_REGION environment variable to populate the region column of the table.

Here’s another video from Rob where he explains regional tables in a little more detail.

The next step is to create a new Lambda that will, when invoked, populate the table with a value for each of the columns in the above table.

Install serverless-postgres

There are many flavors of Postgres that can be used with Node.js Lambda functions: node-postgres, pg-promise and a few more. Each has their own “special” way of handling the Postgres connection. In this post I’ll be using serverless-pg.

Create a new dir at the root of your project and name it pg. Add a new file and name it index.js. Add the following to setup a pg client that can be used, and re-used by multiple Lambda functions.

Create a Lambda Function to POST

Now that you have a method to connect to the database, it’s time to use it.

Create a new file within the v1 directory and name it create.js, and then add the following code:

The query uses INSERT to add a new row to the data table. The values are a date, created when the Lambda is invoked, and a string literal of the AWS_REGION prefixed with “aws”.

Inserting data into the database using this string literal means CockroachDB knows what to with the data and which node from the multi-region database to store it in.

You’ll also need to define the new endpoint in serverless.yml.

You can now deploy the changes using the script you defined earlier.

Test the Create Function

To test the create function, run the following in your terminal.

You should see the following output:

To check the INSERT worked correctly, you can SELECT everything from the data table using the following:

Which should now show you a new row in the table.

You’ll notice the region is aws-eu-central-1. This is because I’m currently in the UK.

If I set my VPN location to the United States and run the POST curl again, followed by SELECT * FROM data;, I’d see a new row in the data table with the region of aws-us-east-1.

Similarly, If I set my VPN location to the Taiwan and run the POST curl again, followed by SELECT * FROM data;, I’d see a new row in the data table with the region of aws-ap-southeast-1.

This confirms that the data is being routed via the correct Lambda and is being added to the database using the AWS_REGION.

Create a Lambda Function to READ

To ensure super snappy reads, rather than using SELECT * from data;, you’ll want to create a Lambda that will use the AWS_REGION in the query — which means CockroachDB only attempts to query data for the region the request was made from.

Create a new file inside the v1 directory and name it read.js.

You’ll see from the above that instead of using SELECT * from data;, I’ve added a WHERE clause that uses the AWS_REGION plus an “aws” prefix.

Using a WHERE clause in this way defines from which regional rows the data should be queried.

You’ll once again need to add the new endpoint to serverless.yml and deploy the changes.

Test the Read Function

If you visit this endpoint in the browser, you’d only see data that was stored in the region where it was created.

For me, in the UK, querying a table that has the three rows I created earlier (one from Europe, one from Taiwan, and one from the United States), I’d only see a single row!

Here’s the endpoint from my API so you can see for yourself: https://api.mr-paulie.net/read.

This is because my request has been routed via the European API Gateway and the AWS_REGION variable will equal eu-central-1 — meaning, CockroachDB will only return data that was created using the eu-central-1 region variable, and only attempts a READ from the node located in Europe, which results in a super snappy response time.

Regional by Row

Hopefully, I’ve demonstrated the power of regional by row, and the ease with which CockroachDB can be configured to enable what I believe to be a superpower. If you have global users and are looking for ways to reduce latency, look no further!

Finished

That just about wraps things up, I know it’s been a long and winding road, but what you’ve effectively made here is an Enterprise-level global application, and that’s something to be very pleased about.

I used this same approach in a recent project for Cockroach Labs. I named the application Silo and I’ve been using it to demonstrate how Data Residency works. You can read more about that project on the Cockroach Labs blog here: The Art of Data Residency and Application Architecture.

If you have any questions about the methods described in this post please come and find me on Twitter: @PaulieScanlon, I’d be more than happy to talk about how you’re using multi-region application architecture in your own projects.