Skip to main content

Configure Site-to-Site Connectivity for Databases with mTLS

This guide provides step-by-step instructions for using ngrok for site-to-site connectivity. This example shows you how to securely run the ngrok agent at an external site to get access to a locally running database. The connection will be end-to-end encrypted using mutual TLS (mTLS). You'll use stunnel at the client site to intercept and encrypt requests from your local database client before they are transmitted to the ngrok agent running at the external site.

For a deeper understanding for how mTLS is implemented within ngrok, reference the Mutual TLS module page.

Prerequisites​

A certificate authority (CA) is required for mTLS. The CA is responsible for issuing and digitally signing certificates (client certificates). The CA will also be used to verify the authenticity of the certificates.

  1. CA certificate to be used by ngrok to verify the clients.
  2. Client certificates signed by the CA used to access the endpoints.

Users are responsible for providing the CA and client certificates. ngrok will not generate them. The CA certificate will be uploaded and hosted on the ngrok platform. The client certificates will need to be distributed to any client/device that will need to access the API endpoints.

Most organizations will have their own certificate mangement infrastructure. You can generate test certificates for reference/demo purposes during implementation.

Install the ngrok agent​

Download the appropriate version and install it on the same subnet as the APIs you want to access.

Get an ngrok API Key​

Create an ngrok API key using the ngrok dashboard. Make sure you save the API key before you leave the screen because it won't be displayed again.

Configure a custom agent ingress address​

Configuring a custom agent ingress address allows you to provide your customers with a dedicated URL to connect to the ngrok platform. Since your customers will connect using your subdomain, they can safely block other ngrok domains to control the tunnels started in their network. You'll provide a subdomain you own, such as connect.{YOUR_DOMAIN}, and delegate DNS (Domain Name Service) control of that subdomain to ngrok.

Create the agent ingress address​

Use the ngrok API to create the custom agent address by running the command below, substituting your own values for the variables:

curl \
-X POST \
-H "Authorization: Bearer {API_KEY}" \
-H "Content-Type: application/json" \
-H "Ngrok-Version: 2" \
-d '{"description":"{DESCRIPTION}","domain": “connect.{YOUR_DOMAIN}”}' \
https://api.ngrok.com/agent_ingresses

You should receive a 201 response similar to the following:

{
	"id": "agin_2esRkfoq4frGvOVUmfhytlr2zS3",
	"uri": "/agent_ingresses/agin_2esRkfoq4frGvOVUmfhytlr2zS3",
	"description": "Custom ingress address for site-to-site connectivity",
	"domain": "connect.configurable-domain.com",
	"ns_targets": [
		"ns-1329.awsdns-38.org",
		"ns-737.awsdns-28.net",
		"ns-1940.awsdns-50.co.uk",
		"ns-427.awsdns-53.com"
	],
	"region_domains": [
		"tunnel.us.connect.configurable-domain.com",
		"tunnel.us-cal-1.connect.configurable-domain.com",
		"tunnel.eu.connect.configurable-domain.com",
		"tunnel.au.connect.configurable-domain.com",
		"tunnel.ap.connect.configurable-domain.com",
		"tunnel.jp.connect.configurable-domain.com",
		"tunnel.sa.connect.configurable-domain.com",
		"tunnel.in.connect.configurable-domain.com"
	],
	"created_at": "2024-04-09T19:37:23Z",
	"certificate_management_policy": null,
	"certificate_management_status": null
}

Save the values from the ns_targets property and the region_domains property as you'll use them later.

Update your DNS​

Create an NS record in your DNS provider's registry for each ns_targets value from the response above, using connect as the name for each entry. The screenshot is from AWS Route53, but you can use any DNS provider you choose.

alt-text

You should have four new records when you’re done.

alt-text

You can run the following command to get the values you need if you didn't save the response.

-X GET \
-H "Authorization: Bearer {API_KEY}" \
-H "Ngrok-Version: 2" \
https://api.ngrok.com/agent_ingresses

Create a custom wildcard domain​

Next, create a custom wildcard domain, which will allow you to create endpoints and receive traffic on any subdomain of your domain.

For example, you might create *.customer1.{YOUR_DOMAIN}. You would then be able to create endpoints on app.customer1.{YOUR_DOMAIN} and dev.customer1.{YOUR_DOMAIN}. It can be helpful to create a separate subdomain for each customer site you wish to connect to.

Run the following command, substituting your API key for {API_KEY} and your domain for {YOUR_DOMAIN}:

curl \
-X POST \
-H "Authorization: Bearer {API_KEY}" \
-H "Content-Type: application/json" \
-H "Ngrok-Version: 2" \
-d '{"domain":"*.customer1.{YOUR_DOMAIN}","region":"us"}' \
https://api.ngrok.com/reserved_domains

You should receive a 201 response similar to the following:

{
	"id": "rd_2euR3S7Mt07gRC6NlVGcXtghElK",
	"uri": "https://api.ngrok.com/reserved_domains/rd_2euR3S7Mt07gRC6NlVGcXtghElK",
	"created_at": "2024-04-10T12:31:16Z",
	"domain": "*.customer1.configurable-domain.com",
	"region": "",
	"cname_target": "hn9tzsiu5lnlbhcl.4wsgv9l5wfesschhy.ngrok-cname.com",
	"http_endpoint_configuration": null,
	"https_endpoint_configuration": null,
	"certificate": null,
	"certificate_management_policy": {
		"authority": "letsencrypt",
		"private_key_type": "ecdsa"
	},
	"certificate_management_status": {
		"renews_at": null,
		"provisioning_job": {
			"error_code": null,
			"msg": "Managed certificate provisioning in progress.",
			"started_at": "2024-04-10T12:31:16Z",
			"retries_at": null
		}
	},
	"acme_challenge_cname_target": "mpgy8nkj.acme-challenge.ngrok-dns.com"
}

You'll need values from the response in the next section.

Update DNS records​

Next, add two CNAME records to your DNS provider's registry, providing values from the previous response.

First add a CNAME record for the wildcard subdomain you just created.

alt-text

Then add another record for the ACME (Automated Certificate Management Environment) validation. For the record, enter acme_challenge_cname_target, and use the value of the
acme_challenge_cname_target property from from previous response for the value.

Verify DNS​

Use the ngrok dashboard to verify that you’ve configured DNS correctly.

  1. Login to the ngrok dashboard.
  2. Click Domains in the left-hand navigation menu.
  3. Click on your wildcard domain under Domain.
  4. Click 2 targets next to DNS Targets in the panel displayed on the right-hand side of the screen, as denoted by the arrow in the screenshot below.

alt-text

  1. Click Check Status, as denoted by the arrow in the screenshot below.

alt-text

  1. You should see a successful response similar to the screenshot below.

alt-text

Create a bot user​

Now, you’ll create a bot user so that in the next section, you can create an agent authtoken independent of any user account. Run the following command to create a new bot user, providing your API key and a description:

curl \
-X POST \
-H "Authorization: Bearer {API_KEY}" \
-H "Content-Type: application/json" \
-H "Ngrok-Version: 2" \
-d '{DESCRIPTION}' \
https://api.ngrok.com/bot_users

You should receive a 201 response similar to the following:

{
	"id": "bot_2fmwUXhnImKswMgZKCsx7cnbt2l",
	"uri": "https://api.ngrok.com/bot_users/bot_2fmwUXhnImKswMgZKCsx7cnbt2l",
	"name": "new bot user from API",
	"active": true,
	"created_at": "2024-04-29T19:39:36Z"
}

Create the agent authtoken​

You should start each agent using a separate authtoken, and that token should belong to a bot user.

To create an authtoken, login to the ngrok dashboard and click Authtokens under Tunnels, then click Add a Tunnel Authtoken. For Owner, select the bot user you use created, and select bind:*.customer1.{YOUR_DOMAIN} for the ACL Rules. This ACL will allow an agent with the authtoken to create tunnels on any subdomain of customer1.{YOUR_DOMAIN}.

alt-text

Configure the ngrok agent API​

You can use the ngrok agent API to start and stop tunnels remotely, collect and replay captured requests, and collect stats and metrics.

The API runs on localhost:4040 wherever the agent runs, but you can start an ngrok tunnel to make the API available remotely.

Please note that tunnels started with the agent API do not persist in the event of an agent restart or network outage.

Update ngrok.yml​

Create a tunnel definition in the ngrok.yml config file. You can then run the ngrok agent and start and stop tunnels as needed using the ngrok agent API.

You'll need to modify the ngrok.yml config file before running the agent. You can modify the file that resides in the default location:

  • Linux: ~/.config/ngrok/ngrok.yml
  • MacOS (Darwin): ~/Library/Application Support/ngrok/ngrok.yml
  • Windows: %HOMEPATH%\AppData\Local\ngrok\ngrok.yml

Rather than edit the default ngrok.yml config file, you can provide your customer with a separate config file. This keeps your config separate from their existing ngrok.yml config file. You can then pass it as an argument when you run the ngrok agent. ngrok merges all config files, and you won't need to modify your customer's config file. This is useful when your customer already uses ngrok and you wish to use the existing agent to serve the ngrok agent API.

Refer to the region_domain values you saved when you created the custom agent ingress address. These are custom addresses for ngrok’s global Points of Presence (POPs). Support for Global Server Load Balancing (GSLB) is not yet available when using a custom agent ingress address, so you’ll need to specify the POP this agent should use to connect to the ngrok platform.

Add the following sections to the ngrok.yml config file, substituting the appropriate values for your environment, including a subdomain to connect to the agent api, such as agent.customer1.configurable-domain.com:

#config.yml

version: "2"
authtoken: { YOUR_AUTHTOKEN_WITH_ACL }
server_addr: { VALUE_FROM_REGION_DOMAINS }

tunnels:
  agent-api:
    proto: http
    domain: agent.customer1.{ YOUR_DOMAIN }
    addr: 4040
    inspect: false

Start the agent to access the agent API​

Now that you’ve updated the ngrok.yml config file, start the tunnel for the agent API by running the following command wherever you installed the agent:

ngrok start agent-api

Note that “agent-api” is the name given to the tunnel in the ngrok.yml example config file. You can use any descriptive name for this tunnel. In this case, you’re starting a tunnel to serve the ngrok agent API on https://agent.customer1.{YOUR_DOMAIN}.

If you wish to provide your customer with a custom ngrok.yml config file rather than modifying the one in the default location, as mentioned previously, you can pass the path to the file when you start the tunnel:

ngrok start agent-api –config /path/to/ngrok.yml

You can pass multiple paths to config files, and ngrok will merge them before starting.

Start tunnels in your customer’s network​

You now have the ngrok agent running in your customer's environment and can use the agent API to start and stop the tunnels necessary to access your customer’s APIs.

To start a tunnel to connect to your customer’s API, run the following command, substituting the appropriate values for your environment:

curl -H "Content-Type: application/json" \
-X POST https://agent.{YOUR_DOMAIN}/api/tunnels \
-d '{"name":"passthrough","proto":"tls","terminate_at": "agent", "addr":"{PORT}", "domain":"db.customer1.{YOUR_DOMAIN}"

The example below creates a tunnel on port 3306 and serves it at db.customer1.configurable-domain.com. Because a previous step created the wildcard domain on *.customer1.configurable-domain.com, a tunnel can be started on db.customer1.configurable-domain.com without reserving that subdomain.

curl -H "Content-Type: application/json" \
-X POST https://agent.customer1.configurable-domain.com/api/tunnels \
-d '{"name":"passthrough","proto":"tls","terminate_at": "agent", "addr":"3306", "domain":"db.customer1.configurable-domain.com",}'

Add mutual TLS (mTLS)​

At this point, you can access APIs running at the external site, but the connection is unencrypted. Next you'll add mutual TLS (mTLS) authentication to the tunnel to allow ngrok to verify the identity of the client and server.

When you start the tunnel, you can send additional properties in the payload to provide the certificate authority (CA) private key, the signed server certificate, and the server’s private key.

If you previously started a tunnel, stop it now. Then, to start a fully encrypted tunnel with mTLS, run the following command, replacing the variables with the appropriate values for your environment:

curl \
-H "Content-Type: application/json" -X  \
POST https://agent.customer1.{YOUR_DOMAIN}/api/tunnels \
-d '{"name":"passthrough","proto":"tls","terminate_at": "agent", "addr":"{PORT}", "domain":"db.customer1.{YOUR_DOMAIN}","mutual_tls_cas":"{PATH_TO_CA_PRIVATE_KEY}", "crt":"{PATH_TO_SIGNED_SERVER_CERT}", "key":"{PATH_TO_SERVER_PRIVATE_KEY}"

Generate test certificates​

The script below generates tests certificates, which may be useful during implementation or proof of concept. You should provide certificates signed by an official Certificate Authority when running tunnels in production.

To generate certificates for testing, run the script below, substituting your wildcard domain for {YOUR_WILDCARD_DOMAIN}. For the example above, you'd use *.customer1.configurable-domain.com.

#!/bin/sh

# Configure the common names for the CA, Server, and Client
CA_COMMON_NAME="{YOUR_WILDCARD_DOMAIN}"
SERVER_COMMON_NAME="{YOUR_WILDCARD_DOMAIN}"
CLIENT_COMMON_NAME="{YOUR_WILDCARD_DOMAIN}"

# Generate a Certificate Authority (CA)
echo "Generating keys for Certificate Authority (CA)..."
echo "You will be prompted for a pass phrase 4 times. Enter the same password each time. "
openssl req -new -x509 -days 9999 -keyout ca-key.pem -out ca-crt.pem -subj "/CN=$CA_COMMON_NAME"

# Generate a Server Certificate
echo "Generating a key for the server..."
openssl genrsa -out server-key.pem 4096

echo "Generating a Server Certificate Signing Request..."
openssl req -new -key server-key.pem -out server-csr.pem -subj "/CN=$SERVER_COMMON_NAME"

echo "Signing the server certificate with the CA..."
echo "You will be prompted for a pass phrase 4 times. Enter the same password each time. "
openssl x509 -req -days 9999 -in server-csr.pem -CA ca-crt.pem -CAkey ca-key.pem -CAcreateserial -out server-crt.pem

echo "Verifying the server certificate..."
openssl verify -CAfile ca-crt.pem server-crt.pem

# Generate a Client Certificate
echo "Generating a key for the client..."
openssl genrsa -out client-key.pem 4096

echo "Generating a Client Certificate Signing Request..."
openssl req -new -key client-key.pem -out client-csr.pem -subj "/CN=$CLIENT_COMMON_NAME"

echo "Signing the client certificate with the CA..."
echo "You will be prompted for a pass phrase 4 times. Enter the same password each time. "
openssl x509 -req -days 9999 -in client-csr.pem -CA ca-crt.pem -CAkey ca-key.pem -CAcreateserial -out client-crt.pem

echo "Verifying the client certificate..."
openssl verify -CAfile ca-crt.pem client-crt.pem

echo "mTLS certificate generation complete."

When the script completes, you should have the following new files:

  • ca-crt.pem
  • ca-crt.srl
  • ca-key.pem
  • client-crt.pem
  • client-csr.pem
  • client-key.pem
  • server-crt.pem
  • server-csr.pem
  • server-key.pem

Some of these are intermediary files you can ignore.

Run the command below from the same directory where you generated the certificate files to start a fully encrypted tunnel, substituting your domain for {YOUR_DOMAIN} and supplying a value for {PORT}:

curl \
-H "Content-Type: application/json" -X  \
POST https://agent.customer1.{YOUR_DOMAIN}/api/tunnels \
-d '{"name":"passthrough","proto":"tls","terminate_at": "agent", "addr":"{PORT}", "domain":"db.customer1.{YOUR_DOMAIN}","mutual_tls_cas":"ca-crt.pem", "crt":"client-crt.pem", "key":"server-key.pem"'

Please note that these certificates should be used for testing purposes only. You should obtain certificates signed by a Certificate Authority for use in production.

Add stunnel​

Now you're ready to make requests from your local database client, such as mysql, to the database running in the external site. You have a secure connection that is TLS-encrypted, but your local database client only knows how to transmit unencrypted TCP requests. This is where stunnel comes into the picture.

You'll run stunnel on the same system as the database client. Then stunnel will intercept all requests from the database client and wrap them in TLS encryption before sending them to the database via the TLS endpoint you've created. When the request reaches the ngrok service, the service forwards the request to the agent running in the external site, which then decrypts the request and forwards it on to the backend database.

Install stunnel​

First, install stunnel using your favorite package manager, like homebrew.

Configure stunnel​

Next, create an stunnel.conf configuration file with the contents below, substituting your domain for {YOUR_DOMAIN} and adding valid paths to the client certificate and key and the path to the certificate authority key.

; Service-level options
[mysql-tls]
client = yes
accept = 127.0.0.1:8080
connect = {YOUR_DOMAIN}:443
sni = {YOUR_DOMAIN}
cert = /path/to/client-crt.pem
key = /path/to/client-key.pem
CAfile = /path/to/ca.crt
verify = 0

Start stunnel​

Finally, start stunnel by runnning the command stunnel from the same directory as the stunnel.conf file.

Connect to the database​

You can now make end-to-end encrypted requests to the database at the external site using the database client running on the same system as stunnel.