Ingress to services in a vcluster on Kubernetes
To use the ngrok Ingress Controller for Kubernetes with vcluster in a local cluster:
The ngrok Ingress Controller for Kubernetes is the official controller for adding public and secure ingress traffic to your k8s services. This open source Ingress Controller works with any cloud, locally-hosted, or on-premises Kubernetes cluster to provide ingress to your applications, APIs, or other services while also offloading network ingress and middleware execution to ngrok's platform.
vcluster is an open source project for creating virtual clusters that run inside regular namespaces, which provides strong isolation and easy access for multiple tenants with low cost and overhead. The pods you deploy on a vcluster are scheduled inside of the underlying cluster, while other resources, like deployments and CRDs, exist only inside the virtual cluster.
Together, the ngrok Ingress Controller and vcluster work to provide secure and load-balanced ingress for services running on a virtual cluster, which lets you isolate development environments, create an internal developer platform (IDP) in cloud native environments, and run experiments or simulations virtually while properly routing external traffic.
With this guide, you'll use an existing Kubernetes cluster, or set up a local development cluster with minikube, to launch a virtual cluster, and deploy a demo application. You'll then deploy the ngrok Ingress Controller to connect your demo application to the ngrok platform to route traffic to your vcluster.
- An ngrok account.
- The vcluster CLI installed locally.
- kubectl installed locally.
- Helm 3.0.0+ installed locally.
- An existing remote or local Kubernetes cluster OR minikube to create a new demo cluster locally.
Step 1: Set up a local virtual cluster with vcluster
To follow along with this guide, you need a remote or local Kubernetes cluster with vcluster installed. If you have an existing cluster with vcluster set up, you can skip this step and proceed to Step 2: Install the ngrok Ingress Controller.
If you don't have a cluster already, create one locally with minikube and install vcluster.
-
Create a local Kubernetes cluster with minikube.
minikube start --profile dc1 --memory 4096
-
Use the
minikube
CLI to ensure your new local cluster is running properly.kubectl get namespaces
NAME STATUS AGE
default Active 5m55s
kube-node-lease Active 5m55s
kube-public Active 5m55s
kube-system Active 5m55s -
Create a new vcluster with the name
my-vcluster
, which creates a new namespace calledvcluster-my-cluster
and automatically switches the active kube context to use your new vcluster.vcluster create my-vcluster --expose-local
-
To ensure your new local cluster is running properly get the namespaces for your instance. Your list of namespaces in the
my-vcluster
context should look something like this.kubectl get namespaces
NAME STATUS AGE
default Active 19s
kube-system Active 19s
kube-public Active 19s
kube-node-lease Active 19sIf you are not connected to your new vcluster and its kube context, you can run
vcluster connect my-vcluster
to try again.You know have a vcluster installed on your local minikube cluster.
noteThese steps are partially based the guide Using the ngrok Ingress Controller to create Preview Environments with vcluster from Loft, the maintainers of vcluster.
Step 2: Install the ngrok Ingress Controller
Now that you have a Kubernetes cluster integrated with vcluster, you can install the ngrok Kubernetes Ingress Controller to provide ingress to any services you want to run on your virtual cluster.
-
Add the ngrok Helm repository if you haven't already.
helm repo add ngrok https://ngrok.github.io/kubernetes-ingress-controller
-
Set up the
AUTHTOKEN
andAPI_KEY
exports, which allows Helm to install the Ingress Controller using your ngrok credentials. Find yourAUTHTOKEN
under Your Authtoken in the ngrok dashboard.To create a new API key, navigate to the API section of the ngrok dashboard, click the New API Key button, change the description or owner, and click the Add API Key button. Copy the API key token shown in the modal window before closing it, as the ngrok dashboard will not show you the token again.
export NGROK_AUTHTOKEN=[YOUR-AUTHTOKEN]
export NGROK_API_KEY=[YOUR-API-KEY] -
Install the ngrok Ingress Controller with Helm under a new
ngrok-ingress-controller
namespace.helm install ngrok-ingress-controller ngrok/kubernetes-ingress-controller \
--namespace ngrok-ingress-controller \
--create-namespace \
--set credentials.apiKey=$NGROK_API_KEY \
--set credentials.authtoken=$NGROK_AUTHTOKEN -
Verify you have installed the ngrok Ingress Controller successfully and that pods are healthy.
kubectl get pods -l 'app.kubernetes.io/name=kubernetes-ingress-controller' --namespace ngrok-ingress-controller
NAME READY STATUS RESTARTS AGE
ngrok-ingress-controller-kubernetes-ingress-controller-man2fg5p 1/1 Running 0 2m23s
Step 3: Install a sample application
At this point, you have a functional vcluster with the ngrok Ingress Controller running and authenticated with your ngrok credentials. To demonstrate how the Ingress Controller simplifies routing external traffic to your primary cluster, virtual cluster, and ultimately an exposed service or endpoint, you can install a sample application.
-
Create a ngrok static subdomain for ingress if you don't have one already. Navigate to the Domains section of the ngrok dashboard and click Create Domain or New Domain. This static subdomain will be your
NGROK_DOMAIN
for the remainder of this guide.By creating a subdomain on the ngrok network, you provide a public route to accept HTTP, HTTPS, and TLS traffic.
-
Create a new Kubernetes manifest (
2048.yaml
) with the below contents. This manifest defines the 2048 application service and deployment, then configures the ngrok Ingress Controller to connect thegame-2048
service to the ngrok edge via yourNGROK_DOMAIN
.tipMake sure you edit line 45 of the manifest below, which contains the
NGROK_DOMAIN
variable, with the ngrok subdomain you created in the previous step. It should look something likeone-two-three.ngrok.app
.apiVersion: v1
kind: Service
metadata:
name: game-2048
namespace: ngrok-ingress-controller
spec:
ports:
- name: http
port: 80
targetPort: 80
selector:
app: game-2048
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: game-2048
namespace: ngrok-ingress-controller
spec:
replicas: 1
selector:
matchLabels:
app: game-2048
template:
metadata:
labels:
app: game-2048
spec:
containers:
- name: backend
image: alexwhen/docker-2048
ports:
- name: http
containerPort: 80
---
# ngrok Ingress Controller Configuration
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: game-2048-ingress
namespace: ngrok-ingress-controller
spec:
ingressClassName: ngrok
rules:
- host: NGROK_DOMAIN
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: game-2048
port:
number: 80 -
Apply the
2048.yaml
manifest to your vcluster.kubectl apply -f 2048.yaml
tipNote: If you get an error when applying the manifest, double check that you've updated the
NGROK_DOMAIN
value and try again. -
Confirm your vcluster successfully deployed your 2048 application by navigating to the Edges section in the ngrok dashboard.
An edge connects your 2048 application, running in your vcluster, to the rest of the world through the ngrok Ingress Controller. You should see a new edge configuration created by
kubernetes-ingress-controller
that matches the domain name you entered on L45 of your2048.yaml
manifest.Under the Backend section of your edge configuration, you can also see cluster details as annotations, like the namespace and connected service, to validate that your demo application is accessible to external traffic via the Ingress Controller.
Click on your edge configuration to see additional details and options for advanced ingress needs, like creating multiple tunnels for load balancing, enabling the Mutual TLS module, adding compression, and more. We'll use one of these options, OAuth, in the next step.
-
Access your 2048 demo app by navigating to the your domain, e.g.
https://one-two-three.ngrok.app
. ngrok's edge and your Ingress Controller will route traffic to your app from any device or external network as long as your vcluster remains operational.
Step 4: Add OAuth protection to your demo app
Let's take your ingress needs a little further by assuming you want to add edge security, in the form of Google OAuth, to the endpoint where your 2048 application is humming along.
By default, ngrok manages OAuth protection entirely at the cloud edge, which means you don't need to add any additional services to your cluster, or alter routes, to ensure ngrok's edge authenticates and authorizes all requests before allowing ingress and access to your endpoint.
-
Edit your existing
2048.yaml
manifest with the following configuration.tip- See L42-43 for the additional annotation that this OAuth setup requires.
- See L58-69 for the newly added configuration for ngrok's OAuth module, replacing
acme.com
orngrok.com
with the domain name for your email address. You can also configure the OAuth module to authenticate individual email addresses.
apiVersion: v1
kind: Service
metadata:
name: game-2048
namespace: ngrok-ingress-controller
spec:
ports:
- name: http
port: 80
targetPort: 80
selector:
app: game-2048
---
# Configuration for the 2048 application, service, and deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: game-2048
namespace: ngrok-ingress-controller
spec:
replicas: 1
selector:
matchLabels:
app: game-2048
template:
metadata:
labels:
app: game-2048
spec:
containers:
- name: backend
image: alexwhen/docker-2048
ports:
- name: http
containerPort: 80
---
# Configuration for ngrok's Kubernetes Ingress Controller
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: game-2048-ingress
namespace: ngrok-ingress-controller
annotations:
k8s.ngrok.com/modules: oauth
spec:
ingressClassName: ngrok
rules:
- host: NGROK_DOMAIN
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: game-2048
port:
number: 80
---
# Configuration for ngrok's OAuth authentication module
kind: NgrokModuleSet
apiVersion: ingress.k8s.ngrok.com/v1alpha1
metadata:
name: oauth
namespace: ngrok-ingress-controller
modules:
oauth:
google:
emailDomains:
- acme.com
- ngrok.com -
Re-apply your
2048.yaml
configuration.kubectl apply -f 2048.yaml
-
Visit the Edges section of the ngrok dashboard, click on the edge for your deployment, then click OAuth to confirm the module is using the configuration you added to your
2048.yaml
manifest. -
Access your 2048 app at your domain (e.g.
https://one-two-three.ngrok.app
) to verify that it requests authentication via Google OAuth before allowing you to access the 2048 app.
What's next?
You've now used the open source ngrok Ingress Controller for Kubernetes to add secure access to your endpoint without worrying about IPs, network interfaces, or VPC routing. Because ngrok offloads ingress and middleware execution to its global edge, you can follow the same procedure listed above for any Kubernetes environment, like EKS, GKE, and OpenShift, with similar results.
If you want to clean up the work you did for this demo application, the easiest way (and the advantage of virtual
clusters in the first place) is to disconnect from your vcluster and then delete it with the vcluster
CLI. That will remove the namespace and all its resources, returning your primary cluster to its initial state.
vcluster disconnect
vcluster delete my-vcluster
You can also extend your new vcluster and ngrok Ingress Controller with additional modules, a fanout configuration to route traffic to multiple services from a single domain or IP address, or even multiple Ingress Controller installations running alongside one another.
Learn more about the ngrok Ingress Controller, or contribute, by checking out the GitHub repository and the project-specific documentation.