User Agent Filter

Overview

The User Agent Filter module enables you to block bots, crawlers, or certain browsers from accessing your web application. It allows or denies traffic based on the User-Agent header of incoming HTTP requests.

The User-Agent header contains information about the requesting browser or application, including its name, version, and operating system.

You define a set of regular expression rules that either allow or deny HTTP requests if they match the User-Agent header.

Example Usage

Agent CLI

ngrok http 80 \
  --ua-filter-allow="(foo/(\d)+.(\d)+)","(bar/(\d)+.(\d)+)" \
  --ua-filter-deny="(buz/(\d)+.(\d)+)"

Agent Config

tunnels:
  example:
    proto: http
    addr: 80
    user_agent_filter:
      allow: ["(foo/(\d)+.(\d)+)", "(bar/(\d)+.(\d)+)"]
      deny: ["(buzz/(\d)+.(\d)+)"]

SSH

ssh -R 443:localhost:80 v2@connect.ngrok-agent.com http \
  --ua-filter-allow="(foo/(\d)+.(\d)+)","(bar/(\d)+.(\d)+)" \
  --ua-filter-deny="(buz/(\d)+.(\d)+)"

Go

import (
	"context"
	"net"

	"golang.ngrok.com/ngrok"
	"golang.ngrok.com/ngrok/config"
)

func ngrokListener(ctx context.Context) (net.Listener, error) {
        return ngrok.Listen(ctx,
                config.HTTPEndpoint(
                        config.WithAllowUserAgent(`(foo)/(\d)+.(\d)+`),
                        config.WithDenyUserAgent(`(bar)/(\d)+.(\d)+`),
                ),
                ngrok.WithAuthtokenFromEnv(),
        )
}

Go Package Docs:

• https://pkg.go.dev/golang.ngrok.com/ngrok/config#WithAllowUserAgent
• https://pkg.go.dev/golang.ngrok.com/ngrok/config#WithDenyUserAgent

Javascript

const ngrok = require("@ngrok/ngrok");

(async function () {
	const listener = await ngrok.forward({
		addr: 8080,
		authtoken_from_env: true,
		allow_user_agent: "foo/(d)+",
		deny_user_agent: "bar/(d)+",
	});

	console.log(`Ingress established at: ${listener.url()}`);
})();

Javascript SDK Docs:

• https://ngrok.github.io/ngrok-javascript/interfaces/Config.html#allow_user_agent
• https://docs.rs/ngrok/latest/ngrok/config/struct.HttpTunnelBuilder.html#method.deny_ua
• https://ngrok.github.io/ngrok-javascript/classes/HttpListenerBuilder.html#allowUserAgent
• https://ngrok.github.io/ngrok-javascript/classes/HttpListenerBuilder.html#denyUserAgent

Python

import ngrok

listener = ngrok.forward("localhost:8080", authtoken_from_env=True,
    allow_user_agent="foo/(\d)+",
    deny_user_agent="bar/(\d)+")

print(f"Ingress established at: {listener.url()}");

Python SDK Docs:

• https://ngrok.github.io/ngrok-python/http_listener_builder.html#ngrok.HttpListenerBuilder.allow_user_agent
• https://ngrok.github.io/ngrok-python/http_listener_builder.html#ngrok.HttpListenerBuilder.deny_user_agent
• https://ngrok.github.io/ngrok-python/index.html#full-configuration

Rust

use ngrok::prelude::*;

async fn listen_ngrok() -> anyhow::Result<impl Tunnel> {
    let sess = ngrok::Session::builder()
        .authtoken_from_env()
        .connect()
        .await?;

    let tun = sess
        .http_endpoint()
        .allow_user_agent(r"foo/(\d)+")
        .allow_user_agent(r"bar/(\d)+")
        .deny_user_agent(r"buz/(\d)+")
        .listen()
        .await?;

    println!("Listening on URL: {:?}", tun.url());

    Ok(tun)
}

Rust Crate Docs:

• https://docs.rs/ngrok/latest/ngrok/config/struct.HttpTunnelBuilder.html#method.allow_ua
• https://docs.rs/ngrok/latest/ngrok/config/struct.HttpTunnelBuilder.html#method.deny_ua

Kubernetes Controller

note

The User Agent Filter module is not yet supported with the Kubernetes Ingress Controller.

Behavior

Rule Evaluation

The User Agent Filter module will check each HTTP request's User-Agent header value against the list of defined allow and deny regular expression rules.

• Requests without the User-Agent header will be denied by default. To allow these requests, you can add an allow rule for empty strings: ^$
• If a request has multiple User-Agent headers, only the first header will be checked.

Match Behavior

Requests are allowed if they match any allow rule or fail to match all deny rules.

You can override deny rules by defining an allow rule that is more specific, for instance blocking all Google bots except Google Images:

{
	"allow": ["(?i)google-images"]
	"deny": ["(?i)google", "[Bb]ing"]
}

Reference

Configuration

Agent Configuration

Parameter	Description
User Agent Filter Allow	A set of regular expressions used to match User-Agents that will be allowed.
User Agent Filter Deny	A set of regular expressions used to match User-Agents that will be denied.

Upstream Headers

This module does not add any upstream headers.

Errors

Code	HTTP Status	Error
ERR_NGROK_3211	403	The server does not authorize requests from your user-agent

Events

When the User Agent Filter module is enabled, it populates the following fields in the http_request_complete.v0 event:

Fields
user_agent_filter.decision

Pricing

This module is available on all plans.

Try it out

Run ngrok with User Agent Filter's allow and deny set to the following:

ngrok http 80 \
  --domain your-domain.ngrok.app \
  --ua-filter-allow="(GoingMerry/(\d)+.(\d)+)","(GomuGomu/(\d)+.(\d)+)" \
  --ua-filter-deny="(Xebec/(\d)+.(\d)+)"

Then make requests to your ngrok domain with the following:

curl --location https://your-domain.ngrok.app -H 'Content-Type: text/plain' -A 'GoingMerry/1.1' --data 'https://www.youtube.com/watch?v=djyTG19Achg' -k -v
curl --location https://your-domain.ngrok.app -H 'Content-Type: text/plain' -A 'GomuGomu/1.1' --data 'https://www.youtube.com/watch?v=djyTG19Achg' -k -v
curl --location https://your-domain.ngrok.app -H 'Content-Type: text/plain' -A 'Xebec/1.1' --data 'https://www.youtube.com/watch?v=djyTG19Achg' -k -v
curl --location https://your-domain.ngrok.app -H 'Content-Type: text/plain' -A '' --data 'https://www.youtube.com/watch?v=djyTG19Achg' -k -v
curl --location https://your-domain.ngrok.app -H 'Content-Type: text/plain' -A 'TwitterBot/1.1' --data 'https://www.youtube.com/watch?v=djyTG19Achg' -k -v