Tutorial: Redirect, Rewrite, and Mirror HTTP with Cilium Gateway API
HyperText Transfer Protocol (HTTP) might have been created in 1989, but it has certainly withstood the test of time. It laid the foundation for the World Wide Web, and it remains ever-popular and the protocol of choice for the vast majority of APIs (despite the more recent alternatives like GraphQL and gRPC). For the past few decades, we’ve used Layer 7-aware load balancers to control, alter, and route HTTP traffic as it entered our network. In this blog post, we are going to explore how you can alter HTTP traffic (redirect, rewrite, and mirror) as it enters Kubernetes clusters with Cilium’s Gateway API, focusing on three specific use cases:
- HTTP Redirect – to tell the client that the resource they are trying to access has moved
- HTTP Path Rewrite – to rewrite the URL or entire path used by the client
- HTTP Mirroring – to copy the traffic sent from the client to another backend
In the world of Kubernetes, the Gateway API’s role is to modify and control traffic as it enters our clusters.
In the previous “Getting Started with the Cilium Gateway API tutorial,” we explored how you can use Cilium’s Gateway API support to load-balance HTTP traffic and alter HTTP header requests and responses.
We’ve also recorded several short tutorials on the Gateway API, including the one below on “TLS Passthrough,” where traffic can be encrypted to the server.
In this tutorial, we will explore these use cases. If you’d rather watch me present it, check out this episode of the eBPF and Cilium weekly show eCHO:
If you’d rather do it yourself, follow the steps below. Let’s start!
Demo environment
Before we start with the first use case—redirecting HTTP traffic—let’s prepare our environment. This tutorial will use an Azure Kubernetes Service (AKS) cluster in BYOCNI mode and an Elastic Kubernetes Service (EKS) cluster in ENI mode. In this mode, a cluster is deployed without a Container Network Interface (CNI) so that we can bring our own Cilium in this instance.
Create an AKS cluster with BYOCNI and fetch the credentials once the cluster is up and running.
Truncated O/P:
===========
Note that your nodes won't show as Ready for now - there's no CNI, so connectivity is limited.
Truncated O/P:
===========
Let’s use the latest Gateway API Custom Resource Definitions (CRDs). Remember to install them before installing Cilium:
Support for the features we are exploring today – and for the 1.0 Gateway API version – was recently added to the main branch of Cilium – to use them, you will need Cilium 1.15.
Use this Cilium CLI command (install the CLI with the instructions here if you’ve not done so already) for both the EKS and AKS clusters:
Expect this output:
Remember that the kubeProxyReplacement feature (KPR) is required for the Cilium Gateway API. Note that, even when creating the cluster in BYOCNI mode, kubeProxy was deployed in the cluster.
You can use this new option instead of skipping its deployment, as it is no longer needed once Cilium is deployed in KPR mode.
After installation, check the Cilium status with:
Expect the status to be:
The status for both Cilium and Operator should be OK. Now that our environment is ready, let’s go ahead and deploy our demo app and our Gateway before deploying the HTTPRoutes to illustrate the use cases we will cover in this blog post.
We will use this YAML manifest I have put on a GitHub repo.
This configuration is a subset of configs used in the Gateway API conformance tests. Remember that one of the benefits of the Gateway API project is consistency. To ensure that each Gateway API provides a consistent user experience, each implementation is tested against a set of conformance tests that creates a series of Gateways and Routes and tests that the implementation matches the API specification.
Deploy this demo app:
Expect this output:
We are re-using the same Gateway configuration we used in the previous blog post. It’s a simple Gateway configuration (it’s simply listening for HTTP traffic):
Expect this output:
Let’s double-check that the Gateway has been installed and received an IP address. Remember that the IP address allocation is done automatically for you in public clouds. In contrast, you would need to use MetalLB or Cilium’s own LoadBalancer IP Address Management feature on a private cloud to assign this IP.
Expect an outcome such as:
Let’s save this IP as the $GATEWAY variable.
HTTP Redirect
One common use case for ingress gateways is sending HTTP redirects to clients to tell them that the resource they are trying to access in the cluster has moved to a different location.
This is a common requirement for migration, content optimization, SSL/TLS enforcement – the list goes on.
Redirects return HTTP 3XX responses to a client, instructing it to retrieve a different resource. With the Gateway API, we can use redirect filters to substitute various URL components independently, as shown below.
Let’s use this HTTPRoute YAML for the four examples in this section. We will review each rule in detail below.
Path Redirect
In this first example, we will do a simple redirect: we only replace a portion of the URL and redirect them there:
You should see the IP address allocated to the Gateway (such as 20.115.194.177).
The following rule will match traffic to /original-prefix and redirect the client to a different URL:
Let’s try using curl.
Notice we use -l in the curl request to follow the redirects (by default, curl will not follow redirects), and we use the verbose option of curl to see the response headers.
The location is used in Redirect messages to tell the client where to go. As you can see, the client is redirected to http://20.115.194.177:80/replacement-prefix. The prefix was replaced with /original-prefix to /replacement-prefix. Note that we support different models that replace the URL. We can replace just a portion of the path or the entire one.
Host & Path redirects
You can also redirect the client to a different host.
Let’s try, with this specification, to redirect the client to example.org:
Let’s make HTTP requests to that external address and path:
Expect an output such as:
As you can see, the client is redirected to http://example.org:80/replacement-prefix.
Both the hostname and the path prefix were modified.
Redirect to new prefix and custom status code
Next, you can also modify the status code. By default, as you saw above, the redirect status code is 302. It means that the resources have been moved temporarily.
To indicate that the resources the client is trying to access have moved permanently, you can use the status code 301. You can also combine it with the prefix replacement.
Let’s use this example:
Try to access this URL:
Expect an output such as:
Note the status code returned is 301 Moved Permanently and the client is redirected to http://20.115.194.177:80/replacement-prefix.
Redirect to new prefix and custom status code
Finally, we can use the Gateway API to impose tighter security controls. You can redirect the client to use HTTPS instead of HTTP, changing the scheme used by the client.
Look at the last line in this specification:
Let’s try it.
Expect an output such as:
As you can see, the client initially tried to connect via HTTP and is redirected to https://example.org:443/scheme-and-host:
HTTP Rewrite
Sometimes, we don’t need redirecting the client to a different URL. Instead, we can rewrite components of a client request.
Let’s explore this with a couple of examples. Create the HTTPRoute by applying the manifest below:
The Gateway will replace the /prefix/one in the URL request to /one.
Let’s now check that traffic based on the URL path is proxied and altered by the Gateway API:
The request is received by an echo server that copies the original request and sends the reply back in the packet’s body.
As you can see, the Gateway changed the original request from /prefix/one to /one.
As we use the Envoy proxy for L7 traffic processing, note that Envoy also adds information about the original path in the packet (see "X-Envoy-Original-Path").
We can also combine this with previous Gateway API features we explored in the previous tutorial. You might want to rewrite traffic and add some headers to it to add some metadata (so that the receiving server can interpret it accordingly).
If you look at the HTTPRoute we’ve just created, you will see that traffic to /rewrite-path-and-modify-headers will not only be partially rewritten, but some headers can be added, removed, or modified.
Expect an output such as:
HTTP Mirroring
In this final example, we will review how to mirror traffic. It has plenty of use cases – monitoring incoming traffic for forensics, analysis, logging, troubleshooting, etc.
With the following rule, we can mirror traffic meant for infra-backend-v1 to infra-backend-v2.
Apply it:
The Gateway will forward the traffic infra-backend-v1 as standard but copy it across to infra-backend-v2 (purple arrow in the image below). The response infra-backend-v1 will be processed normally by the Gateway, but the responses infra-backend-v2 will be ignored (red arrow).
Let’s request the /mirror path.
Look at the output below. Traffic was sent to the infra-backend-v1 Service. Was it also mirrored to infra-backend-v2 ? How can we prove it?
The image used by the echo Pods serving this Service is minimal. Instead of trying to install tcpdump Let’s use the Kubernetes debug functionality instead, with an image (nicolaka/netshoot) designed to troubleshoot networking issues.
The command below will deploy a container in the same Pod as the infra-backend-v2 Pod and will see the traffic coming in:
Once you run the curl command again from a different terminal; you should see traffic appearing (you may have to run it on a few occasions as traffic will be load-balanced between multiple infra-backend-v2 Pods):
This shows that the Gateway mirrored traffic to the infra-backend-v2 Service.
Post Demo Clean-Up
After testing, you can clean up your demo environment and remove both the cluster and the resource group with the following commands:
Delete the AKS Cluster:
Delete the EKS Cluster:
Recap
Cilium might be famously known for being a high-performance CNI and its network policy engine, but it is equally capable of performing L7 load-balancing. Unlike my days as a network engineer, when I would have to drive to a data center, lug a load balancer onto the rack, install it, and configure it, with Cilium, I don’t have to install anything else—it’s just another feature to enable and a YAML manifest to apply.
In this blog post, you will have learned—and hopefully tried alongside me—how Cilium Gateway API can send HTTP redirects, rewrite URL paths, and mirror HTTP traffic. This is another powerful addition to all the other Layer 7 Load Balancing features that Cilium Gateway API already supports.
Do you think this might be useful for your applications? Got any use case in mind? Let me know on Cilium Slack, or you can find me on LinkedIn.
Learn More
Prior to joining Isovalent, Nico worked in many different roles—operations and support, design and architecture, and technical pre-sales—at companies such as HashiCorp, VMware, and Cisco.
In his current role, Nico focuses primarily on creating content to make networking a more approachable field and regularly speaks at events like KubeCon, VMworld, and Cisco Live.
Amit Gupta is a senior technical marketing engineer at Isovalent, powering eBPF cloud-native networking and security. Amit has 21+ years of experience in Networking, Telecommunications, Cloud, Security, and Open-Source. He has previously worked with Motorola, Juniper, Avi Networks (acquired by VMware), and Prosimo. He is keen to learn and try out new technologies that aid in solving day-to-day problems for operators and customers.
He has worked in the Indian start-up ecosystem for a long time and helps new folks in that area outside of work. Amit is an avid runner and cyclist and also spends considerable time helping kids in orphanages.
