The announcement of the retirement of Ingress NGINX at KubeCon NA 2025 sent ripples throughout the Cloud Native ecosystem. Originally developed as an example of how to implement the Ingress API, Ingress NGINX quickly became a popular choice due to its flexibility, features, and being platform agnostic. The retirement of Ingress NGINX leaves many users at a crossroads of what migration path to take.
I am one of the 50% of users that uses Ingress NGINX, and my use case is pretty straightforward. I needed a way to access the Argo CD UI. But, regardless of the use case, I am faced with what most end users are faced with. The two paths off of Ingress NGINX:
- Move to another Ingress Controller
- Adopt Gateway API
Since joining Isovalent, I’ve seen organizations turn to Isovalent Networking for Kubernetes as the need goes beyond just replacing a controller. They need a complete Networking platform for Kubernetes. And, it just so happens that Cilium (part of Isovalent Networking for Kubernetes) is a “batteries included” solution that comes with an Ingress controller.
In this blog I will go over how you can replace Ingress NGINX by going over the two paths individually. First, I will use the Cillium Ingress controller and then I will also go over the replacement using Gateway API. For the purposes of a demo, I will be using Argo CD as an example workload.
Kubernetes Environment
For this example, I am using KIND for my Kubernetes environment since it’s easy to get up and running as all you need is Docker on your system. It’s easy to spin up and tear down, and you can emulate production environments to easily test out scenarios.
For the purpose of the demo, I’m going to customize the KIND cluster to bind to ports 80 and 443 as I want to connect to the Argo CD UI via my browser. Here’s the configuration saved as a config.yaml file.
kind: Cluster
name: argocd-cilium
apiVersion: kind.x-k8s.io/v1alpha4
networking:
disableDefaultCNI: true
nodes:
- role: control-plane
extraPortMappings:
- containerPort: 80
hostPort: 80
listenAddress: 0.0.0.0
- containerPort: 443
hostPort: 443
listenAddress: 0.0.0.0
kubeadmConfigPatches:
- |
kind: InitConfiguration
nodeRegistration:
kubeletExtraArgs:
node-labels: "ingresshost=cilium"A few things to note about this configuration:
- I’ve used
disableDefaultCNIas KIND will deploy its own CNI, and I will be installing Cilium - The port mappings will map my systems 80 and 443 ports to the containers respective ports.
- Since KIND uses
kubeadm, I’ve usedkubeadmConfigPatchesto label my single node withingresshost=cilium. This ensures (as you’ll see later) that Pods run on this node in case I want to add other nodes later.
With that configuration in place, I can now create the KIND Kubernetes cluster on my system by running the following:
kind create cluster --config config.yamlOnce created, If I run kubectl get pods --all-namespaces I will see some pods are in the Pending state:
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system coredns-66bc5c9577-pq55r 0/1 Pending 0 2m31s
kube-system coredns-66bc5c9577-vm6qp 0/1 Pending 0 2m31s
kube-system etcd-argocd-cilium-control-plane 1/1 Running 0 2m38s
kube-system kube-apiserver-argocd-cilium-control-plane 1/1 Running 0 2m38s
kube-system kube-controller-manager-argocd-cilium-control-plane 1/1 Running 0 2m38s
kube-system kube-proxy-rrk9k 1/1 Running 0 2m31s
kube-system kube-scheduler-argocd-cilium-control-plane 1/1 Running 0 2m39s
local-path-storage local-path-provisioner-7b8c8ddbd6-8m494 0/1 Pending 0 2m31sThis is because there is no networking yet on the KIND cluster. I still need to install Cilium as the CNI provider.
Choosing Your Path: Cilium Ingress or Cilium Gateway API
As Dean said in his blog post, many organizations find themselves at a turning point with the retirement of Ingress NGINX. Moving to Cilium Ingress offers a familiar Ingress based experience with minimal changes. This path is ideal if you want a fast, low risk migration and already have applications and tooling built around the Ingress API. Cilium Ingress lets you keep your existing workflows while benefiting from Cilium’s performance, observability, and security, making it a practical choice for teams that want results quickly.
The other option is to adopt Cilium with Gateway API, which represents where Kubernetes networking is headed. Gateway API expands beyond basic north south traffic and provides a more expressive and structured model for managing traffic. This approach makes sense if you are designing new platforms, standardizing traffic management across teams, or preparing for more advanced use cases over time.
A Deep Dive into Cilium Gateway API
The Future of Ingress Traffic Routing
Both paths are valid, and in the next sections I will walk through each approach. You’ll see how to use Cilium Ingress as a straightforward replacement for Ingress NGINX, followed by an example using Gateway API. This should help you decide which option best fits your current needs and where you want to take your platform next.
Installing Cilium with Ingress
There are two primary ways of installing Cilium; using the cilium CLI (which is itself based on the Helm chart) or the Helm Chart itself directly. Since I will use Helm to install Argo CD, I will be using the Helm Chart for Cilium as well. For my system, I will be using the following values.yaml file.
kubeProxyReplacement: true
operator:
replicas: 1
ingressController:
enabled: true
loadbalancerMode: dedicated
hostNetwork:
enabled: true
nodes:
matchLabels:
ingresshost: "cilium"
envoy:
enabled: true
securityContext:
capabilities:
keepCapNetBindService: true
envoy:
- NET_ADMIN
- BPF
- NET_BIND_SERVICEThere are a few options that I’m using that I want to call out:
- Since I am running a single node, I set the operator
replicasto 1 - I am enabling Cilium’s Ingress controller in the
ingressControllersection, which includes binding to the host network. I’m also enabling the node selector to the label I used for the cluster - Since I’m using
hostNetwork, I need to add the required security context capabilities in order for Envoy to bind to ports 80 and 443
I’ve already had the Helm repo added, but if it isn’t for you; this can be done by running helm repo add cilium https://helm.cilium.io and the running the helm repo update command.
Using the values.yaml file I created earlier, I can install Cilium using the following:
helm install cilium --namespace kube-system cilium/cilium --values values.yamlYou can wait for the install by running cilium status --wait. Once finished I ran cilium status to ensure Cilium has been installed:
/¯¯\
/¯¯\__/¯¯\ Cilium: OK
\__/¯¯\__/ Operator: OK
/¯¯\__/¯¯\ Envoy DaemonSet: OK
\__/¯¯\__/ Hubble Relay: disabled
\__/ ClusterMesh: disabled
DaemonSet cilium Desired: 1, Ready: 1/1, Available: 1/1
DaemonSet cilium-envoy Desired: 1, Ready: 1/1, Available: 1/1
Deployment cilium-operator Desired: 1, Ready: 1/1, Available: 1/1
Containers: cilium Running: 1
cilium-envoy Running: 1
cilium-operator Running: 1
clustermesh-apiserver
hubble-relay
Cluster Pods: 3/3 managed by Cilium
Helm chart version: 1.18.4
Image versions cilium quay.io/cilium/cilium:v1.18.4@sha256:49d87af187eeeb9e9e3ec2bc6bd372261a0b5cb2d845659463ba7cc10fe9e45f: 1
cilium-envoy quay.io/cilium/cilium-envoy:v1.34.10-1762597008-ff7ae7d623be00078865cff1b0672cc5d9bfc6d5@sha256:1deb6709afcb5523579bf1abbc3255adf9e354565a88c4a9162c8d9cb1d77ab5: 1
cilium-operator quay.io/cilium/operator-generic:v1.18.4@sha256:1b22b9ff28affdf574378a70dade4ef835b00b080c2ee2418530809dd62c3012: 1Cilium Ingress Enabled Argo CD
To show you how to actually take advantage of Cilium Ingress, we are going to install Argo CD as an example app. Argo CD, being part of the larger Argo Project ecosystem, is installed using the official Argo Project Helm charts. I also recommend doing it this way since you can have the Helm Chart automatically configure the Ingress resource for you. The values.yaml file I’ll be using looks like this:
global:
domain: argocd.7f000001.nip.io
server:
ingress:
annotations:
'"ingress.cilium.io/tls-passthrough"': "enabled"
'"ingress.cilium.io/force-https"': "enabled"
'"ingress.cilium.io/host-listener-port"': "443"
enabled: true
hostname: argocd.7f000001.nip.io
ingressClassName: cilium
tls: trueThe main thing to note in this file is that I’m using the Helm chart to create the Ingress object for me. Which includes setting the hostname (Here I’m using a https://sslip.io/ hosted domain that just points back to my 127.0.01 address), choosing the cilium IngressClass, and setting the needed Cilium Ingress annotations.
ingress.cilium.io/tls-passthroughSetting this toenabledwill enable TLS Passthrough since, by default, Argo CD generates its own certificate.ingress.cilium.io/force-httpsSetting this to enabled will force connections to HTTPSingress.cilium.io/host-listener-portThis instructs the Envoy listener to listen on port 443 on the host network
I’ve already ran helm repo add argo https://argoproj.github.io/argo-helm and helm repo update so I can proceed with installation by running the following:
helm install argocd --namespace argocd \
--create-namespace argo/argo-cd --values values.yamlI can then verify that Argo CD has been installed by running the kubectl get pods -n argocd command which displayed the following:
NAME READY STATUS RESTARTS AGE
argocd-application-controller-0 1/1 Running 0 2m44s
argocd-applicationset-controller-c98846bd4-6f8l8 1/1 Running 0 2m44s
argocd-dex-server-79dff44586-tnwj7 1/1 Running 0 2m44s
argocd-notifications-controller-7656dc87cb-mvvrc 1/1 Running 0 2m44s
argocd-redis-58b579b4c9-j2ntd 1/1 Running 0 2m44s
argocd-repo-server-bbd474d9b-cbr7x 1/1 Running 0 2m44s
argocd-server-7d4d49dd47-mg7k7 1/1 Running 0 2m44sTaking a look at the Ingress object, the Helm chart created it with the right annotations, using kubectl get ing -n argocd argocd-server -o yaml | yq .metadata.annotations I see the annotations are set.
ingress.cilium.io/force-https: enabled
ingress.cilium.io/host-listener-port: "443"
ingress.cilium.io/tls-passthrough: enabled
meta.helm.sh/release-name: argocd
meta.helm.sh/release-namespace: argocdI can now visit my Argo CD UI by using the hostname I set (my case it’s https://argocd.7f000001.nip.io) using admin as the username and using the generated password extract by running:
kubectl get secret -n argocd argocd-initial-admin-secret \
-o jsonpath='{.data.password}' | base64 -dOnce logged in, I am greeted with the default page:
This is great seeing this work with little effort, and this is a good path for those who want to keep using Ingress, and adopt Gateway API at a later time. If you want the quickest path away from Ingress NGINX, Cilium Ingress gets you there.
Cilium with Gateway API
While Cilium Ingress is great, the Kubernetes world is clearly moving towards Gateway API. As mentioned above, Gateway API is a SIG-Network sub-project that builds on the idea of Ingress and expands it by also handling north-south traffic, east-west traffic, and even service mesh entry points using a reliable and flexible model.
Cilium supports all the core Gateway API resources plus most of the extended ones. You also get eBPF powered performance, deeper visibility, and tighter policy integration, which makes it a strong choice as you move into the Gateway API future.
Let’s test how it looks to install Cilium with Gateway API. Let’s delete the cluster in order to see how the steps differ.
kind delete cluster --name argocd-ciliumNow, I will recreate the cluster using the same config.yaml file as before.
kind create cluster --config config.yamlFollowing the Getting Started Guide, I installed the CRDs for Gateway API. Here I’m using --server-side since the CRDs are quite large.
kubectl apply --server-side \
-f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.1/standard-install.yamlI’ve updated the values.yaml file to configure the Cilium Helm chart to use Gateway API instead of Ingress. The configuration looks pretty much the same, except for the Gateway API section is now in place where the Ingress section was:
kubeProxyReplacement: true
operator:
replicas: 1
gatewayAPI: # This section replaces Ingress with Gateway API
enabled: true
hostNetwork:
enabled: true
nodes:
matchLabels:
ingresshost: "cilium"
envoy:
enabled: true
securityContext:
capabilities:
keepCapNetBindService: true
envoy:
- NET_ADMIN
- BPF
- NET_BIND_SERVICEInstalling Cilium is the same as before using Helm:
helm install cilium --namespace kube-system cilium/cilium --values values.yamlAfter running cilium status --wait, the cluster should be up and running and doing kubectl get pods -A shows the cluster is ready:
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system cilium-6wgpk 1/1 Running 0 91s
kube-system cilium-envoy-kl8k5 1/1 Running 0 91s
kube-system cilium-operator-649c888797-qbph4 1/1 Running 0 91s
kube-system coredns-66bc5c9577-l7mw7 1/1 Running 0 6m54s
kube-system coredns-66bc5c9577-qvhqz 1/1 Running 0 6m54s
kube-system etcd-argocd-cilium-control-plane 1/1 Running 0 7m2s
kube-system kube-apiserver-argocd-cilium-control-plane 1/1 Running 0 7m3s
kube-system kube-controller-manager-argocd-cilium-control-plane 1/1 Running 0 7m2s
kube-system kube-proxy-gd4jj 1/1 Running 0 6m54s
kube-system kube-scheduler-argocd-cilium-control-plane 1/1 Running 0 7m2s
local-path-storage local-path-provisioner-7b8c8ddbd6-sfmp9 1/1 Running 0 6m54sPart of installing Cilium with Gateway API support is that it comes with a GatewayClass already installed and configured. I can see this by running the kubectl get gatewayclasses command after install, which showed me:
NAME CONTROLLER ACCEPTED AGE
cilium io.cilium/gateway-controller True 22mA GatewayClass is a cluster level template that defines what kind of Gateways your infrastructure can create. Conceptually, it’s like a blueprint that tells Kubernetes how a Gateway should behave. Once the class exists, you can spin up actual Gateways based on it, which I’ll do in the next section.
Gateway API Enabled Argo CD
Installing Argo CD with Gateway API is a little bit more involved as Gateway API adoption is only starting to grow and other projects need some time to catch up. With Argo CD, more values are needed in the values.yaml file to account for the fact that I am using both HTTPRoute and GRPCRoute (which is needed for the argocd CLI).
The first step is to define a Gateway, which is a Kubernetes resource that actually configures the load balancing infrastructure. When one is created, the GatewayClass controller builds the underlying infrastructure defined by that class.
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: default-gateway
namespace: default
spec:
gatewayClassName: cilium
listeners:
- name: argocd
protocol: HTTPS
port: 443
hostname: "argocd.7f000001.nip.io"
tls:
mode: Terminate
certificateRefs:
- kind: Secret
name: argocd-server-tls
namespace: argocd
allowedRoutes:
namespaces:
from: Selector
selector:
matchLabels:
kubernetes.io/metadata.name: "argocd"A few things I want to call out
- The
specincludes theGatewayClassinstalled by Cilium to use - The listeners I am using, which is 443 and hostname that it should claim
- I’m also setting it up to use TLS termination of a certificate that I will add later.
- I also only am allowing claims to this
Gatewayto come from theargocdnamespace
To read more about the Gateway object, you can find it on the Gateway API documentation page.
Now that I have the Gateway setup, I can apply it to my cluster:
kubectl apply -f argocd-gateway.yamlNow it’s time to install Argo CD. I modified the values file to prepare to install Argo CD with Gateway API support:
configs:
params:
server.insecure: true # Updated for offloading SSL
global:
domain: argocd.7f000001.nip.io
server: # This updated section creates HTTPRoute and GRPCRoute
httproute:
enabled: true
parentRefs:
- name: default-gateway
namespace: default
sectionName: argocd
hostnames:
- argocd.7f000001.nip.io
rules:
- matches:
- path:
type: PathPrefix
value: /
grpcroute:
enabled: true
parentRefs:
- name: default-gateway
namespace: default
sectionName: argocd
hostnames:
- argocd.7f000001.nip.io
rules:
- matches:
- headers:
- name: Content-Type
type: RegularExpression
value: "^application/grpc.*$"A few things I want to go over here:
- I’m going to offload TLS and use my own TLS Certificate, so I disabled Argo CD generating its own by setting
configs.params.server\.insecuretotrue. - I am using the Argo CD Helm chart to configure the
HTTPRouteunder theserver.httproutesection. Defining also theGatewayI’ve created earlier. - Just like the
HTTPRoute, I’m configuring the Argo CD Helm chart to also create theGRPCRoutefor me as well.
More information about the Argo CD specific configurations for Gateway API can be found on the official documentation page for Argo CD.
Installing Argo CD with Helm is the same as before, but with an updated values.yaml file:
helm install argocd --namespace argocd --create-namespace \
argo/argo-cd --values values.yamlThe Helm Chart creates the HTTPRoute object as defined in the values file, which I can see by running the kubectl get httproutes -n argocd command
NAME HOSTNAMES AGE
argocd-server ["argocd.7f000001.nip.io"] 79sThe same goes for the GRPCRoute which I can see with the kubectl get grpcroutes -n argocd command:
NAME HOSTNAMES AGE
argocd-server-grpc ["argocd.7f000001.nip.io"] 2m20sOnce that is in place, I will load the TLS certificate I created for Argo CD, which the Gateway will use to offload TLS. For the purposes of this blog, I will use mkcert to create the certificate.
mkcert argocd.7f000001.nip.ioThen, I created the the Secret in the argocd Namespace:
kubectl create secret tls argocd-server-tls -n argocd \
--cert ./argocd.7f000001.nip.io.pem --key ./argocd.7f000001.nip.io-key.pemI also need to restart the Argo CD pods in order for Argo CD to use the TLS certificate I just created on the cluster:
kubectl rollout restart deployment -l app.kubernetes.io/instance=argocd -n argocdSince I am providing a secret in the argocd namespace that will be used in my Gateway that’s in the default namespace, I will need to set up a ReferenceGrant to allow access. A ReferenceGrant enables cross namespace references for things like HTTPRoutes and, in my case, Secrets.
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata:
name: argocd-reference-grant
namespace: argocd
spec:
from:
- group: gateway.networking.k8s.io
kind: Gateway
namespace: default
to:
- group: ""
kind: Secret
name: argocd-server-tlsNote that the ReferenceGrant is created in the argocd namespace as that is where my Secret TLS for Argo CD is located.
I apply this configuration to Kubernetes:
kubectl apply -f argocd-referencegrant.yaml -n argocd
At the time of this writing, there is a known issue with the GRPCRoute the Argo CD Helm chart creates. It incorrectly sets the backend port to port 80 where it should be 443, which can be worked around by just patching the GRPCRoute resource:
kubectl patch grpcroutes argocd-server-grpc -n argocd --type='json' \
-p='[{"op": "replace", "path": "/spec/rules/0/backendRefs/0/port", "value":443}]'I can now visit my Argo CD UI, as before, by using the hostname I set (https://argocd.7f000001.nip.io) using admin as the username and using the generated password extract by running:
kubectl get secret -n argocd argocd-initial-admin-secret \
-o jsonpath='{.data.password}' | base64 -dOnce logged in, I am greeted the same as before but now I’m using Gateway API:
Setting up Gateway API for Argo CD takes a bit more work than using a simple Ingress, but it’s a great opportunity to learn how the new model fits together. The process is already improving, and it will continue to get smoother as Gateway API adoption grows.
Summary
With the retirement of Ingress NGINX, many Kubernetes users are looking for alternatives. In this blog, I walk through replacing Ingress NGINX with Cilium Ingress for a quick and easy path while also exploring Gateway API, the future of Kubernetes traffic management. Using a KIND cluster, I showed above how to install Cilium, enable Ingress, and deploy an application like Argo CD, then moved to Gateway API with GatewayClasses, Gateways, HTTPRoutes, and GRPCRoutes. Along the way, I highlighted how Cilium leads the way in networking and traffic management by providing both solutions with minimal complexity.
I hope this really helped you and will enable you to make your own, informed decision on how to proceed with your own migrations. If you want to learn more, I recommend you to either take our Cilium Ingress or Cilium Gateway API lab, and also dive deeper into our observability lab to see how much more Cilium does offer you for your daily operations.
And of course, we are also happy to support you in your migration journey, just reach out and contact us.
Do you have any questions?
Christian Hernandez, Technical Marketing Engineer