What is Cilium Cluster Mesh? Cilium cluster mesh can be called "Global Services made accessible across a swarm of clusters." These swarms of clusters could be within the respective cloud provider, span multiple cloud providers, span HA zones, span availability zones, and span on-prem and cloud environments. This tutorial guides you through enabling Cilium Cluster Mesh (using Azure CLI) in an AKS cluster (in Dynamic IP allocation mode) running Isovalent Enterprise for Cilium from Azure Marketplace.
What is Isovalent Enterprise for Cilium?
Isovalent Cilium Enterprise is an enterprise-grade, hardened distribution of open-source projects Cilium, Hubble, and Tetragon, built and supported by the Cilium creators. Cilium enhances networking and security at the network layer, while Hubble ensures thorough network observability and tracing. Tetragon ties it all together with runtime enforcement and security observability, offering a well-rounded solution for connectivity, compliance, multi-cloud, and security concerns.
Why Isovalent Enterprise for Cilium?
For enterprise customers requiring support and usage of Advanced Networking, Security, and Observability features, "Isovalent Enterprise for Cilium” is recommended with the following benefits:
- Advanced network policy: Isovalent Cilium Enterprise provides advanced network policy capabilities, including DNS-aware policy, L7 policy, and deny policy, enabling fine-grained control over network traffic for micro-segmentation and improved security.
- Hubble flow observability + User Interface: Isovalent Cilium Enterprise Hubble observability feature provides real-time network traffic flow, policy visualization, and a powerful User Interface for easy troubleshooting and network management.
- Multi-cluster connectivity via Cluster Mesh: Isovalent Cilium Enterprise provides seamless networking and security across multiple clouds, including public cloud providers like AWS, Azure, and Google Cloud Platform, as well as on-premises environments.
- Advanced Security Capabilities via Tetragon: Tetragon provides advanced security capabilities such as protocol enforcement, IP and port whitelisting, and automatic application-aware policy generation to protect against the most sophisticated threats. Built on eBPF, Tetragon can easily scale to meet the needs of the most demanding cloud-native environments.
- Service Mesh: Isovalent Cilium Enterprise provides sidecar-free, seamless service-to-service communication and advanced load balancing, making it easy to deploy and manage complex microservices architectures.
- Enterprise-grade support: Isovalent Cilium Enterprise includes enterprise-grade support from Isovalent’s experienced team of experts, ensuring that any issues are resolved promptly and efficiently. Additionally, professional services help organizations deploy and manage Cilium in production environments.
Pre-Requisites
The following prerequisites need to be taken into account before you proceed with this tutorial:
- You should have an Azure Subscription.
- Install kubectl
- Install Cilium CLI.
- Users can contact their partner Sales/SE representative(s) at sales@isovalent.com for more detailed insights into the features below and access the requisite documentation and Cilium CLI software images.
- Azure CLI version 2.48.1 or later. Run
az –versionto see the currently installed version. If you need to install or upgrade, see Install Azure CLI. - VNet peering is in place across both clusters.
- Ensure you have enough quota resources to create an AKS cluster. Go to the Subscription blade, navigate to “Usage + Quotas,” and make sure you have enough quota for the following resources:
-Regional vCPUs
-Standard Dv4 Family vCPUs
Creating the AKS clusters
Let's briefly see the commands used to bring up the AKS clusters running Azure CNI powered by Cilium.
Cluster 1
Create an AKS cluster with Azure CNI powered by Cilium in Region A.
az group create --name cluster1 --location canadacentral az network vnet create -g cluster1 --location canadacentral --name cluster1 --address-prefixes 192.168.16.0/22 -o none az network vnet subnet create -g cluster1 --vnet-name cluster1 --name nodesubnet --address-prefixes 192.168.16.0/24 -o none az network vnet subnet create -g cluster1 --vnet-name cluster1 --name podsubnet --address-prefixes 192.168.17.0/24 -o none az aks create -n cluster1 -g cluster1 -l canadacentral \ --max-pods 250 \ --network-plugin azure \ --vnet-subnet-id /subscriptions/<subscription>/resourceGroups/cluster1/providers/Microsoft.Network/virtualNetworks/cluster1/subnets/nodesubnet \ --pod-subnet-id /subscriptions/<subscription>/resourceGroups/cluster1/providers/Microsoft.Network/virtualNetworks/cluster1/subnets/podsubnet \ --network-dataplane cilium az aks get-credentials --resource-group cluster1 --name cluster1Cluster 2
Create an AKS cluster with Azure CNI powered by Cilium in Region B.
az group create --name cluster2 --location northeurope az network vnet create -g cluster2 --location northeurope --name cluster2 --address-prefixes 192.168.8.0/22 -o none az network vnet subnet create -g cluster2 --vnet-name cluster2 --name nodesubnet --address-prefixes 192.168.9.0/24 -o none az network vnet subnet create -g cluster2 --vnet-name cluster2 --name podsubnet --address-prefixes 192.168.10.0/24 -o none az aks create -n cluster2 -g cluster2 -l northeurope \ --max-pods 250 \ --network-plugin azure \ --vnet-subnet-id /subscriptions/<subscription>/resourceGroups/cluster2/providers/Microsoft.Network/virtualNetworks/cluster2/subnets/nodesubnet \ --pod-subnet-id /subscriptions/<subscription>/resourceGroups/cluster2/providers/Microsoft.Network/virtualNetworks/cluster2/subnets/podsubnet \ --network-dataplane cilium az aks get-credentials --resource-group cluster2 --name cluster2
How can you Upgrade your AKS clusters to Isovalent Enterprise for Cilium?
Upgrade the AKS clusters to Isovalent Enterprise for Cilium to enable Cluster Mesh on an AKS cluster. In this tutorial, we will upgrade the AKS clusters using Azure Marketplace.
Note- you can also upgrade the AKS clusters via Azure CLI and Azure ARM Templates.
- Login to the Azure Portal
- Search for Marketplace
- Search for Isovalent Enterprise for Cilium and select the container app.
- Click Create
- Select the Subscription, Resource Group, and Region where the cluster was created
- Select “No” for the option that says “Create New Dev Cluster”
- Click Next
- Select the AKS cluster that was created in the previous section.
- Click Next
- The AKS cluster will now be upgraded to Isovalent Enterprise for Cilium.
What are the Node and Pod IPs across clusters?
Once the AKS clusters are upgraded to Isovalent Enterprise for Cilium, you can check that the nodes and pods on the AKS clusters are on distinct IP addresses.
Note- This is a mandatory requirement. You can leverage support for Overlapping IP CIDR, but that support will be available in a future release of Isovalent Enterprise for Cilium from the Azure marketplace.
Cluster 1
Use kubectl get pods and kubectl get nodes and note the distinct IP addresses used for both the pods and nodes.
kubectl get nodes -o wide -A NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME aks-nodepool1-13443004-vmss000000 Ready agent 2d1h v1.27.7 192.168.16.4 <none> Ubuntu 22.04.3 LTS 5.15.0-1053-azure containerd://1.7.5-1 aks-nodepool1-13443004-vmss000001 Ready agent 2d1h v1.27.7 192.168.16.6 <none> Ubuntu 22.04.3 LTS 5.15.0-1053-azure containerd://1.7.5-1 aks-nodepool1-13443004-vmss000002 Ready agent 2d1h v1.27.7 192.168.16.5 <none> Ubuntu 22.04.3 LTS 5.15.0-1053-azure containerd://1.7.5-1 kubectl get pods -o wide -A NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES azure-extensions-usage-system billing-operator-84cd55c557-qvgr2 5/5 Running 0 2d 192.168.17.32 aks-nodepool1-13443004-vmss000002 <none> <none> kube-system azure-cns-dqspr 1/1 Running 0 114m 192.168.16.5 aks-nodepool1-13443004-vmss000002 <none> <none> kube-system azure-cns-fxdzd 1/1 Running 0 115m 192.168.16.6 aks-nodepool1-13443004-vmss000001 <none> <none> kube-system azure-cns-nrldk 1/1 Running 0 115m 192.168.16.4 aks-nodepool1-13443004-vmss000000 <none> <none> kube-system cilium-48wg7 1/1 Running 0 2d 192.168.16.4 aks-nodepool1-13443004-vmss000000 <none> <none> kube-system cilium-operator-d78f778f7-kq4hd 1/1 Running 0 2d 192.168.16.4 aks-nodepool1-13443004-vmss000000 <none> <none> kube-system cilium-operator-d78f778f7-qswpm 1/1 Running 0 2d 192.168.16.5 aks-nodepool1-13443004-vmss000002 <none> <none> kube-system cilium-wgs4b 1/1 Running 0 2d 192.168.16.6 aks-nodepool1-13443004-vmss000001 <none> <none> kube-system cilium-xrpjl 1/1 Running 1 (33h ago) 2d 192.168.16.5 aks-nodepool1-13443004-vmss000002 <none> <none> kube-system cloud-node-manager-6skgj 1/1 Running 0 2d1h 192.168.16.6 aks-nodepool1-13443004-vmss000001 <none> <none> kube-system cloud-node-manager-7nkgb 1/1 Running 0 2d1h 192.168.16.5 aks-nodepool1-13443004-vmss000002 <none> <none> kube-system cloud-node-manager-f4zbj 1/1 Running 0 2d1h 192.168.16.4 aks-nodepool1-13443004-vmss000000 <none> <none> kube-system clustermesh-apiserver-555ccb8654-xg68h 2/2 Running 0 2d 192.168.17.39 aks-nodepool1-13443004-vmss000001 <none> <none> kube-system coredns-789789675-5745t 1/1 Running 0 2d1h 192.168.17.42 aks-nodepool1-13443004-vmss000001 <none> <none> kube-system coredns-789789675-8mm2h 1/1 Running 0 2d1h 192.168.17.27 aks-nodepool1-13443004-vmss000002 <none> <none> kube-system coredns-autoscaler-649b947bbd-zn96n 1/1 Running 0 2d1h 192.168.17.34 aks-nodepool1-13443004-vmss000002 <none> <none> kube-system csi-azuredisk-node-f6l7d 3/3 Running 0 2d1h 192.168.16.5 aks-nodepool1-13443004-vmss000002 <none> <none> kube-system csi-azuredisk-node-sspm4 3/3 Running 0 2d1h 192.168.16.6 aks-nodepool1-13443004-vmss000001 <none> <none> kube-system csi-azuredisk-node-stljp 3/3 Running 0 2d1h 192.168.16.4 aks-nodepool1-13443004-vmss000000 <none> <none> kube-system csi-azurefile-node-c5dqz 3/3 Running 0 2d1h 192.168.16.4 aks-nodepool1-13443004-vmss000000 <none> <none> kube-system csi-azurefile-node-hdxxv 3/3 Running 0 2d1h 192.168.16.6 aks-nodepool1-13443004-vmss000001 <none> <none> kube-system csi-azurefile-node-wf8ww 3/3 Running 0 2d1h 192.168.16.5 aks-nodepool1-13443004-vmss000002 <none> <none> kube-system extension-agent-59ff6f87bc-6wsdv 2/2 Running 0 2d 192.168.17.6 aks-nodepool1-13443004-vmss000000 <none> <none> kube-system extension-operator-59fcdc5cdc-zbgxt 2/2 Running 0 2d 192.168.17.20 aks-nodepool1-13443004-vmss000000 <none> <none> kube-system hubble-relay-76ff659b59-vk4kp 1/1 Running 0 2d 192.168.17.5 aks-nodepool1-13443004-vmss000000 <none> <none> kube-system konnectivity-agent-6854bdfcf9-f74gw 1/1 Running 0 2d 192.168.17.50 aks-nodepool1-13443004-vmss000001 <none> <none> kube-system konnectivity-agent-6854bdfcf9-j6b9r 1/1 Running 0 2d 192.168.17.10 aks-nodepool1-13443004-vmss000000 <none> <none> kube-system metrics-server-5467676b76-ngmn7 2/2 Running 0 2d1h 192.168.17.40 aks-nodepool1-13443004-vmss000001 <none> <none> kube-system metrics-server-5467676b76-rtvrx 2/2 Running 0 2d1h 192.168.17.16 aks-nodepool1-13443004-vmss000000 <none> <none>Cluster 2
Use kubectl get pods and kubectl get nodes and note the distinct IP addresses used for both the pods and nodes.
kubectl get nodes -o wide -A NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME aks-nodepool1-42826393-vmss000000 Ready agent 2d1h v1.27.7 192.168.9.5 <none> Ubuntu 22.04.3 LTS 5.15.0-1053-azure containerd://1.7.5-1 aks-nodepool1-42826393-vmss000001 Ready agent 2d1h v1.27.7 192.168.9.4 <none> Ubuntu 22.04.3 LTS 5.15.0-1053-azure containerd://1.7.5-1 aks-nodepool1-42826393-vmss000002 Ready agent 2d1h v1.27.7 192.168.9.6 <none> Ubuntu 22.04.3 LTS 5.15.0-1053-azure containerd://1.7.5-1 kubectl get pods -o wide -A NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES azure-extensions-usage-system billing-operator-84cd55c557-k7spc 5/5 Running 0 2d 192.168.10.25 aks-nodepool1-42826393-vmss000000 <none> <none> kube-system azure-cns-5psrk 1/1 Running 0 2d1h 192.168.9.6 aks-nodepool1-42826393-vmss000002 <none> <none> kube-system azure-cns-krcfn 1/1 Running 0 2d1h 192.168.9.5 aks-nodepool1-42826393-vmss000000 <none> <none> kube-system azure-cns-nmfkl 1/1 Running 0 2d1h 192.168.9.4 aks-nodepool1-42826393-vmss000001 <none> <none> kube-system cilium-59zm4 1/1 Running 0 2d 192.168.9.6 aks-nodepool1-42826393-vmss000002 <none> <none> kube-system cilium-operator-d78f778f7-mhkg8 1/1 Running 0 2d 192.168.9.4 aks-nodepool1-42826393-vmss000001 <none> <none> kube-system cilium-operator-d78f778f7-s5r92 1/1 Running 0 2d 192.168.9.5 aks-nodepool1-42826393-vmss000000 <none> <none> kube-system cilium-rcmbd 1/1 Running 0 2d 192.168.9.4 aks-nodepool1-42826393-vmss000001 <none> <none> kube-system cilium-s9czz 1/1 Running 0 2d 192.168.9.5 aks-nodepool1-42826393-vmss000000 <none> <none> kube-system cloud-node-manager-7t5cz 1/1 Running 0 2d1h 192.168.9.5 aks-nodepool1-42826393-vmss000000 <none> <none> kube-system cloud-node-manager-c66ft 1/1 Running 0 2d1h 192.168.9.4 aks-nodepool1-42826393-vmss000001 <none> <none> kube-system cloud-node-manager-dtdqk 1/1 Running 0 2d1h 192.168.9.6 aks-nodepool1-42826393-vmss000002 <none> <none> kube-system clustermesh-apiserver-555ccb8654-grxtt 2/2 Running 0 2d 192.168.10.44 aks-nodepool1-42826393-vmss000002 <none> <none> kube-system coredns-789789675-7k5zv 1/1 Running 0 2d1h 192.168.10.17 aks-nodepool1-42826393-vmss000001 <none> <none> kube-system coredns-789789675-hb5rr 1/1 Running 0 2d1h 192.168.10.49 aks-nodepool1-42826393-vmss000002 <none> <none> kube-system coredns-autoscaler-649b947bbd-rggpl 1/1 Running 0 2d1h 192.168.10.18 aks-nodepool1-42826393-vmss000001 <none> <none> kube-system csi-azuredisk-node-hrdj2 3/3 Running 0 2d1h 192.168.9.6 aks-nodepool1-42826393-vmss000002 <none> <none> kube-system csi-azuredisk-node-sngjz 3/3 Running 0 2d1h 192.168.9.5 aks-nodepool1-42826393-vmss000000 <none> <none> kube-system csi-azuredisk-node-zn2ft 3/3 Running 0 2d1h 192.168.9.4 aks-nodepool1-42826393-vmss000001 <none> <none> kube-system csi-azurefile-node-46st2 3/3 Running 0 2d1h 192.168.9.5 aks-nodepool1-42826393-vmss000000 <none> <none> kube-system csi-azurefile-node-4t2lg 3/3 Running 0 2d1h 192.168.9.6 aks-nodepool1-42826393-vmss000002 <none> <none> kube-system csi-azurefile-node-v7kdj 3/3 Running 0 2d1h 192.168.9.4 aks-nodepool1-42826393-vmss000001 <none> <none> kube-system extension-agent-59ff6f87bc-k9rb8 2/2 Running 0 2d 192.168.10.5 aks-nodepool1-42826393-vmss000001 <none> <none> kube-system extension-operator-59fcdc5cdc-cgx2t 2/2 Running 0 2d 192.168.10.29 aks-nodepool1-42826393-vmss000000 <none> <none> kube-system hubble-relay-76ff659b59-h9vh6 1/1 Running 0 2d 192.168.10.41 aks-nodepool1-42826393-vmss000002 <none> <none> kube-system konnectivity-agent-9b76986cd-96qhh 1/1 Running 0 2d 192.168.10.26 aks-nodepool1-42826393-vmss000000 <none> <none> kube-system konnectivity-agent-9b76986cd-kncrq 1/1 Running 0 2d 192.168.10.40 aks-nodepool1-42826393-vmss000002 <none> <none> kube-system metrics-server-5467676b76-6tjfv 2/2 Running 0 2d1h 192.168.10.24 aks-nodepool1-42826393-vmss000000 <none> <none> kube-system metrics-server-5467676b76-8vc95 2/2 Running 0 2d1h 192.168.10.48 aks-nodepool1-42826393-vmss000002 <none> <none>
How can you Peer the AKS clusters?
Use VNet peering to peer the AKS clusters across the two chosen regions. This step only needs to be done in one direction. The connection will automatically be established in both directions.
- Login to the Azure Portal
- Click Home
- Click Virtual Network
- Select the respective Virtual Network
- Click Peerings
- Click Add
- Give the local peer a name
- Select “Allow cluster1 to access cluster2”
- Give the remote peer a name
- Select the virtual network deployment model as “Resource Manager”
- Select the subscription
- Select the virtual network of the remote peer
- Select “Allow cluster2 to access cluster1”
- Click Add
Are the pods and nodes across clusters reachable?
With VNet peering in place, ensure that pods and nodes across clusters are reachable. As you can see below, the cluster mesh pods are reachable from distinct nodes on the AKS clusters running in different regions:
Cluster 1
Clustermesh pods in Cluster 2 should be reachable from Cluster 1.
root@aks-nodepool1-13443004-vmss000000:/# ping 192.168.10.44 PING 192.168.10.44 (192.168.10.44) 56(84) bytes of data. 64 bytes from 192.168.10.44: icmp_seq=1 ttl=63 time=82.8 ms 64 bytes from 192.168.10.44: icmp_seq=2 ttl=63 time=82.5 ms 64 bytes from 192.168.10.44: icmp_seq=3 ttl=63 time=82.0 ms 64 bytes from 192.168.10.44: icmp_seq=4 ttl=63 time=82.3 ms 64 bytes from 192.168.10.44: icmp_seq=5 ttl=63 time=86.0 ms 64 bytes from 192.168.10.44: icmp_seq=6 ttl=63 time=82.2 ms 64 bytes from 192.168.10.44: icmp_seq=7 ttl=63 time=82.3 ms --- 192.168.10.44 ping statistics --- 7 packets transmitted, 7 received, 0% packet loss, time 6007ms rtt min/avg/max/mdev = 81.981/82.855/85.965/1.289 msCluster 2
Clustermesh pods in Cluster 1 should be reachable from Cluster 2.
root@aks-nodepool1-42826393-vmss000000:/# ping 192.168.17.39 PING 192.168.17.39 (192.168.17.39) 56(84) bytes of data. 64 bytes from 192.168.17.39: icmp_seq=1 ttl=63 time=83.3 ms 64 bytes from 192.168.17.39: icmp_seq=2 ttl=63 time=83.1 ms 64 bytes from 192.168.17.39: icmp_seq=3 ttl=63 time=83.3 ms 64 bytes from 192.168.17.39: icmp_seq=4 ttl=63 time=83.1 ms 64 bytes from 192.168.17.39: icmp_seq=5 ttl=63 time=83.1 ms 64 bytes from 192.168.17.39: icmp_seq=6 ttl=63 time=83.1 ms --- 192.168.17.39 ping statistics --- 6 packets transmitted, 6 received, 0% packet loss, time 5006ms rtt min/avg/max/mdev = 83.052/83.145/83.301/0.094 ms
How can you enable Cluster Mesh?
You can enable Cluster Mesh via Azure CLI or an ARM template.
Note- since you are altering the ClusterID, it will recreate all Cilium identities in a production cluster with existing connections, which should be handled appropriately.
Azure CLI
Follow the detailed instructions on Azure CLI to create and upgrade a cluster on AKS running Isovalent Enterprise for Cilium.
Cluster 1
You can enable Clustermesh by using Azure CLI. In this case, you can enable it on an AKS cluster running Isovalent Enterprise for Cilium and name the cluster 'cluster1'.
az k8s-extension update -c cluster1 -t managedClusters -g cluster1 -n cilium --configuration-settings cluster.name=cluster1 az k8s-extension update -c cluster1 -t managedClusters -g cluster1 -n cilium --configuration-settings cluster.id=1 az k8s-extension update -c cluster1 -t managedClusters -g cluster1 -n cilium --configuration-settings kubeProxyReplacement=strict az k8s-extension update -c cluster1 -t managedClusters -g cluster1 -n cilium --configuration-settings k8sServicePort=<API_SERVER_PORT> az k8s-extension update -c cluster1 -t managedClusters -g cluster1 -n cilium --configuration-settings k8sServiceHost=<API_SERVER_FQDN> az k8s-extension update -c cluster1 -t managedClusters -g cluster1 -n cilium --configuration-settings clustermesh.config.enabled=false az k8s-extension update -c cluster1 -t managedClusters -g cluster1 -n cilium --configuration-settings clustermesh.useAPIServer=true az k8s-extension update -c cluster1 -t managedClusters -g cluster1 -n cilium --configuration-settings clustermesh.apiserver.service.type=LoadBalancerCluster 2
You can enable Clustermesh by using Azure CLI. In this case, you can enable it on an AKS cluster running Isovalent Enterprise for Cilium and name the cluster 'cluster2'.
az k8s-extension update -c cluster2 -t managedClusters -g cluster2 -n cilium --configuration-settings cluster.name=cluster2 az k8s-extension update -c cluster2 -t managedClusters -g cluster2 -n cilium --configuration-settings cluster.id=2 az k8s-extension update -c cluster1 -t managedClusters -g cluster1 -n cilium --configuration-settings kubeProxyReplacement=strict az k8s-extension update -c cluster1 -t managedClusters -g cluster1 -n cilium --configuration-settings k8sServicePort=<API_SERVER_PORT> az k8s-extension update -c cluster1 -t managedClusters -g cluster1 -n cilium --configuration-settings k8sServiceHost=<API_SERVER_FQDN> az k8s-extension update -c cluster2 -t managedClusters -g cluster2 -n cilium --configuration-settings clustermesh.config.enabled=false az k8s-extension update -c cluster2 -t managedClusters -g cluster2 -n cilium --configuration-settings clustermesh.useAPIServer=true az k8s-extension update -c cluster2 -t managedClusters -g cluster2 -n cilium --configuration-settings clustermesh.apiserver.service.type=LoadBalancer
How can you verify Cluster Mesh status?
Check the status of the clusters by running cilium clustermesh status on either of the clusters.
cilium clustermesh status --context cluster1 --wait
cilium clustermesh status --context cluster2 --waitWhat's the status of the clustermesh-api pod?
The Cluster Mesh API Server contains an etcd instance to keep track of the cluster’s state. The state from multiple clusters is never mixed. Cilium agents in other clusters connect to the Cluster Mesh API Server to watch for changes and replicate the multi-cluster state into their cluster. Access to the Cluster Mesh API Server is protected using TLS certificates.
Access from one cluster to another is always read-only, ensuring that failure domains remain unchanged. A failure in one cluster never propagates to other clusters.
Ensure that the clustermesh-api pod is running on both clusters.
kubectl get pods -A -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
azure-extensions-usage-system billing-operator-84cd55c557-k7spc 5/5 Running 0 2d 192.168.10.25 aks-nodepool1-42826393-vmss000000 <none> <none>
kube-system azure-cns-5psrk 1/1 Running 0 2d 192.168.9.6 aks-nodepool1-42826393-vmss000002 <none> <none>
kube-system azure-cns-krcfn 1/1 Running 0 2d 192.168.9.5 aks-nodepool1-42826393-vmss000000 <none> <none>
kube-system azure-cns-nmfkl 1/1 Running 0 2d 192.168.9.4 aks-nodepool1-42826393-vmss000001 <none> <none>
kube-system cilium-59zm4 1/1 Running 0 47h 192.168.9.6 aks-nodepool1-42826393-vmss000002 <none> <none>
kube-system cilium-operator-d78f778f7-mhkg8 1/1 Running 0 2d 192.168.9.4 aks-nodepool1-42826393-vmss000001 <none> <none>
kube-system cilium-operator-d78f778f7-s5r92 1/1 Running 0 2d 192.168.9.5 aks-nodepool1-42826393-vmss000000 <none> <none>
kube-system cilium-rcmbd 1/1 Running 0 47h 192.168.9.4 aks-nodepool1-42826393-vmss000001 <none> <none>
kube-system cilium-s9czz 1/1 Running 0 47h 192.168.9.5 aks-nodepool1-42826393-vmss000000 <none> <none>
kube-system cloud-node-manager-7t5cz 1/1 Running 0 2d 192.168.9.5 aks-nodepool1-42826393-vmss000000 <none> <none>
kube-system cloud-node-manager-c66ft 1/1 Running 0 2d 192.168.9.4 aks-nodepool1-42826393-vmss000001 <none> <none>
kube-system cloud-node-manager-dtdqk 1/1 Running 0 2d 192.168.9.6 aks-nodepool1-42826393-vmss000002 <none> <none>
kube-system clustermesh-apiserver-555ccb8654-grxtt 2/2 Running 0 47h 192.168.10.44 aks-nodepool1-42826393-vmss000002 <none> <none>
kube-system coredns-789789675-7k5zv 1/1 Running 0 2d 192.168.10.17 aks-nodepool1-42826393-vmss000001 <none> <none>
kube-system coredns-789789675-hb5rr 1/1 Running 0 2d 192.168.10.49 aks-nodepool1-42826393-vmss000002 <none> <none>
kube-system coredns-autoscaler-649b947bbd-rggpl 1/1 Running 0 2d 192.168.10.18 aks-nodepool1-42826393-vmss000001 <none> <none>
kube-system csi-azuredisk-node-hrdj2 3/3 Running 0 2d 192.168.9.6 aks-nodepool1-42826393-vmss000002 <none> <none>
kube-system csi-azuredisk-node-sngjz 3/3 Running 0 2d 192.168.9.5 aks-nodepool1-42826393-vmss000000 <none> <none>
kube-system csi-azuredisk-node-zn2ft 3/3 Running 0 2d 192.168.9.4 aks-nodepool1-42826393-vmss000001 <none> <none>
kube-system csi-azurefile-node-46st2 3/3 Running 0 2d 192.168.9.5 aks-nodepool1-42826393-vmss000000 <none> <none>
kube-system csi-azurefile-node-4t2lg 3/3 Running 0 2d 192.168.9.6 aks-nodepool1-42826393-vmss000002 <none> <none>
kube-system csi-azurefile-node-v7kdj 3/3 Running 0 2d 192.168.9.4 aks-nodepool1-42826393-vmss000001 <none> <none>
kube-system extension-agent-59ff6f87bc-k9rb8 2/2 Running 0 2d 192.168.10.5 aks-nodepool1-42826393-vmss000001 <none> <none>
kube-system extension-operator-59fcdc5cdc-cgx2t 2/2 Running 0 2d 192.168.10.29 aks-nodepool1-42826393-vmss000000 <none> <none>
kube-system hubble-relay-76ff659b59-h9vh6 1/1 Running 0 2d 192.168.10.41 aks-nodepool1-42826393-vmss000002 <none> <none>
kube-system konnectivity-agent-9b76986cd-96qhh 1/1 Running 0 47h 192.168.10.26 aks-nodepool1-42826393-vmss000000 <none> <none>
kube-system konnectivity-agent-9b76986cd-kncrq 1/1 Running 0 47h 192.168.10.40 aks-nodepool1-42826393-vmss000002 <none> <none>
kube-system metrics-server-5467676b76-6tjfv 2/2 Running 0 2d 192.168.10.24 aks-nodepool1-42826393-vmss000000 <none> <none>
kube-system metrics-server-5467676b76-8vc95 2/2 Running 0 2d 192.168.10.48 aks-nodepool1-42826393-vmss000002 <none> <none>How can you connect the clusters?
Note- Since the isovalent image available from the marketplace is 1.12, the Cilium CLI is recommended.
Connect the AKS clusters using Cilium CLI. This step only needs to be done in one direction. The connection will automatically be established in both directions:
cilium clustermesh connect --context cluster1 --destination-context cluster2
✅ Detected Helm release with Cilium version 1.12.17
⚠️ Cilium Version is less than 1.14.0. Continuing in classic mode.
✨ Extracting access information of cluster cluster2...
🔑 Extracting secrets from cluster cluster2...
ℹ️ Found ClusterMesh service IPs: [ 20.116.185.121]
✨ Extracting access information of cluster cluster1...
🔑 Extracting secrets from cluster cluster1...
ℹ️ Found ClusterMesh service IPs: [52.142.113.148]
✨ Connecting cluster cluster1 -> cluster2...
🔑 Secret cilium-clustermesh does not exist yet, creating it...
🔑 Patching existing secret cilium-clustermesh...
✨ Patching DaemonSet with IP aliases cilium-clustermesh...
✨ Connecting cluster cluster2 -> cluster1...
🔑 Secret cilium-clustermesh does not exist yet, creating it...
🔑 Patching existing secret cilium-clustermesh...
✨ Patching DaemonSet with IP aliases cilium-clustermesh...
✅ Connected cluster cluster1 and cluster2!The output would be something as below:
Cluster 1
Cluster Mesh Output in Cluster 1
cilium clustermesh status --context cluster1 --wait ✅ Service "clustermesh-apiserver" of type "LoadBalancer" found ✅ Cluster access information is available: - 20.116.185.121:2379 ✅ Deployment clustermesh-apiserver is ready ✅ All 3 nodes are connected to all clusters [min:1 / avg:1.0 / max:1] 🔌 Cluster Connections: - cluster2: 3/3 configured, 3/3 connected 🔀 Global services: [ min:10 / avg:10.0 / max:10 ]Cluster 2
Cluster Mesh Output in Cluster 2
cilium clustermesh status --context cluster2 --wait ✅ Service "clustermesh-apiserver" of type "LoadBalancer" found ✅ Cluster access information is available: - 52.142.113.148:2379 ✅ Deployment clustermesh-apiserver is ready ✅ All 3 nodes are connected to all clusters [min:1 / avg:1.0 / max:1] 🔌 Cluster Connections: - cluster1: 3/3 configured, 3/3 connected 🔀 Global services: [ min:10 / avg:10.0 / max:10 ]
Can you do Clustermesh amongst clusters with varied Kubernetes Distribution types?
- As of 10-02-2025, added clustermesh with three Kubernetes clusters running Isovalent Cilium Enterprise 1.15.9.
- The Kubeadm cluster was connected using ExpressRoute to the AKS cluster running Isovalent Enterprise for Cilium from Azure Marketplace and BYOCNI (Bring Your Own CNI).
- The service type used in this case is
Nodeport.
- The clustermesh cluster from Azure Marketplace is running Isovalent Cilium Enterprise 1.15.9-cee.1
kubectl exec -n kube-system -ti ds/cilium -- cilium-dbg status --all-clusters
Defaulted container "cilium-agent" out of: cilium-agent, config (init), mount-cgroup (init), apply-sysctl-overwrites (init), mount-bpf-fs (init), clean-cilium-state (init), install-cni-binaries (init), block-wireserver (init)
KVStore: Ok Disabled
Kubernetes: Ok 1.29 (v1.29.9) [linux/amd64]
Kubernetes APIs: ["EndpointSliceOrEndpoint", "cilium/v2::CiliumClusterwideNetworkPolicy", "cilium/v2::CiliumEndpoint", "cilium/v2::CiliumNetworkPolicy", "cilium/v2::CiliumNode", "cilium/v2alpha1::CiliumCIDRGroup", "core/v1::Namespace", "core/v1::Pods", "core/v1::Service", "networking.k8s.io/v1::NetworkPolicy"]
KubeProxyReplacement: True [eth0 10.224.0.4 fe80::6245:bdff:fe5b:e41a (Direct Routing)]
Host firewall: Disabled
SRv6: Disabled
CNI Chaining: none
CNI Config file: CNI configuration management disabled
Cilium: Ok 1.15.10-cee.1 (v1.15.10-cee.1-f3d547b9)
NodeMonitor: Listening for events on 4 CPUs with 64x4096 of shared memory
IPAM: IPv4: delegated to plugin,
ClusterMesh: 2/2 remote clusters ready, 1 global-services
byocni: ready, 3 nodes, 10 endpoints, 6 identities, 1 services, 0 reconnections (last: never)
└ etcd: 1/1 connected, leases=0, lock leases=1, has-quorum=true: endpoint status checks are disabled, ID: 64beb9bc3f0a6306
└ remote configuration: expected=true, retrieved=true, cluster-id=2, kvstoremesh=false, sync-canaries=true
└ synchronization status: nodes=true, endpoints=true, identities=true, services=true
kubeadm: ready, 1 nodes, 7 endpoints, 4 identities, 1 services, 0 reconnections (last: never)
└ etcd: 1/1 connected, leases=0, lock leases=1, has-quorum=true: endpoint status checks are disabled, ID: 9286b3b399c31d09
└ remote configuration: expected=true, retrieved=true, cluster-id=1, kvstoremesh=false, sync-canaries=true
└ synchronization status: nodes=true, endpoints=true, identities=true, services=true
IPv4 BIG TCP: Disabled
IPv6 BIG TCP: Disabled
BandwidthManager: Disabled
Host Routing: Legacy
Masquerading: Disabled
Controller Status: 61/61 healthy
Proxy Status: OK, ip 169.254.23.0, 0 redirects active on ports 10000-20000, Envoy: embedded
Global Identity Range: min 196608, max 262143
Hubble: Ok Current/Max Flows: 1306/4095 (31.89%), Flows/s: 42.56 Metrics: Disabled
Encryption: Disabled
Cluster health: Probe disabledThe AKS BYOCNI cluster is running Isovalent Cilium Enterprise 1.15.9-cee.1.
kubectl exec -n kube-system -ti ds/cilium -- cilium-dbg status --all-clusters
Defaulted container "cilium-agent" out of: cilium-agent, config (init), mount-cgroup (init), apply-sysctl-overwrites (init), mount-bpf-fs (init), wait-for-node-init (init), clean-cilium-state (init), install-cni-binaries (init)
KVStore: Ok Disabled
Kubernetes: Ok 1.29 (v1.29.8) [linux/amd64]
Kubernetes APIs: ["EndpointSliceOrEndpoint", "cilium/v2::CiliumClusterwideNetworkPolicy", "cilium/v2::CiliumEndpoint", "cilium/v2::CiliumNetworkPolicy", "cilium/v2::CiliumNode", "cilium/v2alpha1::CiliumCIDRGroup", "core/v1::Namespace", "core/v1::Pods", "core/v1::Service", "networking.k8s.io/v1::NetworkPolicy"]
KubeProxyReplacement: True [eth0 192.168.20.5 fe80::222:48ff:fed6:564e (Direct Routing)]
Host firewall: Disabled
SRv6: Disabled
CNI Chaining: none
CNI Config file: successfully wrote CNI configuration file to /host/etc/cni/net.d/05-cilium.conflist
Cilium: Ok 1.15.9-cee.1 (v1.15.9-cee.1-a2a2b7ad)
NodeMonitor: Listening for events on 2 CPUs with 64x4096 of shared memory
Cilium health daemon: Ok
IPAM: IPv4: 3/254 allocated from 10.20.0.0/24,
ClusterMesh: 2/2 remote clusters ready, 1 global-services
cmesh115: ready, 2 nodes, 12 endpoints, 9 identities, 0 services, 0 reconnections (last: never)
└ etcd: 1/1 connected, leases=0, lock leases=1, has-quorum=true: endpoint status checks are disabled, ID: ba550d68d124e2b7
└ remote configuration: expected=true, retrieved=true, cluster-id=3, kvstoremesh=false, sync-canaries=true
└ synchronization status: nodes=true, endpoints=true, identities=true, services=true
kubeadm: ready, 1 nodes, 7 endpoints, 4 identities, 1 services, 0 reconnections (last: never)
└ etcd: 1/1 connected, leases=0, lock leases=1, has-quorum=true: endpoint status checks are disabled, ID: 9286b3b399c31d09
└ remote configuration: expected=true, retrieved=true, cluster-id=1, kvstoremesh=false, sync-canaries=true
└ synchronization status: nodes=true, endpoints=true, identities=true, services=true
IPv4 BIG TCP: Disabled
IPv6 BIG TCP: Disabled
BandwidthManager: Disabled
Host Routing: Legacy
Masquerading: IPTables [IPv4: Enabled, IPv6: Disabled]
Controller Status: 31/31 healthy
Proxy Status: OK, ip 10.20.0.220, 0 redirects active on ports 10000-20000, Envoy: embedded
Global Identity Range: min 131072, max 196607
Hubble: Ok Current/Max Flows: 4095/4095 (100.00%), Flows/s: 13.16 Metrics: Disabled
Encryption: Disabled
Cluster health: 6/6 reachable (2025-02-10T10:40:33Z)
Modules Health: Stopped(0) Degraded(0) OK(12)The Kubeadm cluster is running Isovalent Cilium Enterprise 1.15.9-cee.1.
kubectl exec -n kube-system -ti ds/cilium -- cilium-dbg status --all-clusters
Defaulted container "cilium-agent" out of: cilium-agent, config (init), mount-cgroup (init), apply-sysctl-overwrites (init), mount-bpf-fs (init), clean-cilium-state (init), install-cni-binaries (init)
KVStore: Ok Disabled
Kubernetes: Ok 1.29 (v1.29.9) [linux/amd64]
Kubernetes APIs: ["EndpointSliceOrEndpoint", "cilium/v2::CiliumClusterwideNetworkPolicy", "cilium/v2::CiliumEndpoint", "cilium/v2::CiliumNetworkPolicy", "cilium/v2::CiliumNode", "cilium/v2alpha1::CiliumCIDRGroup", "core/v1::Namespace", "core/v1::Pods", "core/v1::Service", "networking.k8s.io/v1::NetworkPolicy"]
KubeProxyReplacement: True [eth0 192.168.10.4 fe80::6245:bdff:fe27:5287 (Direct Routing)]
Host firewall: Disabled
SRv6: Disabled
CNI Chaining: none
CNI Config file: successfully wrote CNI configuration file to /host/etc/cni/net.d/05-cilium.conflist
Cilium: Ok 1.15.9-cee.1 (v1.15.9-cee.1-a2a2b7ad)
NodeMonitor: Listening for events on 2 CPUs with 64x4096 of shared memory
Cilium health daemon: Ok
IPAM: IPv4: 9/254 allocated from 10.10.0.0/24,
ClusterMesh: 2/2 remote clusters ready, 1 global-services
byocni: ready, 3 nodes, 10 endpoints, 6 identities, 1 services, 0 reconnections (last: never)
└ etcd: 1/1 connected, leases=0, lock leases=1, has-quorum=true: endpoint status checks are disabled, ID: 64beb9bc3f0a6306
└ remote configuration: expected=true, retrieved=true, cluster-id=2, kvstoremesh=false, sync-canaries=true
└ synchronization status: nodes=true, endpoints=true, identities=true, services=true
cmesh115: ready, 2 nodes, 12 endpoints, 9 identities, 0 services, 0 reconnections (last: never)
└ etcd: 1/1 connected, leases=0, lock leases=1, has-quorum=true: endpoint status checks are disabled, ID: ba550d68d124e2b7
└ remote configuration: expected=true, retrieved=true, cluster-id=3, kvstoremesh=false, sync-canaries=true
└ synchronization status: nodes=true, endpoints=true, identities=true, services=true
IPv4 BIG TCP: Disabled
IPv6 BIG TCP: Disabled
BandwidthManager: Disabled
Host Routing: Legacy
Masquerading: IPTables [IPv4: Enabled, IPv6: Disabled]
Controller Status: 66/67 healthy
Name Last success Last error Count Message
remote-etcd-cluster1 never 1m34s ago 12 timed out while waiting for etcd session. Ensure that etcd is running on [https://cluster1.mesh.cilium.io:32379]
Proxy Status: OK, ip 10.10.0.16, 0 redirects active on ports 10000-20000, Envoy: embedded
Global Identity Range: min 65536, max 131071
Hubble: Ok Current/Max Flows: 4095/4095 (100.00%), Flows/s: 34.10 Metrics: Disabled
Encryption: Disabled
Cluster health: 6/6 reachable (2025-02-10T12:43:56Z)
Modules Health: Stopped(0) Degraded(0) OK(12)What can you do now that cluster mesh is enabled?
Load-balancing & Service Discovery
The global service discovery of Cilium’s multi-cluster model is built using standard Kubernetes services and designed to be completely transparent to existing Kubernetes application deployments.
Create a global service that can be accessed from both clusters. Establishing load-balancing between clusters is achieved by defining a Kubernetes service with an identical name and namespace in each cluster and adding the annotation io.cilium/global-service: "true" to declare it global. Cilium will automatically perform load-balancing to pods in both clusters.
apiVersion: v1
kind: Service
metadata:
name: rebel-base
annotations:
io.cilium/global-service: "true"
spec:
type: ClusterIP
ports:
- port: 80
selector:
name: rebel-baseDeploying a Service
- In Cluster 1, let's deploy a service
kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/v1.12/examples/kubernetes/clustermesh/global-service-example/rebel-base-global-shared.yaml
kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/v1.12/examples/kubernetes/clustermesh/global-service-example/cluster1.yaml- In Cluster 2, let's deploy a service
kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/v1.12/examples/kubernetes/clustermesh/global-service-example/rebel-base-global-shared.yaml
kubectl apply -f https://raw.githubusercontent.com/cilium/cilium/v1.12/examples/kubernetes/clustermesh/global-service-example/cluster2.yaml- From either cluster, access the global service. Notice the response is received from pods across both clusters.
kubectl exec -ti deployments/x-wing -- /bin/sh -c 'for i in $(seq 1 10); do curl rebel-base; done'
{"Galaxy": "Alderaan", "Cluster": "Cluster-2"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-2"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-2"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-2"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}Service Affinity
Using the Local Service Affinity annotation, it is possible to load balance traffic only to local endpoints unless the local endpoints aren’t available. Then, traffic will be sent to endpoints in remote clusters. Load-balancing across multiple clusters might not be ideal in some cases. The annotation io.cilium/service-affinity: "local|remote|none" can be used to specify the preferred endpoint destination.
For example, if the value of annotation io.cilium/service-affinity is local, the Global Service will load-balance across healthy local backends, and only user remote endpoints if and only if all of the local backends are not available or unhealthy.
- In cluster 1, add
io.cilium/service-affinity="local"to existing global service
kubectl annotate service rebel-base io.cilium/service-affinity=local --overwrite- From cluster 1, access the global service. You will see replies from pods in
cluster 1only.
kubectl exec -ti deployments/x-wing -- /bin/sh -c 'for i in $(seq 1 10); do curl rebel-base; done'
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}- From cluster 2, access the global service. As usual, you will see replies from pods in both clusters.
kubectl exec -ti deployments/x-wing -- /bin/sh -c 'for i in $(seq 1 10); do curl rebel-base; done'
{"Galaxy": "Alderaan", "Cluster": "Cluster-2"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-2"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-2"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-2"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-1"}- From cluster 1, check the service endpoints; the local endpoints are marked as preferred.
kubectl exec -n kube-system -ti ds/cilium -- cilium service list --clustermesh-affinity
Defaulted container "cilium-agent" out of: cilium-agent, mount-cgroup (init), apply-sysctl-overwrites (init), mount-bpf-fs (init), clean-cilium-state (init), install-cni-binaries (init), block-wireserver (init)
ID Frontend Service Type Backend
1 10.0.0.1:443 ClusterIP 1 => 20.123.98.50:443 (active)
2 10.0.0.10:53 ClusterIP 1 => 192.168.10.17:53 (active)
2 => 192.168.10.49:53 (active)
3 10.0.116.42:443 ClusterIP 1 => 192.168.10.24:4443 (active)
2 => 192.168.10.48:4443 (active)
4 10.0.202.68:8443 ClusterIP
5 10.0.124.112:8443 ClusterIP
6 10.0.26.200:443 ClusterIP 1 => 192.168.10.25:8443 (active)
7 10.0.181.227:80 ClusterIP 1 => 192.168.10.41:4245 (active)
8 10.0.11.10:443 ClusterIP 1 => 192.168.9.5:4244 (terminating)
2 => 192.168.9.4:4244 (active)
3 => 192.168.9.6:4244 (active)
9 10.0.191.205:2379 ClusterIP 1 => 192.168.10.44:2379 (active)
10 192.168.9.5:32379 NodePort 1 => 192.168.10.44:2379 (active)
11 0.0.0.0:32379 NodePort 1 => 192.168.10.44:2379 (active)
12 52.142.113.148:2379 LoadBalancer 1 => 192.168.10.44:2379 (active)
18 10.0.201.56:80 ClusterIP 1 => 192.168.10.19:80 (active) (preferred)
2 => 192.168.10.30:80 (active) (preferred)
3 => 192.168.17.18:80 (active)
4 => 192.168.17.23:80 (active)- In cluster 1, change
io.cilium/service-affinitythe value toremotefor existing global service
kubectl annotate service rebel-base io.cilium/service-affinity=remote --overwrite- From cluster 1, access the global service. This time, the replies are coming from pods in
cluster 2only.
kubectl exec -ti deployments/x-wing -- /bin/sh -c 'for i in $(seq 1 10); do curl rebel-base; done'
{"Galaxy": "Alderaan", "Cluster": "Cluster-2"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-2"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-2"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-2"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-2"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-2"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-2"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-2"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-2"}
{"Galaxy": "Alderaan", "Cluster": "Cluster-2"}Conclusion
Hopefully, this post gave you a good overview of deploying Cluster mesh in an AKS cluster running Isovalent Enterprise for Cilium. If you have any feedback on the solution, please share it with us. You’ll find us on the Cilium Slack channel.
Try it Out
Isovalent Enterprise for Cilium on the Azure marketplace.
Further Reading
- Topology Aware Routing and Service Mesh across clusters with Cluster Mesh
- Upgrade to cilium in Azure
- Azure and Isovalent main partner page
Amit Gupta, Senior Technical Marketing Engineer