Multi Availability Zone, or multi AZ, Kubernetes is the right default for most production platforms. You get stronger availability, cleaner failure isolation, and more room to scale without betting everything on a single failure domain. In public cloud terms, a zone is an isolated failure domain within a region, typically one or more data centers with independent power and networking. AWS calls these Availability Zones, Azure uses Availability Zones, and Google Cloud calls them zones, but the intent is the same, spread workloads across zones so a single site outage does not take down your platform. The catch is that a resilient architecture can become an expensive architecture if traffic starts bouncing between zones as a matter of routine. Many teams end up paying for cross zone data movement simply because the networking layer is doing the safe thing, spreading traffic across every healthy endpoint, even when an equally healthy option exists in the same zone. This post is about keeping the benefits of multi AZ while reducing unnecessary cross zone traffic. The goal is simple. Keep traffic local by default, and only cross zones when it is genuinely needed, for capacity, failover, or specific service requirements. Designing for locality in a multi AZ cluster You can think about this as choosing the smallest safe blast radius for each flow. Most platform teams want two things at the same time, strong resilience, and predictable spend. You get both when you keep everyday traffic close to where it starts, and only use cross zone paths when they are earning their keep. A practical way to apply that is a simple preference order. Same node, when the destination is a per node platform component or cache Same zone, when the destination is a replicated service with healthy backends nearby Any zone, when you need capacity, failover, or an explicit requirement to spread load wider This is not about forcing everything to stay local. It is about making cross zone traffic a deliberate choice. When you design for locality first, the platform still fails over across zones when it must, but it does not do it for routine requests that have a perfectly good local option. The hidden cost drivers in multi AZ clusters In most clusters, the extra spend is not caused by one big mistake, it comes from a build up of normal traffic patterns that become cross zone more often than anyone intends. East west traffic that crosses zones: Service to service calls are often the biggest driver. A chatty application can generate a large number of small requests, which quietly adds up when a portion of that traffic leaves the local zone. This gets worse when the cluster is large, the service mesh is adding extra hops, or retries amplify traffic during partial failures. Shared platform services that become remote by accident: Some of the noisiest traffic in a cluster is not your application traffic at all. It is platform plumbing. DNS queries, metrics scrapes, log shipping, telemetry exports, and even ingress controller backends can become cross zone if clients are routed to a backend that happens to live elsewhere. It still works, but you are paying for distance on every request. Egress paths that hairpin across zones: Egress is another common trap. A workload in one zone might send traffic to an egress point in another zone, then exit the cloud network from there. You get the same outcome, but the path includes a cross zone hop that you did not intend. Over time, that pattern can dominate the bill for a few high volume services. The Cilium feature set that enables cost conscious multi AZ Cilium gives platform teams a practical way to keep multi AZ as the default for resiliency, while avoiding cross zone traffic as the default for everyday requests. The approach is simple. Prefer the closest healthy option first, and keep a reliable fallback path when local capacity is not available.. Prefer the closest healthy option first, and keep a reliable fallback path when local capacity is not available. You apply this selectively, service by service, and you can prove the result with real traffic signals. Zone local routing for Services For many clusters, the biggest source of cross zone traffic is routine service traffic. A client in zone A calls a Service, the Service has healthy backends in zone A, but the request still lands on a backend in zone B often enough to add up. Kubernetes now gives you a clear way to express intent using trafficDistribution. When you set a Service to prefer the same zone, Cilium can enforce that preference in the dataplane so that, in normal conditions, requests stay within the same zone. If local endpoints are missing or unhealthy, the Service still works because it can fall back to endpoints in other zones. The below short video shows the feature in action when using Cilium with its Kube-Proxy replacement feature (Read more about why Kube-Proxy could be hindering the performance of your Kubernetes platform). Node local routing with Local Redirect Policy Some of the busiest traffic in a cluster is platform traffic, not application traffic. DNS, metrics, logging, and telemetry pipelines can generate a steady stream of requests across every node. When that traffic hops between nodes and zones, it can turn into cross zone traffic without anyone making that choice explicitly.1 Cilium Local Redirect Policy helps you keep this kind of traffic local when there is a local instance available. Instead of sending requests to an arbitrary backend somewhere else in the cluster, Cilium can redirect them to a backend on the same node. You cut out unnecessary hops on traffic that runs constantly, which is a win for latency, stability, and cost. DNS is the clearest example. If you run NodeLocal DNSCache on every node, most DNS queries should never need to leave the node. Local Redirect Policy lets you keep the normal kube-dns Service address, while ensuring queries are served by the local cache when it exists. If the cache misses, the request continues upstream as normal, so you keep correctness while improving locality. Local Redirect Policy first shipped in Cilium 1.9 and has since become a common pattern for teams that want to keep high volume platform traffic close to where it originates. Trendyol highlights this approach in its production story with Cilium, including NodeLocal DNSCache and Local Redirect Policy as part of its cluster performance improvements. To see what this looks like in practice, the video below walks through Local Redirect Policy and shows how it keeps traffic local while preserving the Service abstraction. Alternatively, get hands on with our “Cilium Traffic Optimization” Lab and test this feature out in our sandboxed environments. Zone aligned egress with Egress Gateway Egress is another common source of cross zone traffic. A workload in one zone sends traffic to an egress point in another zone, then exits the cloud network from there. The application sees no difference, but you have added a cross zone hop to every outbound connection. Egress Gateway lets you control where selected pod traffic exits the cluster. The cost aware way to use it in a multi AZ cluster is to design for zone alignment. Run gateway nodes in each zone, and have workloads exit through the gateway in their own zone by default. When a zone is unhealthy, you can intentionally move egress elsewhere as part of your incident response, rather than discovering after the fact that your steady state path was crossing zones all along. Raphael shows a demonstration of topology awareness configuration alongside the use of Cilium Egress Gateway. Guardrails with Bandwidth Manager Even with good locality, traffic can still spiral out of control when under load. A saturated link increases latency, latency triggers retries, and retries multiply traffic. What starts as a small performance dip can become sustained, high volume network usage that is painful to operate and expensive to absorb. Cilium Bandwidth Manager helps keep traffic behavior predictable when the cluster is under pressure. It lets you apply limits where it makes sense, so a small number of workloads cannot overwhelm links and trigger a retry storm across your platform. If you want the full technical walkthrough, including practical testing approaches you can reuse in your own environment, this post goes deep on how Bandwidth Manager works and how to validate it. To see the feature in action, the video below walks through the concepts and what to look for when you apply limits in a real cluster. Using Hubble to observe traffic patterns and changes When you tune locality in a multi AZ cluster, intent is easy, proving the outcome can be harder. Hubble gives you that proof by showing real network flows enriched with Kubernetes context, pods, namespaces, services, labels, and policy verdicts, so you can confirm where traffic actually went. That makes it straightforward to validate the changes in your Cilium configuration. You can see whether Service calls stayed in the same zone when local backends were available, whether node local dependencies stayed on node, and whether fallback to other zones happened only when it was needed. If something breaks, drops show up with clear reasons, so you can troubleshoot quickly. Here is a simple example using the Hubble CLI to compare traffic before and after enabling trafficDistribution on the echo service. With trafficDistribution enabled, you see the client consistently selecting the same nearby backends. With it disabled, the client cycles across different echo pods, including pods on other nodes and zones, which is exactly the pattern that quietly drives cross zone traffic. Hubble Timescape UI gives you a live service map that makes it easy to understand how services are communicating in real time, and it is equally useful for day to day platform observability as it is for validating changes during a rollout or incident. Hubble Timescape extends that experience with historical flow storage and fast search across a chosen time window, so you can correlate traffic changes with rollouts and incidents instead of relying on short retention buffers. With built-in RBAC, platform teams can make this a self-service tool for application teams, giving each team access to its own namespaces and workloads only, so they can understand service dependencies, spot drops, and validate policy outcomes without seeing anyone else’s traffic. Operational guidance Locality controls should make the platform safer and more cost effective, not harder to operate. Here are a few practical guidelines that will help you test and implement the discussed features and minimize operational interruption. Roll out in phases Start with opt in changes on a small number of Services, especially those with high request volume and replicated backends in each zone. Once the team trusts the behavior, expand the pattern to platform traffic, then to egress. Watch for skew If most clients sit in one zone, preferring the same zone backends can overload that zone’s capacity. The right response is usually to scale the workloads in that zone, or to improve workload spreading, rather than removing locality controls entirely. Always demonstrate fallback Every locality choice should have an obvious failure scenario: Remove local endpoints and confirm cross zone fallback Drain a node and confirm local behavior still holds Fail an egress gateway node and confirm the expected egress path Be explicit about exceptions Some paths should remain global by design. Shared data stores, stateful systems with strict topology requirements, and any component where locality would concentrate risk should be called out as exceptions. This helps platform engineers apply the patterns with judgment rather than as a blanket rule. Wrap up Multi AZ is the foundation for resilient Kubernetes platforms, but it can quietly increase spending when everyday traffic crosses zones without a clear reason. Cilium lets you keep multi AZ as the default while being intentional about locality. Prefer same zone routing for selected Services, keep platform traffic on nodes where it makes sense, and design egress so it does not hairpin across zones by accident. Cross zone routing remains available for capacity and failure, but it becomes the fallback, not the steady state. Learn More Isovalent Labs: Cilium Traffic Optimization Cilium Egress Gateway Isovalent Blogs: How to Deploy Cilium and Egress Gateway in Elastic Kubernetes Service (EKS) Topology Aware Routing and Service Mesh across Clusters with Cluster Mesh Isovalent Guides: Kubernetes Traffic Engineering for Network Engineers: Cilium Best Practices