Introduction

In today’s cloud-native landscape, enterprise networking is undergoing a transformation that demands both agility and robustness. As organizations continue to build complex, distributed systems, connecting Kubernetes to existing networks can be challenging.

Enter Cilium’s advanced Border Gateway Protocol (BGP) implementation, powered by the GoBGP control plane, a solution that not only addresses these challenges but also adds unprecedented flexibility to your network configurations. In addition to BGP, Cilium offers the Load Balancer IP Address Management (LB IPAM) feature, a game-changer for enterprises requiring dynamic IP allocation and multi-tenancy. This blog post aims to demystify Cilium’s BGP and LB IPAM features, highlighting their technical properties and the business value they bring to modern, cloud-native infrastructures. Whether you’re dealing with on-premise deployments or a multi-cloud strategy, read on to discover how Cilium can elevate your networking capabilities and learn about BGP features such as multihop and graceful restart.

GoBGP: The Control Plane Powering Cilium’s BGP Implementation

At the heart of Cilium’s BGP capabilities lies GoBGP, a modern control plane designed to provide robust, scalable, and easily customizable BGP solutions supporting both IPv4 and IPv6. GoBGP enables Cilium to create and manage BGP sessions with routers in your network infrastructure, thereby allowing for a flexible and efficient routing setup. Utilizing GoBGP’s feature-rich toolkit, Cilium provides not only BGP session establishment and termination but also advanced routing policies and configurations. From a business perspective, the integration of GoBGP into Cilium ensures that enterprises can achieve optimized network routing, reduced latency, and enhanced redundancy, all while maintaining a simplified, unified control plane for their cloud-native deployments.

Intro to BGP on Cilium lab

Learn how to connect your Kubernetes Clusters with your on-premises network using BGP !

Start BGP on Cilium Lab

LB IPAM: Empowering Multi-Tenancy and Dynamic IP Allocation in Cloud-Native Networking

Cilium’s Load Balancer IP Address Management (LB IPAM) feature is an advanced capability designed to streamline the allocation of IP addresses in a multi-tenant environment. Utilizing a label-based selector mechanism, LB IPAM allows businesses to allocate IP addresses from different pools for different workloads or tenants, providing a fine-grained control that is often required in complex, large-scale deployments. Beyond its application in multi-tenancy, this feature also enhances network security by compartmentalizing traffic and reducing the risk surface. From an operational standpoint, the LB IPAM feature simplifies IP management tasks, reduces configuration overhead, and enables efficient use of address space, while providing significant business value in terms of operational efficiency and security compliance. With LB IPAM, enterprises can create more agile, secure, and highly customizable networking configurations, aligned perfectly with their specific operational needs and regulatory requirements.

Example of a CiliumBGPPeeringPolicy CRD configuring Cilium to peer with two routers. 

apiVersion: "cilium.io/v2alpha1"
kind: CiliumBGPPeeringPolicy
metadata:
  name: blue-peering-policy
spec:
  nodeSelector:
    matchLabels:
      bgp-policy: blue
  virtualRouters:
  - localASN: 64512
  exportPodCIDR: true
  neighbors:
  - peerAddress: '10.0.0.1'
    peerASN: 64512
  - peerAddress: '10.0.0.2'
    peerASN: 64512

Example of a CiliumLoadBalancerIPPool CRD. Note the serviceSelector: configuration to select Service with specific labels. 

apiVersion: "cilium.io/v2alpha1"
kind: CiliumLoadBalancerIPPool
metadata:
  name: "blue-pool"
spec:
  cidrs:
  - cidr: "192.0.2.0/24"
  - cidr: "2001:db8:1:1::1/128"
  serviceSelector:
    matchExpressions:
      - {key: color, operator: In, values: blue}

Example of a Service CRD that will be assigned an IP address from blue-pool CiliumLoadBalancerIPPool based on its labels:

apiVersion: v1
kind: Service
metadata:
  name: service-blue
  namespace: blue
  labels:
    color: blue
spec:
  type: LoadBalancer
  ports:
  - port: 80

LB-IPAM and BGP on Cilium lab

Learn how to use the LoadBalancer IPAM and BGP Service Advertisement features!

Start LB-IPAM and BGP on Cilium Lab

Introduction to BGP Enhancements in Cilium

The rapidly evolving world of cloud-native networking requires solutions that are not only robust but also adaptable to complex routing scenarios. Recognizing these demands, Cilium has introduced a series of enhancements to its BGP capabilities, designed to offer both operational flexibility and high performance. Read on to explore how these BGP enhancements can transform your enterprise networking experience.

BGP Graceful Restart: Ensuring High Availability and Seamless Operations

One of the often-overlooked yet critical features for enterprise-level networking is BGP Graceful Restart. This capability ensures that BGP sessions can recover smoothly from disruptions or system failures without causing an immediate drop in data forwarding. With Cilium, BGP Graceful Restart provides a dual advantage: it maintains the stability of routing information during rolling updates or unplanned downtimes, and it ensures that traffic flows remain uninterrupted. For enterprises, this translates to higher availability and a superior end-user experience, even in the face of network instability or system failures. The benefit extends beyond just reliability; it also adds a layer of operational resilience, allowing you to perform necessary system maintenance or updates without incurring downtime. Therefore, BGP Graceful Restart becomes an indispensable feature for enterprises that aim for a zero-downtime, high-availability networking environment in their data centers.

Cilium BGP Daemon has BGP Session Established with a Router. 

Without Graceful Restart configured, restarting Cilium will result in the BGP session being lost. 

With Graceful Restart configured, restarting Cilium will not result in the routes being removed in the ToR (Router-A) as the ToR will keep the routes in the BGP table for a limited period. Note the routes are being marked as Stale.

Custom BGP Timers in Enterprise Networking: Flexibility, Efficiency, and Considerations

Custom BGP Timers offer a layer of fine-grained control over BGP session parameters, making them particularly valuable for enterprises seeking to optimize their network performance. By adjusting timers like HoldTime and KeepAlive, organizations can tailor the frequency of BGP message exchanges to match the specific requirements of their networking environment. For instance, a reduced KeepAlive interval can facilitate quicker detection of network failures, enabling faster rerouting of traffic and thus minimizing service disruption. Similarly, extending HoldTime may be useful in scenarios where network stability is preferred over quick failure detection.

While the ability to customize BGP timers brings flexibility and performance optimization, it comes with its own set of caveats. Changing timer values can impact the overall stability and behavior of the BGP sessions. A shortened HoldTime might lead to frequent session resets if the network is experiencing minor instabilities, adding unnecessary overhead.

In conclusion, custom BGP Timers in Cilium offer a valuable tool for enterprises to fine-tune their networking according to their unique operational needs. However, caution and thorough testing are advised when modifying these timers to ensure they align well with your overall networking strategy and do not inadvertently introduce instability or inefficiencies.

BGP Multihop Support: Extending Cilium’s Capabilities for Complex Network Topologies

The integration of BGP Multihop support within Cilium adds another layer of sophistication and utility for enterprises dealing with intricate, multi-layered network topologies. In a standard networking setup, eBGP is constrained to a single hop, which can limit flexibility in larger, more complex environments. BGP Multihop support in Cilium breaks this limitation, allowing BGP sessions to be established over multiple hops, thereby making it possible to interact with BGP routers that are not directly connected. This functionality is particularly beneficial for enterprises that have multiple routing layers or segregated network zones, as it provides a more seamless way to manage and propagate routes across diverse network segments.

From a business perspective, BGP Multihop support not only enhances routing flexibility but also simplifies network management tasks, reducing the operational complexity and costs associated with maintaining large-scale, intricate network infrastructures.

Preserving the Original Client Source IP

The externalTrafficPolicy: Local setting in Kubernetes Services provides a valuable mechanism for optimizing client traffic routing, especially in scenarios where preserving the client’s source IP is crucial. When this policy is set to Local, traffic is only directed to local endpoints within the same node where the incoming connection was received and the source IP of the client is preserved. In default Kubernetes, If there are no local endpoints, the traffic is dropped instead of being forwarded to another node. 

When using Cilium’s BGP Control Plane, what happens instead is that any node that does not run any endpoint will stop advertising LoadBalancer IPs and therefore will not attract any external traffic. 

This feature is particularly useful for applications that require real client IP for functionalities like access controls or logging. By enabling this policy, enterprises can benefit from improved network performance and more accurate request attribution, which in turn, enhances security and analytics capabilities. It’s a feature that combines well with Cilium’s BGP capabilities, especially when managing ingress traffic in complex enterprise architectures. 

While externalTrafficPolicy: Local offers advantages in terms of network optimization and preserving client source IPs, it comes with potential drawbacks when used in unbalanced pod deployments. In scenarios where the distribution of pods across nodes is uneven, setting this policy to Local can result in suboptimal load balancing as nodes receive the same share of traffic while some have more pods scheduled than other nodes. 

Therefore, while externalTrafficPolicy: Local offers several advantages, its effective use requires careful planning and monitoring, particularly when dealing with dynamic scaling or unbalanced pod deployments.

Advanced BGP Lab

Cilium 1.14 introduced advanced BGP features: BGP timers, eBGP multihop and BGP Graceful restart!

SRv6 and L3VPN: Enhancing Cilium’s Networking Capabilities with BGP Integration

Segment Routing over IPv6 (SRv6) represents a milestone in Isovalent Enterprise for Cilium’s advanced networking feature set, particularly when it comes to Layer 3 VPNs (L3VPN). SRv6 enables a more simplified, scalable, and programmable networking model by embedding routing information directly into the IPv6 headers. This allows for the efficient steering of packets through specified network paths, which is crucial for VPN configurations. When integrated with Cilium’s BGP Control Plane, SRv6 can provide a comprehensive L3VPN solution that is both agile and robust. BGP can be used for advertising SRv6-enabled routes, facilitating the interoperation of SRv6 and traditional IP networks. This combination gives enterprises the ability to implement flexible VPN solutions that are highly configurable and aligned with modern cloud-native architectures. The seamless integration of SRv6 with BGP in Cilium’s ecosystem ensures that businesses can deploy L3VPNs without sacrificing performance, scalability, or security, thereby offering a cutting-edge solution that meets the demands of contemporary enterprise networking.

Conclusion

Cilium’s enhanced BGP capabilities provide a robust foundation for modern, cloud-native enterprise networking. From BGP Graceful Restart for higher reliability to custom BGP timers for fine-tuned control, these features offer both performance optimization and operational simplicity. Coupled with advanced options like LB IPAM  and SRv6 for L3VPN, Cilium is setting the standard for next-generation networking solutions.

Learn More

• Lab: BGP on Cilium
• Lab: Advanced BGP Features
• Lab: Cilium LoadBalancer IPAM and BGP Service Advertisement
• YouTube: Cilium BGP Graceful Restart
• YouTube: Cilium BGP Timers
• YouTube: Cilium LB IPAM 
• YouTube: Come BGP with Me (From CiliumCon)
• Documentation: Cilium BGP Control Plane
• Documentation: Cilium LB IPAM