How did Cilium get started? Thomas Graf, Co-Creator of Cilium, explores the foundation story of the Cilium open source project: how it got started, how it relates to eBPF superpowers, and what led to all major cloud providers adopting it as the default networking and security layer for at least one of their managed Kubernetes platforms. Cilium Introduction Welcome everybody. My name is Thomas Graf. I'm one of the creators of Cilium, and I'm also a co-founder of Isovalent, the company we created around Cilium. Today this session is about the history of Cilium: when we started it, why we started it, and how it evolved. First of all, what is Cilium? Most of you probably already have a good understanding of that, so I only have one slide on this in case somebody has never heard about it before. Cilium is a CNCF project at the incubation stage. It provides networking, security, observability, service mesh, ingress, and Gateway API support, and it does so primarily using eBPF. Whenever we can, we use eBPF, and we'll get to the story of why we do that. Sometimes we also use the Envoy proxy, in particular when we operate at layer 7. Cilium is widely deployed today across many different end users, and the logos on the slide are only a subset. There is a long list of Cilium users in the GitHub repository where you can find details on who is using Cilium, why they are using it, and what they are using it for. Initially, Cilium was created by engineers who later founded Isovalent. We then donated Cilium to the Cloud Native Computing Foundation, so Cilium today is owned by the CNCF. We of course have a variety of contributors who are not from Isovalent, and the set of people working on Cilium is very diverse today. At the foundation of Cilium is eBPF, and as we go through the history of Cilium you will see that this played a major role. What is eBPF? In fact, eBPF is the reason why we created Cilium. It is even more extreme than that: we actually continued working on eBPF for about a year to get it to the point where we could start building Cilium. So it is fair to say that we saw eBPF, we saw the potential, we kept developing it and making progress on it, and then about a year later we were finally at the point where we could use eBPF to build what we envisioned. So what is eBPF? Some of you may have seen a slide like this before. eBPF is the ability to make the Linux kernel, and now also the Windows kernel, programmable. It makes the Linux kernel programmable in a secure and efficient way. We can write programs, attach them to the kernel, and execute them when certain things happen in the kernel, for example when a process makes a system call or when a network packet is being received or transmitted. If you want to boil it down to one sentence that makes sense to most people, eBPF is to the Linux kernel what JavaScript is to the browser. JavaScript makes the browser programmable. eBPF makes the Linux kernel programmable. Conceptually that is similar, even though the implementation, performance, and environment are obviously very different. With JavaScript you might run some code when a website user clicks a submit button. For eBPF, you have an operating system and a process. When the process makes a system call, you can run an eBPF program. The code looks very different, but the basic idea is similar: programmable logic that reacts to events. Early eBPF work Cilium is not just one project. There is of course the CNI, which most of you have probably heard about before. This is the best-known Cilium project. It provides Kubernetes or container networking that is highly scalable, highly secure, and high performance. Then we have Cilium Service Mesh, which elevates Cilium to operate at layer 7 and implements a sidecar-free service mesh, as well as a fully conformant ingress controller and a Gateway API implementation. Many of you have also heard about Hubble, the network observability layer of Cilium. It requires Cilium underneath and provides extensive observability functionality with Grafana dashboards, Prometheus metrics, and more. Lastly there is Tetragon, the security observability and runtime enforcement layer that works with Cilium but can also run independently of the Cilium CNI. What connects all of these projects is eBPF. They are all eBPF-based and they all leverage eBPF to the greatest extent possible. We built all of these projects to exploit the power of eBPF. The origins of Cilium - Founding Team When did we start it, why did we start it, and what was the vision? This was the founding team all the way back then. It was actually the first-ever Cilium design summit at a place called Diavolezza in Switzerland. It is high up in the mountains, and the weather was not really great. You can see the founding team there: Andre Martins, Daniel Borkmann, Mårten, and myself. I took the picture, so I am not actually in it. We did a social expedition into the mountains as a team activity while we were designing the first version of Cilium, and it was in that moment that we realized observability probably matters as well, because the visibility was absolutely horrendous. That was the founding team back in 2016 when we wrote the first lines of Cilium code. Kubernetes Networking The original vision was very basic and very simple. Kubernetes was already on the rise, but back then there was also Docker, so we had support for both Docker and Kubernetes and treated them equally. In this initial phase we focused exclusively on networking: container networking and layer 3 and layer 4 load balancing. We had a very clear vision in mind. We wanted it to be intent- and identity-based, high performance, scalable, and focused on container networking. Around that time, in 2016, a lot of the industry was moving away from OpenStack and traditional virtualization. Containers, Docker, and Kubernetes were rising very quickly, and it became clear that the existing solutions would not scale to the demands of containers. This is also where Thomas explains that Cilium operates all the way from layer 2 to layer 7. It mostly operates at layer 3 and layer 4, but it can also operate at layer 2. For example, it can do multicast at L2 and ARP passthrough, and it can also operate at layer 7. The first time we talked publicly about Cilium was at LinuxCon 2016 in Toronto. We positioned Cilium as fast IPv6 container networking with BPF and XDP. In fact, the first version was IPv6-only. We did not even implement IPv4 support at all. We were convinced we could provide connectivity without requiring end users to understand or manage subnets directly. We already had policy and network security decoupled from addressing, which is the identity-based security model, and it was built around a giant flat L3 network. Today we take that for granted, but back then it was still a relatively new concept. The basic architecture from that period is still how Cilium operates at its foundation today, even though it is of course more sophisticated now. Network Security & Encryption Fast forward a little bit: the next major problem we tackled was network security from layer 3 to layer 7. We did not just build a firewall for container network policy enforcement, segmentation, or network firewalling. We built support all the way up to layer 7, including HTTP, Kafka, gRPC, and more. We also built in encryption, initially using IPsec, to transparently encrypt network traffic. That was effectively our network security 1.0. That first larger feature set led to the famous DockerCon 2017 talk, which was the first really big conference talk we did. DockerCon 2017 was also a major moment in the rise of Kubernetes, so Cilium and Kubernetes share quite a bit of history there. That was also when we founded Isovalent, about a year after creating Cilium. Around this time we also created the modern Cilium logo. Before that it looked different and was more rounded. We moved to the hexagonal shape that everyone recognizes today. The next major step was Hubble, which provided network observability. Back then, observability was often an afterthought. The only practical option was to run tcpdump on a node and try to figure out what was happening, which was horrible for container networking and Kubernetes at scale. Hubble changed that. It can run in the Kubernetes cluster, see network activity, provide flow logs, explain what is being dropped and why, and collect metrics so you can understand traffic volume, DNS failures, HTTP request latency, and more. We also added integrations so that we could extract the security-relevant portion of this network observability data and feed it into a SIEM for security teams to query. In 2018 we continued to grow. We celebrated two thousand GitHub stars, though of course we waited until 2048 because we are engineers after all. Around that time we also built Cluster Mesh, the ability to connect multiple Kubernetes clusters together while preserving features like network security and observability. It also introduced global services so you could do load balancing across multiple clusters. That extended Cilium east-west. Instead of handling a single cluster, we could handle multiple clusters. We demoed this at KubeCon by connecting a Kubernetes cluster running in Google Cloud with one running in AWS across multiple VPCs. The culture was always important as well. Thomas talks about fondue, skiing, Zurich dinners, and the fact that the project always kept its Swiss mountain roots. By 2021 things started to get really serious. This was when Cilium joined the CNCF in the cloud-native network category. It was also the year when Google picked Cilium as the default networking layer for both GKE and Anthos, and Amazon picked Cilium for EKS Anywhere. Then things got even bigger when Microsoft adopted Cilium as part of Azure CNI powered by Cilium. At that point, all three major cloud providers had at least one managed Kubernetes offering using Cilium by default. That momentum allowed Isovalent to continue developing both the open source project and the enterprise offering. Last year was also when we introduced Tetragon. That took us beyond the network layer and into runtime security and runtime observability. Tetragon lets you observe system calls, file access, privilege escalation, and deeper system activity, including network behavior that is not visible on the wire and instead requires instrumentation inside the stack. It was not just observability either. Tetragon also added real-time runtime enforcement. You can define what an application is allowed to do and what it is not allowed to do, and Tetragon can enforce those policies. You can run Tetragon independently, but if you run it together with Cilium and Hubble they all use data from each other and provide a better overall security posture. Thomas also explains that Tetragon exposes rich observability through eBPF probes in the kernel and integrates with Prometheus, Grafana, SIEMs, Fluentd, Elasticsearch, OpenTelemetry, and more. It can provide visibility into TLS handshakes, process execution, process trees, and file integrity monitoring. He also shares a lighter moment from a team ski trip in Switzerland, including the famous snow-chain incident where three engineers tried to figure out how to mount chains on a mountain pass while one person watched YouTube and another recorded the whole thing. Another major step was the load balancer. Up to that point, Cilium handled load balancing inside the Kubernetes cluster or across clusters. With the standalone load balancer, you could run a Cilium-based load balancer in front of the cluster and balance traffic across multiple clusters or do canary routing, while still gaining observability and policy control. Then came Timescape. Users kept asking how to store all of this observability data over time. They wanted a time machine that could answer questions like what happened two weeks ago or three hours ago, regardless of where Cilium was running. Timescape, or Hubble Timescape, is a ClickHouse-based time series database that can ingest observability data from the load balancer, Cilium CNI, Hubble, and Tetragon across many clusters. On top of that, you can run Hubble UI dashboards or analytics queries directly. Thomas also explains why you might choose Timescape instead of Grafana Mimir. Mimir is fully compatible and great for persistent flow logging, but Timescape understands the observability data schema more deeply, which makes it more efficient for richer analytics and security investigations. The next major innovation cycle was service mesh. Users said they already loved the CNI layer and observability and asked why Cilium was not also a service mesh. The main pain point was the sidecar model used by traditional service meshes. So Cilium introduced service mesh as a feature you can enable directly, without requiring sidecars for many use cases. In the first iteration, the main focus was layer 7 load balancing and tracing, because Cilium already had network policy and encryption. Users can now use Ingress, Gateway API, or service annotations to perform layer 7 load balancing and enable HTTP tracing that can be exported as metrics or through OpenTelemetry. Thomas emphasizes that the Cilium service mesh differs from many alternatives in two ways. First, it removes the need to run one proxy per pod. Second, for many use cases it can run eBPF natively with no proxy at all. That means no connection redirection, no connection termination, and no sidecar overhead when eBPF can handle the data path directly. Where a proxy is still needed, for example for TLS termination or origination when Cilium only controls one side, Envoy is used as the fallback. He also explains why Tetragon differs from other eBPF-based runtime security tools. Many tools only collect observability in the kernel and then react in user space. Tetragon pushes much more of the logic into the kernel itself, which enables real-time enforcement and more efficient observability. The talk then looks forward to what Thomas calls network security 2.0. This combines service mesh and network security more tightly. The goal is not just to add mutual TLS, but to bring authentication directly into network policy. In other words, service mesh-style identity and authentication should not be limited to workloads that run a service mesh. It should be applicable across the network. This also extends to the load balancer through Gateway API support, and eventually to more centralized orchestration of policy and load balancing across multiple clusters. Toward the end, Thomas leaves the audience with a couple of pointers: cilium.io for the open source project, and isovalent.com for resources such as interactive labs, webinars, and information about the enterprise distribution. The Q and A covered a wide range of topics, including web3 and peer-to-peer networking, Windows support, the role of Envoy as a fallback when eBPF cannot manage a feature natively, Hubble visibility across multiple clusters through Timescape or Hubble Relay, service mesh across clusters, profiling with eBPF, sidecars versus shared node-level proxies, guardrail network policies for security teams, transparent web proxy handling, smart NIC offload, and the public roadmap. Thomas closes by encouraging people to join the Cilium Slack, reach out with questions, and keep following the project as it continues to evolve.