Over 14,000 GitHub stars! Cilium’s popularity keeps on growing. If you’re new to the world of Cilium – like I was myself a year ago – the first thing you’ll want to do is install it. What I’ve discovered over the past 12 months is that there are many different ways to deploy Cilium! Whether you’re a newcomer to Cilium or an experienced user, I hope you’ll learn in this post some valuable tips and tricks to help you on your Cilium journey.
A privilege of my job is that I get to work closely with the engineers that are building and improving Cilium on a day-to-day basis. And when they release a new feature, well, I often want to be amongst the first ones to test it out.
Which means I often have to install the very latest Cilium version: not necessarily a mainline version (i.e 1.12.0), or even a release candidate (i.e 1.13.0-RC1). Release candidates are released on a regular basis in the run-up to the main release, to let users test some of the new features. Sometimes I have to test and deploy a Cilium version that has only just been pushed, either because it includes a bug fix or because it has a feature I really want to play with.
I learned a few things along the way and wanted to share some of my learnings on this post. Let’s go through the options at our disposal.
The “Easy” Way
Of course, you should always start with the official documentation when installing Cilium and this post doesn’t intend to replace the official docs: it’s rather a collection of tips and tricks.
The docs focus primarily on the two most commonly used installation tools for Cilium: cilium-cli and helm. We will cover both tools in this post.
Helm is the popular Kubernetes package manager, while cilium-cli is a purpose-built tool to install and manage Cilium. We’ll start with cilium-cli first. You will see how we can use either tool, or even both of them together.
I would recommend most first-time users to install Cilium with the cilium-cli tool and its command cilium install. It would install the latest stable release by default:
nicovibert:~$ cilium install🔮 Auto-detected Kubernetes kind: kind
✨ Running "kind" validation checks
✅ Detected kind version "0.12.0"ℹ️ using Cilium version "v1.11.3"🔮 Auto-detected cluster name: kind-kind
🔮 Auto-detected IPAM mode: kubernetes
🔮 Auto-detected datapath mode: tunnel
ℹ️ helm template --namespace kube-system cilium cilium/cilium --version 1.11.3 --set cluster.id=0,cluster.name=kind-kind,encryption.nodeEncryption=false,...
ℹ️ Storing helm values filein kube-system/cilium-cli-helm-values Secret
🔑 Created CA in secret cilium-ca
🔑 Generating certificates for Hubble...
🚀 Creating Service accounts...
🚀 Creating Cluster roles...
🚀 Creating ConfigMap for Cilium version 1.11.3...
🚀 Creating Agent DaemonSet...
🚀 Creating Operator Deployment...
⌛ Waiting for Cilium to be installed and ready...
✅ Cilium was successfully installed! Run 'cilium status' to view installation health
Some configuration options can be specified with cilium install. For example, cilium install --encryption wireguard can be used to enable transparent encryption with WireGuard, as described in this tutorial and this lab.
But if you want to fully customize your installation, you should use cilium-cli with helm-set (to specify configuration values on the command line based on Helm values) or helm-values to pull these values out of a YAML file. For example, this is what we use to install Cilium with BGP enabled in our BGP lab:
You can see in the code snippet above that we are enabling features that are disabled by default (like BGP Control Plane).
The “Specific Release” Way
The examples shown so far install the latest stable release but the cilium-cli also lets you install a specific Cilium release. You can even check which Cilium version the cilium-cli supports with this command:
nicovibert:~$ cilium install --list-versions
v1.13.0-rc4
v1.13.0-rc3
v1.13.0-rc2
v1.13.0-rc1
v1.13.0-rc0
v1.12.5 (default)v1.12.4
v1.12.3
v1.12.2
[edited for brevity - there were about 120 versions!]v1.6.10
v1.6.9
v1.6.8
v1.6.7
v1.6.6
v1.6.5
v1.9-dev
v1.8-dev
v1.7-dev
v1.6-dev
By default, Cilium will install the stable release (at time of writing, v1.12.5). I can specify another version from that list, for example v1.11.1:
root@server:~# cilium install --version=v1.11.1🔮 Auto-detected Kubernetes kind: kind
✨ Running "kind" validation checks
✅ Detected kind version "0.17.0"ℹ️ Using Cilium version 1.11.1
🔮 Auto-detected cluster name: kind-kind
🔮 Auto-detected datapath mode: tunnel
🔮 Auto-detected kube-proxy has been installed
ℹ️ helm template --namespace kube-system cilium cilium/cilium --version 1.11.1 --set cluster.id=0,cluster.name=kind-kind,encryption.nodeEncryption=false,ipam.mode=kubernetes,kubeProxyReplacement=disabled,operator.replicas=1,serviceAccounts.cilium.name=cilium,serviceAccounts.operator.name=cilium-operator,tunnel=vxlan
ℹ️ Storing helm values filein kube-system/cilium-cli-helm-values Secret
🔑 Created CA in secret cilium-ca
🔑 Generating certificates for Hubble...
🚀 Creating Service accounts...
🚀 Creating Cluster roles...
🚀 Creating ConfigMap for Cilium version 1.11.1...
🚀 Creating Agent DaemonSet...
level=warning msg="spec.template.spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[1].matchExpressions[0].key: beta.kubernetes.io/os is deprecated since v1.14; use \"kubernetes.io/os\" instead"subsys=klog
level=warning msg="spec.template.metadata.annotations[scheduler.alpha.kubernetes.io/critical-pod]: non-functional in v1.16+; use the \"priorityClassName\" field instead"subsys=klog
🚀 Creating Operator Deployment...
⌛ Waiting for Cilium to be installed and ready...
✅ Cilium was successfully installed! Run 'cilium status' to view installation health
You will notice that cilium-cli auto-detected the type of environment (kind in the example above) but it also works with AKS, EKS, etc… Cilium is indeed deployed differently depending on the type of underlying platform it’s built upon.
Detecting the environment is certainly a great benefit of cilium-cli and is useful even if you would rather use Helm: as you can see above, cilium-cli uses helm template in its backend to generate the manifests. If you look at the terminal snippets above and below, cilium-cli even gives the user the helm template command to use if they just want to use cilium-cli to auto-detect the right flags for their clusters.
In other words – you can use cilium-cli for its ease, helm because it’s commonly used for app deployment, or finally you can use both, by using cilium-cli to auto-detect the environment values and to help you set the adequate Helm values.
In addition, cilium-cli also includes a way to auto-generate Helm values in a file, with --helm-auto-gen-values. This flag effectively gives you a dry-run option: it generates the values, but doesn’t actually install it (you can refer to these values when deploying Cilium with Helm later on):
I can then install Cilium with Helm by referring to the Helm value file created by cilium-cli:
root@server:~# cilium install --version=v1.11.1 --encryption wireguard --helm-values helm-values.yaml 🔮 Auto-detected Kubernetes kind: kind
✨ Running "kind" validation checks
✅ Detected kind version "0.17.0"ℹ️ Using Cilium version 1.11.1
🔮 Auto-detected cluster name: kind-clab-bgp-cplane-devel
🔮 Auto-detected datapath mode: tunnel
🔮 Auto-detected kube-proxy has been installed
ℹ️ L7 proxy disabled due to Wireguard encryption
ℹ️ helm template --namespace kube-system cilium cilium/cilium --version 1.11.1 --set cluster.id=0,cluster.name=kind-clab-bgp-cplane-devel,encryption.enabled=true,encryption.nodeEncryption=false,encryption.type=wireguard,ipam.mode=kubernetes,kubeProxyReplacement=disabled,l7Proxy=false,operator.replicas=1,serviceAccounts.cilium.name=cilium,serviceAccounts.operator.name=cilium-operator,tunnel=vxlan
ℹ️ Storing helm values filein kube-system/cilium-cli-helm-values Secret
🔑 Created CA in secret cilium-ca
🔑 Generating certificates for Hubble...
🚀 Creating Service accounts...
🚀 Creating Cluster roles...
🚀 Creating ConfigMap for Cilium version 1.11.1...
🚀 Creating Agent DaemonSet...
level=warning msg="spec.template.spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[1].matchExpressions[0].key: beta.kubernetes.io/os is deprecated since v1.14; use \"kubernetes.io/os\" instead"subsys=klog
level=warning msg="spec.template.metadata.annotations[scheduler.alpha.kubernetes.io/critical-pod]: non-functional in v1.16+; use the \"priorityClassName\" field instead"subsys=klog
🚀 Creating Operator Deployment...
⌛ Waiting for Cilium to be installed and ready...
✅ Cilium was successfully installed! Run 'cilium status' to view installation health
Here is another example where I install Cilium by referring to them in the command-line instead of in a separate YAML file. This is what I used to install one of the recent release candidates version to test SCTP support on Cilium.
As you can see, I use helm instead of cilium-cli to install the specific version. Note I specify the version 1.13.0-rc3 (SCTP support was introduced in RC1) and I also enabled SCTP and other additional features that are disabled by default, such as Hubble.
The “CI Build Image” Way
Where it gets more complex is working out how to install a version of Cilium that includes a particular fix or a new feature, but that is not yet available as an official release.
For example, I wanted to test a Gateway API feature that lets us load-balance traffic across backends. During my tests, I identified a bug which was quickly resolved by Tam, one of our rock-star developers working on Cilium. Once Tam got his PR merged, I was able to use the images generated during the CI process and published on the container image repository quay.io (with the Cilium images published on quay.io/cilium).
First, I had to get the details of the images generated during the build process:
What I copied at the end of this short video was this image location and its digest: quay.io/cilium/cilium-ci:db5b10436fa730de83275655d8f00e2e64db5e45.
To install this version, all I need is to reference these specific images for both the Cilium agent and the Cilium operator. Note that in my case, I also had to make sure the Helm chart was correct, which is why I had to clone the repo locally to use the latest Helm chart:
The end result was a demo I posted on LinkedIn – you can also see it below:
Using cilium-cli and the helm-set flags, we can apply the same configuration with cilium install instead:
root@server:~# cilium install --helm-set=image.repository=quay.io/cilium/cilium-ci --helm-set=image.useDigest=false --helm-set=image.tag=db5b10436fa730de83275655d8f00e2e64db5e45 --helm-set=operator.image.useDigest=false --helm-set=operator.image.repository=quay.io/cilium/operator --helm-set=operator.image.suffix=-ci --helm-set=operator.image.tag=db5b10436fa730de83275655d8f00e2e64db5e45🔮 Auto-detected Kubernetes kind: kind
✨ Running "kind" validation checks
✅ Detected kind version "0.17.0"ℹ️ Using Cilium version 1.12.2
🔮 Auto-detected cluster name: kind-kind
🔮 Auto-detected datapath mode: tunnel
🔮 Auto-detected kube-proxy has been installed
ℹ️ helm template --namespace kube-system cilium cilium/cilium --version 1.12.2 --set cluster.id=0,cluster.name=kind-kind,encryption.nodeEncryption=false,image.repository=quay.io/cilium/cilium-ci,image.tag=db5b10436fa730de83275655d8f00e2e64db5e45,image.useDigest=false,ipam.mode=kubernetes,kubeProxyReplacement=disabled,operator.image.repository=quay.io/cilium/operator,operator.image.suffix=-ci,operator.image.tag=db5b10436fa730de83275655d8f00e2e64db5e45,operator.image.useDigest=false,operator.replicas=1,serviceAccounts.cilium.name=cilium,serviceAccounts.operator.name=cilium-operator,tunnel=vxlan
ℹ️ Storing helm values filein kube-system/cilium-cli-helm-values Secret
🔑 Created CA in secret cilium-ca
🔑 Generating certificates for Hubble...
🚀 Creating Service accounts...
🚀 Creating Cluster roles...
🚀 Creating ConfigMap for Cilium version 1.12.2...
🚀 Creating Agent DaemonSet...
🚀 Creating Operator Deployment...
⌛ Waiting for Cilium to be installed and ready...
✅ Cilium was successfully installed! Run 'cilium status' to view installation health
root@server:~# cilium status /¯¯\ /¯¯\__/¯¯\ Cilium: OK
\__/¯¯\__/ Operator: OK
/¯¯\__/¯¯\ Hubble: disabled
\__/¯¯\__/ ClusterMesh: disabled
\__/
Deployment cilium-operator Desired: 1, Ready: 1/1, Available: 1/1
DaemonSet cilium Desired: 3, Ready: 3/3, Available: 3/3
Containers: cilium Running: 3 cilium-operator Running: 1Cluster Pods: 4/4 managed by Cilium
Image versions cilium-operator quay.io/cilium/operator-generic-ci:db5b10436fa730de83275655d8f00e2e64db5e45: 1 cilium quay.io/cilium/cilium-ci:db5b10436fa730de83275655d8f00e2e64db5e45: 3
You don’t always have to use the Helm flag options with the cilium-cli: you can instead use some of the built-in options. For example, instead of using the --helm-set=operator.image, we can use the simpler --operator-image flag instead. Check out the full list of options with cilium install -h.
nicovibert:~$ cilium install -h
Install Cilium in a Kubernetes cluster
[edited for brevity]Flags:
--agent-image string Image path to use for Cilium agent
--api-versions strings Kubernetes API versions to use for helm's Capabilities.APIVersions incase discovery fails
--azure-client-id string Client (application) ID of Azure Service Principal to use for installing Cilium (will create one if none provided) --azure-client-secret string Client secret of Azure Service Principal to use for installing Cilium (will create one if none provided) --azure-resource-group string Azure resource group name the cluster is in(required) --azure-subscription az account show Azure subscription name the cluster is in(default az account show) --azure-tenant-id string Tenant ID of Azure Service Principal to use for installing Cilium (will create one if none provided) --chart-directory string Helm chart directory
--cilium-ready-timeout duration Timeout for Cilium to become ready before restarting unmanaged pods (default 5m0s) --config strings Set ConfigMap entries {key=value[,key=value,..]} --datapath-mode string Datapath mode to use { tunnel | aws-eni | gke | azure | aks-byocni }(default: autodetected).
--disable-check strings Disable a particular validation check
--encryption string Enable encryption of all workloads traffic { disabled | ipsec | wireguard }(default "disabled") --helm-auto-gen-values string Write an auto-generated helm values into this file --helm-set stringArray Set helm values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2) --helm-set-file stringArray Set helm values from respective files specified via the command line (can specify multiple or separate values with commas: key1=path1,key2=path2) --helm-set-string stringArray Set helm STRING values on the command line (can specify multiple or separate values with commas: key1=val1,key2=val2) --helm-values strings Specify helm values in a YAML file or a URL (can specify multiple) --helm-values-secret-name string Secret name to store the auto-generated helm values file. The namespace is the same as where Cilium will be installed (default "cilium-cli-helm-values") -h, --help helpforinstall --image-suffix string Set all generated images with this suffix
--image-tag string Set all images with this tag
--inherit-ca string Inherit/import CA from another cluster
--k8s-version string Kubernetes server version incase auto-detection fails
--list-versions List all the available versions without actually installing
--node-encryption Enable encryption of all node to node traffic
--nodes-without-cilium strings List of node names on which Cilium will not be installed
--operator-image string Image path to use for Cilium operator
--restart-unmanaged-pods Restart pods which are not being managed by Cilium (default true) --rollback Roll back installed resources on failure (default true) --version string Cilium version to install(default "v1.12.2") --wait Wait for status to report success (no errors)(default true) --wait-duration duration Maximum time to waitfor status (default 5m0s)[edited for brevity]
The “Latest” Way
If you don’t want a specific image build but rather the very latest version, you can simply use the -version=latest. Here is a quick demo:
That’s a really useful command and one I wish I had discovered earlier!
There is another obvious method to run the latest Cilium version: build it from the source code.
The “Compiled from Source” Way
Finally, let’s cover what might look like the most intimidating method on paper: compiling and installing Cilium from source. It turned out to be easier than expected, with a little help.
As soon as the most recent BGP feature was released (covered at length in this eCHO episode), I wanted to try it out for myself. I leveraged some scripts from the Cilium repo to simplify it. Once I had installed Go and Make on my machine, I was able to successfully deploy the latest version of Cilium on a Kind cluster, using this very handy script:
It only took 5 minutes from start to finish, as you can see in the video below:
Summary
In this tutorial, I shared some tips and tricks I learned over the course of the past year deploying many different Cilium versions across different environments for very different use cases. I shared a few different options on how Cilium can be installed and why you might want to use a method over another or why you might use them together!
Nico Vibert is a Senior Staff T echnical Marketing Engineer at Isovalent, the company behind the open- source cloud- native solution Cilium.
Prior to joining Isovalent, Nico worked in many different roles – operations and support, design and architecture, technical pre- sales – at companies such as HashiCorp, VMware, and Cisco.
In his current role, Nico focuses primarily on creating content to make networking a more approachable field and regularly speaks at events like KubeCon, VMworld, and Cisco Live.
Nico has held over 15 networking certifications, including the prestigious Cisco Certified Internetwork Expert CCIE (# 22990).
Nico is now the Lead Subject Matter Expert on the Cilium Certified Associate (CCA) certification.
Outside of Isovalent, Nico’s passionate about intentional diversity & inclusion initiatives and is Chief DEI Officer at the Open Technology organization OpenUK. You can find out more about him on his blog.
