Learn how to combine Canary Tokens and Kernel events using Tetragon. Trigger your canary tokens to alert of potential malcious intent, from events read from the Kernel. Introduction Hello everyone. Today we're going to cover the Isovalent blog post tutorial on setting up a cybersecurity honeypot with Tetragon to trigger Canary Tokens. In my lab today I already have Cilium installed so I can run a couple of containers, and I also have a demo application installed. That demo app is the same application we use in the tutorial itself. I’m essentially going to walk through the same information that’s in the blog post, but now you can see those actions happen in real time. Security observability The first thing I want to do is get Tetragon installed. If you go to the Tetragon website, you’ll see that after the 1.0 release we gave both the website and documentation a major refresh. For those of you who haven’t come across Tetragon before, it’s an eBPF-based security observability and runtime enforcement tool, and it’s Kubernetes-aware. That means it can match kernel-level events with Kubernetes metadata, which you’re going to see in some of the examples I show. Tetragon lets you create policies and filtering using eBPF at the kernel level. That gives you reduced overhead, visibility into process activity, and real-time enforcement at the kernel level itself. Policy configuration Once Tetragon is installed, the main job becomes creating policies. I’m going to show you one example policy today, but the documentation also covers the concepts in more detail and includes a policy library with lots of examples. A Tetragon policy is essentially built around two key ideas. The first is the hook point, which is where we tap into the kernel to collect data. The second is the selector, which is how we filter the events we collect and decide what action to take. Because Tetragon understands Kubernetes metadata when it is running in a Kubernetes environment, we can also filter on pods and namespaces, which is extremely useful. Configure Tetragon For today’s example, I want to use a Canary Token. If you’re not familiar with the term, a canary token is a cybersecurity mechanism used to alert you when something accesses a resource that should not normally be touched. That resource could be a file, an image, a URL, a fake credential, or even a virtual device. In our case, we’re going to use the canary token as an alerting mechanism when we observe an event that should not happen in the environment. Specifically, we’re going to trigger it when someone tries to modify the passwd file in the container. I’m using Canarytokens.org to generate a free URL token. Once I generate the token, I go back into my Tetragon tracing policy and insert that URL as the action endpoint. The policy itself is based on one of the example file observability policies. We’re hooking into the security_file_permission kernel probe, looking for operations against the path /etc/passwd, and checking whether the action is a read or write. At the end of the policy, we attach the match action that calls the Canary Token URL. Then I apply that policy into my environment. In a second terminal, I’m also watching the live Tetragon output so I can see the event appear immediately. Now I’m going to trigger the policy by writing some data into /etc/passwd. As soon as I do that, Tetragon captures the event and we can see the process execution in the live output. If I then go back to the Canary Token management page, I can see that the token has been triggered and I can inspect the source IP and the associated metadata. Conclusion So we’ve got two useful things happening here. First, Tetragon gives us kernel-level visibility into the suspicious event itself. Second, we use the canary token as a lightweight external alerting mechanism. You could apply the same idea to lots of other honeypot-style scenarios. For example, you could place a file in a container or VM that should never be accessed by normal applications. If someone touches it, Tetragon can trigger an alert immediately. That gives you a very simple but powerful way to detect either malicious activity or serious misconfiguration. That wraps up this quick walkthrough. Thank you very much.