Back to blog

eBPF Runtime Security at Scale: Top Tetragon Use Cases (Part 2)

Jeremy Colvin
Jeremy Colvin
Published: Updated: Tetragon
ebpf runtime security tetragon

Building off the community interest in the top security observability use cases with Tetragon and eBPF, we decided to focus on another set of use cases around runtime security controls.

In Part 1, we highlighted Tetragon’s capabilities as the leading eBPF runtime security observability tool. Covering the quick adoption by platform and security teams and how it provides context-rich telemetry combining network, process, and Kubernetes metadata to solve for a variety of use cases; for example, generating reliable audit trails, detecting container escapes, deploying dynamic policies for zero-day vulnerabilities, gaining deeper visibility into runtime events, or monitoring network traffic with minimal overhead.

Here, we shift our emphasis towards granular runtime control over our environments. This manifests through Tetragon policies that implement precise security enforcement, streamline integrations with existing security tooling, and offer real-time insights to detect attacker behavior.

eBPF Security Observability: Top Tetragon Use Cases (Part 1)

Tetragon is the standard for eBPF-based security observability, let’s look at what that means for the top use cases being solved.

See the eBPF security observability use cases

Reduce tool sprawl and eliminate data silos  

Data is gold, and Tetragon has it in troves. Easily pipe out events to a preferred SIEM or metrics dashboard

Integrate modern observability and policy standards such as Kubernetes, Prometheus, fluentd, OpenTelemetry, Open Policy Agent, and traditional SIEM platforms. Teams shouldn’t have to work off a black boxed security solution; instead, Tetragon makes it easy to get the right data into the right hands in real time.  

Let’s look at Splunk for example, providing deep insights from runtime telemetry. Tetragon easily integrates with Splunk to bring out insights from runtime security data.

In the Splunk dashboard below, a team is using the data from Tetragon to detect weak TLS cipher versions. Here the TLS protocol versions are tied to the source Kubernetes pod and namespace, with added context on the cipher key length and SNI. By combining Tetragon with your preferred SIEM, teams make it easy to get real-time actionable insights deep into network and runtime layers. 

A core ethos of Tetragon is to simplify runtime security for teams, and that naturally extends to simplifying the interoperability between extracted data (from Tetragon) and the tools that glean or visualize insights. Integrate rich security data with your existing tools in real-time, without speed bumps.

Monitoring Access to Sensitive Files in Splunk

This example shows an integration with Splunk to track access to sensitive files while providing context of the access such as Kubernetes metadata, container image, binary, and user information.

See the TLS & FIM dashboards

Correlate network and runtime events

It’s no longer enough to know a network connection is suspicious; with Tetragon teams tie that network activity to a specific binary and identity that launched that connection. Tie together events across different layers to paint a picture of everything happening in your environment.

Teams struggle in securing Kubernetes environments because traditional IP-based tools are ineffective in dynamic cloud native environments; platform and security engineers need tools that map network and node processes together, really breaking down traditional silos around what data has what context.

Strong host-based visibility provides a unified layer of insight, with Tetragon’s ease of deployment saving thousands of hours across deployments, correlating data, and pinpointing the root cause of issues. 

Linking network activity to specific processes and identities offers deeper visibility into Linux environments. For example, you can identify exactly which process sent out an HTTP request or identify weak TLS cipher versions in real time.

We need to know which process sent out this HTTP call on Tuesday at 3:15 p.m. from host A. How can we possibly get this info?

[Tetragon] gives us the depth all the way down to the process on a host. Even pods are just processes. And the speed of deployment is awesome.

Senior Security Engineer, Top Developer Platform

How to guide for host-based Kubernetes visibility

Learn how Tetragon’s lightweight eBPF sensor captures K8s telemetry down to the binary, tying process to network data with no application changes.

Read more about host based visibility

Implement file integrity monitoring at scale 

Traditional tools relied on periodic scanning to understand file modifications, reads, or writes – leaving environments vulnerable to common TOCTOU (time of check time of use) attacks or delayed incident response. File monitoring with eBPF represents a significant leap forward in flexibility, hardening, and depth.

Isovalent Enterprise for Tetragon monitors inline at the kernel level, using inodes. Inodes are kernel structures that are a stronger guarantee of monitoring the sensitive file.

For example, instead of evaluating based off of filename (which can be modified or moved), the inode is constant regardless of how the filename or filepath changes. Let’s imagine a financial institution needs to monitor sensitive files (i.e. customer account records) across their Linux servers. With legacy FIM solutions the security team relies on periodic scans or event notifications from mechanisms like inotify. These methods are at risk of TOCTOU attacks when new files are created or fail to handle context around an event, such as which user or application accessed the file.

With eBPF, Tetragon sits in the kernel monitoring new files as soon as they are created, leaving no time for an attacker to tamper with the file. 

File monitoring is a difficult problem at scale, with extremely high volumes of I/O activity to evaluate. Traditionally, this high-volume activity incurs a resource overhead from nearly every file event needing policy evaluation plus potential alerting to user space. eBPF programs resolve this overhead issue and leap us forward by monitoring file activity and filtering events directly in the kernel, this means extensive filtering is performed in-kernel so that no events are pushed to user space unless they match an applied policy.

File Monitoring with eBPF and Tetragon (Part 1)

Deep dive into Tetragon’s file monitoring capabilities, focused on implementing low-overhead, highly scalable monitoring directly in the kernel with eBPF.

Read the dev blog on Tetragon's file monitoring

Track anomalous behavior in real time

Because Tetragon is uniquely positioned to extract data at the Linux kernel level, it is able to collect telemetry all the way down to the library or binary level of your applications. This brings rich user behavior data for any application security use case, and with the right data correlation, should allow any anomalous behavior to surface quite early on.

Monitor and visualize anomalous behavior in any Linux environment in real time. A top use case for security teams, effectively visualize and analyze suspicious user behavior, map out potential attack paths, and monitor for compromised accounts. This proactive approach using the richness of Tetragon’s data enables organizations to stay ahead of threats and respond quickly to emerging security challenges no matter where it originates.

For instance, imagine a service account begins executing system calls that are inconsistent with typical behavior or a situation where a normally dormant endpoint begins transmitting large volumes of data unexpectedly. Tetragon is capturing all the runtime telemetry and context needed to highlight a possible data exfiltration attempt, helping teams swiftly intervene and mitigate.

With the growing number of security incidents where malicious actors have unfettered access to an environment, it’s important to have the data to answer whether third party or end user behavior is expected or abnormal. Tetragon generates clear audit trails, and gives the flexibility in policy to block malicious runtime events from executing.

Brewing the Kubernetes Storm Center: Open Source Threat Intelligence for the Cloud Native Ecosystem

Brewing the Kubernetes Storm Center: Open Source Threat Intelligence for the Cloud Native Ecosystem

Based on a given threat model: a) generates a Kubernetes-based environment with embedded trip-wires b) exposes these simulated environments to the wild to observe quantitative threat intelligence, and c) informs cost-effective decisions for a security team.

Watch the KubeCon session

Proactive security enforcement and control

Anything you can observe with Tetragon, you can enforce upon. 

This level of granular enforcement is a game changer for Linux environments, particularly when it comes to context-aware process control. Leveraging identity-aware mappings, Tetragon provides a sophisticated alternative to traditional seccomp filters, allowing for precise control over syscalls based on the identity and context of workloads. This allows security teams to enforce exceedingly fine-grained policies with ease, addressing sensitive namespaces or workloads without compromising on operational overhead.

Tetragon’s use of eBPF combines the advantages of traditional tools while avoiding common pitfalls around asynchronous enforcement, high overhead, and depth of telemetry

In-kernel enforcement policies are stronger than traditional reactive policies, as the decision logic + action both happen in the kernel. eBPF in-kernel enforcement policies offer a more robust approach to security by blocking malicious actions before they execute, providing a proactive security model through blocking (i.e. SIGKILL) or overriding events. 

Tetragon’s enforcement capabilities are built on two main mechanisms: overriding values and signal actions

  • Override Return Value: Modifies a function’s return value, preventing it from being executed.
  • Signals: Sends signals like SIGKILL to terminate processes matching specific criteria.

eBPF policies are flexible, and both mechanisms can be used together in the same policy for a layered approach, offering a stronger guarantee the malicious event is not completed.

Sandbox policies for simplified auditing and enforcement of system calls

In this example sandbox policy, we have set up a list of allowed syscalls and prevent all other sensitive syscalls outside this list (operator:

Test the policy snippet

Catch supply chain attacks 

Ensuring the integrity of your build process is critical. Supply-chain vulnerabilities like SolarWinds, log4shell, and XZ Utils continue to expose gaps in visibility into our downstream software dependencies. 

With Tetragon and eBPF, teams are building proactive policies and verifiable attestation pipelines to ensure that every software package is delivered exactly as the author intended, free from malicious interference. 

In his presentation at Kubecon EMEA 2024, Tom Meadows, Software Engineer at TestifySec, uses the metadata from Tetragon’s eBPF programs to secure the build system of a Kubernetes environment. Specifically, generating an attestation that nothing was tampered with or injected into the build. This expands the use case out into not just monitoring specific programmatic libraries, but also opens up towards covering your CI/CD environment and generating SBOM’s (software bill of material) for any runtime dependencies. 

Tom-Meadows-Talk- metadata tetragon kubecon

[Tetragon] maps all of these eBPF events seamlessly straight to the pod, and just basically did my job for me. It was [doing] what I wanted to do. All the magic’s done inside this project called Tetragon.

Tom Meadows, Software Engineer, TestifySec

Bee-Lieve in the Metadata: Pollenating Build Attestations on Kubernetes with Tetragon and eBPF

Like bees making honey, we developers forage the internet for code that we borrow and thread together to create software. It’s built, uploaded and shared, resulting in hundreds of thousands (if not millions) of software artifacts being built each day. For an unlucky few, attackers seek to infiltrate the build process like a hornet attacking a beehive.

Watch the recording

Embedded zero trust across Linux infrastructure

Zero trust involves juggling numerous tools across your organization, each addressing different aspects of security – identity, network, OS, service mesh, and on – resulting in a fragmented and complex control over your environment. Tetragon solves this fragmented approach to zero trust, giving teams a central tool for all Linux workloads to be monitored and enforced with zero trust policies. 

Traditional solutions may cover network security, but overlook OS-level protections, creating gaps to be exploited. Using eBPF, Tetragon is the glue between the multiple layers of your runtime and network observability. These flexible and dynamic policies also give teams granular control over their environment to enforce rules that block malicious activity or surface insights into who or what is spawning suspicious traffic.

So, rather than enforcing zero trust in isolation based on only network events or only runtime processes, eBPF programs bring these layers together and give a unified approach to collecting, observing, and enforcing on security-significant events.

Tetragon process tree zero trust

Zero Trust Security with Cilium & Tetragon

Implement advanced runtime security capabilities covering protocol enforcement, IP and port whitelisting, and automatic application-aware policy generation.

Read the Zero Trust walkthrough

Enriched threat detection and response

Detect and mitigate attacks at runtime. Tetragon excels in threat detection and response focusing on zero-day exploits, indicators of compromise, and lateral movement in containers. There is also a growing policy library of open source detection and enforcement use cases, in addition to the Isovalent Enterprise library of CVE policies. Isovalent’s policy library spans common CVE’s targeting the Linux kernel, container runtimes, and daemons or services.

Tetragon’s unique ability to look across runtime layers with eBPF gives the necessary context to collect meaningful real-time insights from system calls, network activity, and process events.

Two core use cases expand on this: first, real time enforcement using TracingPolicies to respond to suspicious activity as they occur, and second, generate context-rich alerts for your platform or security team with detailed telemetry that improves swift and effective responses.  

Binary-tmp-policy- tetragon simplified ootb policy

Privilege Escalation and Kernel CVEs Protection

Address infrastructure risk by monitoring for privilege escalations and specific kernel CVEs, safeguarding against known exploits in the kernel, container runtimes, and Linux daemons.

Read about CVE protection

Summary of the top Tetragon use cases

Tetragon’s key advantage lies in its ability to provide detailed, real-time security data from networks, processes, and workloads with incredible amounts of context. Teams use this data to quickly and clearly understand what’s happening in their systems, making it possible to detect and respond to threats in real time.

As the leading eBPF runtime security tool, Tetragon has proven its wide range of uses for illuminating security blind spots. From enforcing strict security rules, observing network health, to monitoring file activity, Tetragon keeps critical systems secure and running smoothly.

Contact Isovalent (creators of eBPF, Cilium, and Tetragon) for expertise into where Tetragon’s eBPF runtime security fits into your architecture and gives your team the modern security toolkit for protecting highly-distributed enterprise deployments.

Jeremy Colvin
AuthorJeremy ColvinSenior Technical Marketing Engineer

Related

Briefs

The guide to host-based Kubernetes visibility

Correlate process-to-network data. Learn how Tetragon’s lightweight eBPF sensor captures K8s telemetry down to the binary, tying process to network data with no application changes. Decode DNS, TLS, HTTP, UDP, TCP , and more while matching to process ancestry information, all with Kubernetes identity-aware metadata (labels, pod names, etc). Read the solution brief and get under the hood with Tetragon.

By
Jeremy Colvin

Fireside Chat: Cloud Native Security & Compliance

Join ControlPlane’s CEO Andrew Martin and Isovalent’s Chief Open Source Officer Liz Rice for a fireside chat where they demystify Cloud Native Security and Kubernetes Compliance.

Online
Blogs

eBPF & Tetragon: Tools for Detecting XZ Utils CVE 2024-3094 Exploit

Detecting XZ Utils liblzma CVE 2024-3094 backdoor exploit with Tetragon and eBPF. Includes ready to apply yaml policy.

By
Jeremy ColvinDjalal Harouni

Industry insights you won’t delete. Delivered to your inbox weekly.