We're excited to announce Tetragon v1.8.0, the largest release in the project's history! Over 50 contributors made it possible, including more than 30 new contributors. A release like this only happens when the community shows up, and it's been a real joy to see so many new names in our pull requests this cycle.
Tetragon uses eBPF to provide runtime visibility and enforcement across your environment and workloads. At this level of granularity, one ongoing challenge is giving users the right abstractions to write policies that target behavior precisely and runs safely at scale. We keep improving on this with every release, and much of the feature set in this release leans largely into this. Stateful policies let one event change how later events are enforced. Command-line arguments bring more precise in-kernel decisions. Uprobe runtime function replacement patches user-space behavior in place. Tetragon is also safer and easier to run at scale, with policy domains, per-node policy loading, persistent gRPC policies, a least-privilege agent, and a range of performance and usability improvements in this release cycle.
With KubeCon + CloudNativeCon ongoing in Salt Lake City, we’re looking forward to meeting the community again and continuing these conversations in person. Tetragon maintainers and friends from the community will be at KubeCon, and we’d love to hear from you. You can also join the conversation anytime in the #tetragon channel on the Cilium Slack.
What's new in Tetragon v1.8?
- Stateful policies: Carry state between hooks, so one event can change how the policy handles later events. (more details)
- Command-line argument filtering: Match events on the process's command-line arguments. (more details)
- Uprobe Caller Function Filtering: Trigger Uprobes only when a given function appears in the call stack. (more details)
- Runtime Function Replacement: Redirect function calls to a replacement implementation without restarting the application. (more details)
- Policy Domains: Isolate tracing policies by source (Kubernetes, gRPC, static files), so policies with the same name from different sources do not interfere with each other. (more details)
- Workload-Scoped Selectors: Give each selector its own pod, container, or host scope, so one policy can enforce different rules on different workload types. (more details)
- Per-Node Policy Loading: Load tracing policies only on nodes whose labels match, so each policy only ever gets loaded on the node it's meant for. (more details)
- Persistent gRPC Policies: Save tracing policies added through the gRPC API to disk and restore them on restart, so they keep enforcing across agent restarts without a lapse. (more details)
- Performance, Operational and Other Improvements: A leaner and least privileged agent, use a custom string for redaction filters, and more! (more details)
Stateful Policies
When writing policies for behavioral detection, the signal of attack is often a sequence of events rather than a single one. Before v1.8, there was no way to express that sequence as a policy, because one selector could not take an earlier event into account when evaluating the next. Correlating the sequence meant exporting events to user space, which added overhead and ruled out enforcement based on that sequence.
Tetragon v1.8 introduces stateful policies that let decisions reflect a sequence of events as it unfolds, rather than judging each event in isolation. One part of a policy records that an event occurred, and other parts can then use that state to decide what to do next. Because the state lives in the kernel and is shared by every hook in the policy, the change takes effect on the very next event, without a round trip to userspace or a policy reload.
A good way to reason about stateful policies is to think of them as policy-wide if-else statements. Each state is a variable that the policy owns. You declare it by name under spec.state. A hook changes the variable with the UpdateState action when it sees a particular event. The Set operator stores a u64 value, and Delete clears it. Any selector in the policy can then check the variable with matchStates, using Equal or NotEqual, before its actions run. Together, these give you a rough equivalent of if (state) { do this } else { do that }. A state that has never been set only matches NotEqual, so before the trigger event, the policy follows only the else branch.
For example, an attacker who exploits a remote code execution (RCE) bug in a web application, such as the recent PHP2Shell vulnerabilities in PHP, will often get the web server to start a shell, then add a user to /etc/passwd or a job to establish persistence. Administrators will typically make the same changes all the time, so a policy can't block them outright without interfering with legitimate activity. After a web server starts a shell, this same activity strongly signals an attempt to set up persistence. A stateful policy can connect the two events. Once it sees the web server start a shell, it kills any process on the node that attempts to modify these files, while the web server keeps serving requests.
With Stateful policies, we can write policies that depend on a precondition. Until a specific signal appears, the policy can ignore an event; once this signal appears, the same event can be reported, blocked, or handled as the policy defines.
Filtering on command-line arguments with matchCmdArgs
The same binary can be perfectly ordinary in one invocation and suspicious in another, depending entirely on the arguments it was given. This release introduces an additional selector, matchCmdArgs, that enables filtering based on the command-line arguments a process was started with. Policies can now distinguish between processes running the same executable but invoked with different command-line arguments for different purposes,
Tetragon already matches a process's executable with the matchBinaries selector. Telling apart different invocations of the same executable meant exporting events to user space and inspecting the command-line arguments afterward. The new matchCmdArgs selector makes this context available on every supported hook type. Policies can now scope to specific invocations of a binary, rather than treating every run of that binary alike. This granularity also helps reduce noise from legitimate tool use.
For example, a container image might include the Python binary solely to run its service. Users can now write policies that specifically allow outbound connections from python3 /app/server.py while killing other Python processes that attempt to connect, including a python3 -c <inline code> one-liner launched from an exploited shell. The tracing policy below demonstrates this.
spec:
kprobes:
- call: tcp_connect
syscall: false
args:
- index: 0
type: sock
selectors:
- matchBinaries:
- operator: Prefix
values:
- /usr/bin/python3
matchCmdArgs:
- index: 0
operator: NotEqual
values:
- /app/server.py
matchActions:
- action: Sigkill
Uprobe Runtime Function Replacement
Uprobe function replacement in Tetragon v1.8 lets teams apply targeted patches, add security checks, and test application behavior without rebuilding or restarting the application.
It extends the existing Override action for Uprobes, which could already change a user-space function's return value. In v1.8, a Uprobe Override can also redirect calls to a different function entirely. The replacement can be a function already in the application, or it can come from a separate shared library.
For example, suppose an authentication service has a vulnerability in its is_authorized() function. A team can write a patched version, is_authorized_patched(), compile it into a shared library, and use a policy to redirect calls to it while the service keeps running.
spec:
uprobes:
- path: /opt/authsvc/authsvc
symbols:
- is_authorized
binaryDigests:
- sha256:749b9d8023e8b8a6190a5106f3e7bbbb9cb54f59b37dd4edba1bbdd58c00fbaa
selectors:
- matchActions:
# Redirect every call to the fixed function from the patch library.
- action: Override
argNewSymbol: is_authorized_patched
sopath: /opt/authsvc/patches/fix-auth.so
N/B: Shared-library replacement through sopath is experimental and requires Linux 6.18 or later and an application dynamically linked against glibc 2.34 or later. The replacement must have exactly the same function signature as the original. Tetragon cannot verify that compatibility, and a mismatch can possibly crash the process.
Source-Isolated Policy Domains
Tetragon accepts tracing policies from three sources: the Kubernetes API, the gRPC API (used by the tetra CLI), and static files loaded from disk. Until now, the Tetragon agent identified a policy only by name and namespace, regardless of the source. A tetra operation could therefore unintentionally affect a Kubernetes-managed policy with the same name.
Tetragon v1.8 gives each source its own policy domain. Policies with the same name can run side by side in different domains without interfering with each other, and each source can only add, delete, or reconfigure policies within its own domain
Reusing tracing policy names across domains is useful in practice. It lets you apply and test the same tracing policy under one consistent name across different environments and sources, without renaming it to avoid conflicts.
To make ownership clear, the tetra tracingpolicy list command now shows each policy's domain, and the new tetra tracingpolicy domains command lists the domains currently in use.
Workload-Scoped Selectors
The previous release of Tetragon added support for hostSelector, giving teams control over whether a policy applies to processes running directly on the host, alongside the existing pod and container filters. This feature is only supported for Tetragon installed in Kubernetes
Tetragon v1.8 builds on that foundation with the new matchWorkloads selector, which lets a single policy take different actions for different workloads. For example, a policy can report suspicious behavior in kube-system without interrupting system services, and kill processes that show the same behavior in application pods.
Previously, workload filters applied to the whole policy, so handling these workloads differently meant maintaining separate, nearly identical policies. Now both cases can live in one policy, reducing duplication and making enforcement easier to maintain.
Within matchWorkloads, the podSelector, containerSelector, and hostSelector filters decide which workloads an individual selector matches.
The tracing policy below watches for shells starting inside pods and treats them differently by workload. A shell in a kube-system pod is reported, while a shell in any other pod is killed. It is one policy carrying two workload-scoped selectors, so a single rule applies different actions depending on where the process runs.
spec:
kprobes:
- call: security_bprm_check
syscall: false
args:
- index: 0
type: linux_binprm
selectors:
# shell in kube-system, report it.
- matchWorkloads:
podSelector:
matchExpressions:
- key: k8s:io.kubernetes.pod.namespace
operator: In
values:
- kube-system
matchArgs:
- index: 0
operator: Postfix
values:
- /sh
- /bash
matchActions:
- action: Post
# shell in any other pod, kill it.
- matchWorkloads:
podSelector: {}
matchArgs:
- index: 0
operator: Postfix
values:
- /sh
- /bash
matchActions:
- action: Sigkill
Per-Node Policy Loading
In Kubernetes, Tetragon runs an agent on every node. Previously, every agent loaded every policy and attached its hooks, whether or not the policy was needed on that node.
Tetragon v1.8 introduces spec.nodeSelector, which controls which nodes load a policy. Each agent checks its own node's Kubernetes labels and loads the policy only when they match. Agents on other nodes attach no hooks for it, so the policy adds no overhead there.
For example, a policy can target a GPU node pool, be limited to one CPU architecture, or roll out enforcement on a canary pool before wider coverage. When a node's labels change, the Tetragon agent re-evaluates the selectors and loads or unloads policies. A policy that a node skips shows as skipped in tetra tracingpolicy list, so a policy that was deliberately left out can be told apart from one that is missing.
The nodeSelector is supported on both TracingPolicy and TracingPolicyNamespaced CRDs.
N/B: Because node labels determine where a policy is enforced, anyone who can edit them can disable enforcement on a node. Enforcement policies should select labels set at provisioning time and restrict who can modify them.
Uprobe Caller Function Filtering
Uprobes attach eBPF programs to functions in user-space binaries and shared libraries. Until now, a Uprobe fired on every call to its function, no matter where in the execution stack the call came from. Many functions are shared helpers called from all over a program, and a policy usually cares about only a subset of these code paths.
Take an application where every outbound request goes through a single send_request() function. It's called constantly to refresh a cache, and occasionally to export user data. A Uprobe on send_request() reports every call, so the few exports that matter are buried alongside cache refreshes:
The new matchUserCallers selector filters on the call stack, so the Uprobe only triggers when a given function appears among the callers:
spec:
uprobes:
- path: /opt/myapp/myapp
symbols:
- send_request
selectors:
# Only when export_user_data is two frames up the call stack:
- matchUserCallers:
- depth: "2"
symbol: export_user_data
Previously, monitoring send_request() captured both paths. With matchUserCallers, a policy can match on only requests originating from export_user_data().
Teams can match the function that made the call directly or look further back through the call chain across application code and shared libraries. Policies can check a specific position or search the 15 most recent stack frames.
Matching works regardless of where the operating system loads the application in memory and is tied to the application or library build targeted by the policy. The application and libraries in the call chain must be compiled with frame pointers so Tetragon can trace where each call originated from.
Persistent gRPC Policies
Previously, policies loaded through the gRPC API, including those added via the tetra CLI, were lost whenever the Tetragon agent restarted, and teams had to reapply them to restore coverage.
Tetragon v1.8 introduces --persist-grpc-policies option, which lets these policies survive agent restarts. The agent saves each policy to disk, along with its enabled state and enforcement mode, then validates and reloads it at startup. Adding, deleting, enabling, disabling, or changing a policy's mode updates the saved state automatically, so existing tetra workflows stay the same.
Combined with two existing options, persistence also lets you restart the agent without interrupting enforcement. The --keep-sensors-on-exit keeps BPF programs attached in the kernel when the agent exits, and --release-pinned-bpf=false option preserves them when it starts again.
With persistence enabled, the agent restores the policies from disk and loads their replacements before detaching the old programs:
sudo ./tetragon \
--keep-sensors-on-exit \
--release-pinned-bpf=false \
--persist-grpc-policies
This handoff now also covers policies that use LSM hooks, so it preserves their enforcement across restarts.
The --persist-grpc-policies option on its own restores policies after a restart, but leaves a gap in enforcement while they reload. Use all three options to keep enforcement running throughout the restart.
Performance, Operational and Other Improvements
Alongside the major features and improvements highlighted above, this release of Tetragon includes several smaller performance and operational improvements worth mentioning. These include faster policy loading, more efficient event processing, and clearer feedback on policy validation and enforcement.
- Smaller programs and faster policy loading: Tetragon now loads only the selector-filtering code a policy needs. In a test with a single prefix match, selector programs were 41–58% smaller and the policy loaded about 14% faster. See the test policy and stats in the pull request.
- More efficient event processing: CPU time in the event-handling path dropped by about 27%, with around 2.1 million fewer allocations in the profiling run. See the performance gains numbers in the pull request.
- Policy validation outside Kubernetes: Policies loaded through tetra or from files now get the same schema, and CEL checks the Kubernetes API server applies, so invalid policies are rejected before they load.
- Configurable secret redaction: Each redaction filter can now set its own replacement string with
redact_str. In exported events, that shows both that a value was redacted and which filter caught it. For example,"redact_str": "<redacted:private-key>"replaces the default ***** with a descriptive label. Environment variables are also redacted more thoroughly. - Clearer policy status. Policies excluded from a node by nodeSelector now appear as skipped in
tetra tracingpolicy list. Previously they didn't appear at all, which looked the same as a policy that was never applied. - Warnings when enforcement is disabled: If monitor-only mode is suppressing a policy's enforcement, Tetragon now logs a warning saying so.
- Least-Privilege Agent: The Helm chart now runs the Tetragon agent with an explicit, minimal set of Linux capabilities instead of as a privileged container.
Upgrading to Tetragon v1.8
To upgrade an existing Kubernetes installation, update the Helm repository and upgrade to the new chart version:
helm repo update
helm upgrade tetragon cilium/tetragon -n kube-system --version 1.8.0
For more details, see the Kubernetes upgrade guide. If you run Tetragon as a systemd service installed from a release tarball, follow the package upgrade instructions.
N/B: Before upgrading, note one change to the Helm chart: the agent container now runs unprivileged by default, with an explicit set of Linux capabilities. A dedicated privileged init container, mount-bpf-fs, mounts the BPF filesystem on the host. If your cluster's admission policies block privileged containers, you should add an exception for this init container. To keep the agent privileged as before, set the tetragon.securityContext.privileged=true.
The v1.8.0 release notes has the complete list of changes and any other upgrade considerations for this release.
Ready to Learn More?
- Tetragon documentation
- Tetragon v1.8.0 release notes
- Tetragon on GitHub
- Join the Cilium Slack and find us in the #tetragon channel
Paul Arah, Community Builder, Security