Tetragon, EC2 Image Builder & Network Flow Monitor

“At any given moment, you have the power to say, this is not how the story will end.”—Christine Mason Miller. With that, we resume where we left off when we announced the availability of Tetragon with the EC2 Image Builder. When customers operate in cloud-native environments, they often encounter new challenges. One common issue is the lack of network visibility as traffic moves to the cloud. Even after gaining visibility, managing and making sense of the massive volume of data can be overwhelming. Tetragon helps address this by enabling the publication of TCP metrics from EC2 instances to Network Flow Monitor, a feature of Amazon CloudWatch Network Monitoring. The EC2 Image builder integration of Tetragon allows customers to deploy Tetragon at scale on AWS by building their custom EC2 image with Tetragon as a build component. This tutorial shows how to build the custom image with Tetragon from AWS Marketplace, configure and publish network telemetry to Amazon CloudWatch Network Flow Monitor.

Prerequisites

Take note of the following prerequisites:

Access to AWS Marketplace. Use your existing Marketplace account or create a new account for free.
Network Flow Monitor, a feature of Amazon CloudWatch Network Monitoring, is supported on EC2 instances in one of the supported AWS regions.
Linux kernel version minimum 5.4.
Permission to create an IAM role and attach an IAM policy.

What is EC2 Image Builder?

EC2 Image Builder offers a fully integrated platform for creating, managing, and deploying standardized Golden AMIs. This service addresses several operational challenges related to software deployment and security across cloud environments.

One of the most significant benefits of EC2 Image Builder is standardization. Manually configuring instances can lead to discrepancies that undermine security and efficiency. With Image Builder, organizations can create preconfigured standardized images to meet specific requirements, reducing variability and ensuring consistency across deployments.
Another key advantage is automation. Regular updates, including security patches, can be labor-intensive and disruptive if performed manually. Image Builder automates these processes, ensuring images remain current with minimal effort. Moreover, it integrates vulnerability scanning and compliance checks, providing an added layer of security before images are deployed.
The service also handles complex dependency management. Different operating systems often require unique configurations, leading to compatibility issues. Image Builder eliminates this problem by taking a templated, version-controlled approach to image creation. This ensures software dependencies are met, configurations are consistent, and errors are minimized.

Capabilities like these translate into tangible benefits for organizations, including reduced operational costs, improved resource efficiency, and enhanced visibility. In addition, the service’s logging and versioning features provide greater control over image configurations and updates, simplifying troubleshooting and audit processes.

What is Network Flow Monitor?

Network Flow Monitor is a feature of Amazon CloudWatch Network Monitoring. Network Flow Monitor uses agents that you install in your AWS workloads to return performance and availability metrics about network flows. Using Network Flow Monitor, you can access near real-time metrics, including TCP retransmissions, round-trip time and data transferred, for your actual workloads. You can also identify whether an underlying AWS network issue occurred for the network flows tracked by a monitor by checking network health indicator (NHI) values.

What are the key features of Network Flow Monitor?

Network flow Monitor collects near real-time latency and packet loss metrics for TCP traffic to and from AWS workloads.
When your AWS workloads experience performance issues, Network flow monitor correlates the collected network telemetry and the health of the AWS Network infrastructure and quickly determines if the issue is due to a Network degradation.

How to use Network Flow Monitor?

Network flow monitor leverages lightweight agents on compute instances to collect, aggregate and export performance metrics to service backend. With the built-in integration, customers can utilize Tetragon agents to report performance metrics to Network Flow Monitor service instead of the native agent. This allows customers to leverage a single agent to send observability telemetry to Amazon CloudWatch as well as Security Telemetry to Isovalent.

Agents collect performance metrics such as TCP round-trip time, TCP retransmissions, and data transferred and send them to the Network Flow monitor service endpoints along with metadata such as instance ID, availability zones, VPC ID, etc. for further correlation.

How do you get started with EC2 Image Builder and Tetragon?

This section will guide you through creating an image pipeline using the EC2 image builder. The EC2 Image Builder creates EC2 instances that will run Tetragon.

Log in to the AWS Marketplace with your credentials.
Once logged in, search for Tetragon from the Marketplace.

Click > View purchase options

Click >Continue to Configuration

Click > Launch on the EC2 Image Builder console

Click> Create image pipeline

Provide a name for the pipeline.

You can schedule the builder ( to run at an appropriate time) or manually initiate the pipeline.
- Click > Manual to initiate the pipeline manually.

Click > Create a new recipe and select the output type as AMI.

Provide the name of the recipe and a version number.
Select managed images

Click > Image Operating System and select Ubuntu.
Click > Quick start and select Image as Ubuntu Server 22 LTS x86.

Click > Add build components > AWS Marketplace-new> Select Tetragon.

Click on the box next to Tetragon to select and add to the recipe.

Use the default EBS volume.
- Adding additional EBS volume is optional.

Click > Default workflows for the image recipe.

Click > Create infrastructure configuration using service defaults.

Click > Next to complete the infrastructure configuration.

Click > Create distribution settings using service defaults.

Click > Next

This completes the creation of the pipeline.

Click > Actions > Run pipeline

You can also check the image being built. Click > Images > Workflow

This triggers the creation of an AMI.

Click > Launch an instance from AMI.

Provide a name for the instance(s).

Create a new VPC or use an existing VPC.
Define the security group rules for the instance(s).
- Allow ports 22 (for access to the VMs), 80, and 443 (network flow monitor in the next section).

Click > Launch instance.

ssh -i "############.pem" ubuntu@####################.compute-1.amazonaws.com

root@ip-172-31-51-100:/home/ubuntu# sudo systemctl status tetragon-enterprise.service
● tetragon-enterprise.service - "Tetragon Enterprise - eBPF-based Security Observability and Runtime Enforcement Service"
     Loaded: loaded (/lib/systemd/system/tetragon-enterprise.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2025-02-13 08:17:41 UTC; 35min ago
       Docs: https://docs.isovalent.com/
   Main PID: 333 (tetragon)
      Tasks: 7 (limit: 1130)
     Memory: 189.2M
        CPU: 2.292s
     CGroup: /system.slice/tetragon-enterprise.service
             └─333 /usr/local/bin/tetragon

Feb 13 08:17:58 ip-172-31-51-100 tetragon[333]: time="2025-02-13T08:17:58Z" level=info msg="Read ProcFS /proc/ appended 105/168 entries"
Feb 13 08:17:58 ip-172-31-51-100 tetragon[333]: time="2025-02-13T08:17:58Z" level=info msg="Loaded BPF maps and events for sensor successfully" sen>
Feb 13 08:17:58 ip-172-31-51-100 tetragon[333]: time="2025-02-13T08:17:58Z" level=info msg="Cgroup rate disabled (0/0s)"
Feb 13 08:17:58 ip-172-31-51-100 tetragon[333]: time="2025-02-13T08:17:58Z" level=info msg="sensor controller starts"
Feb 13 08:17:58 ip-172-31-51-100 tetragon[333]: time="2025-02-13T08:17:58Z" level=info msg="Available sensors" sensors=
Feb 13 08:17:58 ip-172-31-51-100 tetragon[333]: time="2025-02-13T08:17:58Z" level=info msg="Registered sensors (policy-handlers)" policy-handlers=">
Feb 13 08:17:58 ip-172-31-51-100 tetragon[333]: time="2025-02-13T08:17:58Z" level=info msg="Registered probe types" types="skmsg, enforcer, sockops>
Feb 13 08:17:58 ip-172-31-51-100 tetragon[333]: time="2025-02-13T08:17:58Z" level=info msg="Perf ring buffer size (bytes)" percpu=68K total=1020K
Feb 13 08:17:58 ip-172-31-51-100 tetragon[333]: time="2025-02-13T08:17:58Z" level=info msg="Perf ring buffer events queue size (events)" size=63K
Feb 13 08:17:58 ip-172-31-51-100 tetragon[333]: time="2025-02-13T08:17:58Z" level=info msg="Listening for events..."
lines 1-21/21 (END)
root@ip-172-31-51-100:/home/ubuntu#

How can you enable Network Flow Monitor logs with Tetragon?

Tetragon agents gather performance metrics and send them to the Network Flow Monitor backend. Tetragon agents receive only the “bpf_sock_ops” structure from the Linux kernel. This structure provides the local and remote IP address, local and remote TCP port, counters, and round-trip times. Tetragon agents are automatically enabled when EC2 instances are created using the image builder (as explained in the previous section).

apt update -y 
apt install amazon-ec2-utils -y

sudo mkdir -p /etc/tetragon/tetragon.conf.d
sudo bash -c "echo true > /etc/tetragon/tetragon.conf.d/enable-aws-sonar"
sudo bash -c "echo $(ec2-metadata --availability-zone | awk '{print substr($2, 1, length($2)-1)}') > /etc/tetragon/tetragon.conf.d/aws-sonar-region"
sudo systemctl restart tetragon-enterprise.service

Create an IAM role.
- In the AWS Management Console, in the Amazon EC2 console, locate the EC2 instances that you plan to install Network Flow Monitor agents on.
- Attach the CloudWatchNetworkFlowMonitorAgentPublishPolicy to the IAM role for each instance.
- If an instance doesn’t have an IAM role attached, choose a role by doing the following:
  - Under Actions, choose Security.
  - Choose Modify IAM role, or create a new role by choosing Create new IAM role.
  - Choose a role for the instance, and attach the CloudWatchNetworkFlowMonitorAgentPublishPolicy policy.

Add this policy to the IAM role.
- Network Flow Monitor requires you to grant it specific permissions so that the feature can send metrics to Amazon CloudWatch on your behalf and create topologies of network flows. You must enable the feature to grant the service-linked role the required permissions and create a scope for monitoring in Network Flow Monitor.

{
  "Version" : "2012-10-17",
  "Statement" : [
    {
      "Effect" : "Allow",
      "Action" : [
        "networkflowmonitor:Publish"
      ],
      "Resource" : "*"
    }
  ]
}

Attach the IAM policy to the EC2 instance(s).
- Click > Instance > Actions > Security > Modify IAM role
- Select the newly created IAM role.

Create a TCP Tracing Policy

apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "tcp"
spec:
  parser:
    tcp:
      enable: true

Add the TCP tracing policy.

tetra tracingpolicy add tcp.yaml

Validate the installation to check if Tetragon is up and running.

systemctl status tetragon-enterprise.service

● tetragon-enterprise.service - "Tetragon Enterprise - eBPF-based Security Observability and Runtime Enforcement Servic>
     Loaded: loaded (/usr/lib/systemd/system/tetragon-enterprise.service; enabled; preset: enabled)
     Active: active (running) since Tue 2025-01-28 06:21:00 UTC; 1 week 3 days ago
       Docs: https://docs.isovalent.com/
   Main PID: 1829 (tetragon)
      Tasks: 10 (limit: 4586)
     Memory: 321.5M (peak: 1.5G)
        CPU: 1h 13min 30.394s
     CGroup: /system.slice/tetragon-enterprise.service
             └─1829 /usr/local/bin/tetragon

Feb 03 06:21:16 ip-172-31-17-69 tetragon[1829]: time="2025-02-03T06:21:16Z" level=info msg="BPF events statistics: 5762>
Feb 03 06:21:16 ip-172-31-17-69 tetragon[1829]: time="2025-02-03T06:21:16Z" level=info msg="Observer events statistics">
Feb 04 06:21:16 ip-172-31-17-69 tetragon[1829]: time="2025-02-04T06:21:16Z" level=info msg="BPF events statistics: 6722>
Feb 04 06:21:16 ip-172-31-17-69 tetragon[1829]: time="2025-02-04T06:21:16Z" level=info msg="Observer events statistics">
Feb 05 06:21:16 ip-172-31-17-69 tetragon[1829]: time="2025-02-05T06:21:16Z" level=info msg="BPF events statistics: 7684

Check if the TCP tracing policy was loaded correctly.

tetra tracingpolicy list
ID   NAME   STATE     FILTERID   NAMESPACE   SENSORS          KERNELMEMORY
1    tcp    enabled   0          (global)    layer3_sensors   116.02 kB

How can you enable Network Flow Monitor in AWS?

Your Amazon EC2 instances must be able to connect to Amazon CloudWatch endpoints. This can be by Internet Gateway, NAT gateway, or Amazon CloudWatch Interface VPC endpoints. Interface VPC endpoints are powered by AWS PrivateLink, an AWS technology that enables private communication between AWS services using an elastic network interface with private IP addresses. To enable this private communication to Network Flow Monitor endpoints create the VPC endpoint.

Create VPC Endpoint

Click > VPC > Endpoints > Create Endpoint
- Give a name for the endpoint
- Enter the service name as networkflowmonitor
- Select the networkflowmonitor service.

Select the VPC for which the endpoint needs to be created.
- Select the subnet.
- Specify the security groups associated with the endpoint network interface.
  - The security group rules control the traffic to the endpoint network interface from resources in your VPC.
- When you create an interface endpoint, you can attach an endpoint policy that controls access to the service you are connecting to.

Enable Network Flow Monitor

Open the Amazon CloudWatch console
Click > Network Monitoring > Flow monitors.
Click > Enable Network Flow Monitor.

Create Network Monitors

Open the Amazon CloudWatch console
Click > Network Monitoring > Flow monitors.
You can also see the network flows that the agents are detecting. In Workload insights, you can identify top-talkers or flows with impairments. You can then use this information to create monitors.
Click > Create monitor.
For the Monitor name, enter the name you want to use for this monitor in Network Flow Monitor.
Select the local resources (one or more) for the network flows you want to monitor.
- Choose Subnet, VPC, or Availability Zone, then choose a resource. If you start by choosing specific network flows to monitor on the Workload insights page before beginning the Create monitor process, network flows are pre-populated for you.

How can you Monitor and Analyze Network Flows?

To see information for a monitor, on the Monitors tab, choose a monitor in the Monitors table. Then, select one of the tabs for more details.

Metrics are shown only for traffic flows that Network Flow Monitor classifies into designated destination categories. Metrics are displayed for flows between AZs (INTER_AZ), within AZs (INTRA_AZ), between VPCs (INTER_VPC).

Network health indicator (NHI): NHI alerts you to whether there were AWS network issues for one or more of the network flows tracked by your monitor during the time frame you’ve selected for viewing performance metrics. NHI is a binary value, that is, 1 or 0, which is shown in the console as Degraded or Healthy. NHI is shown as Degraded if there are issues with the portion of the AWS network. If the NHI is Degraded, you can view the Network health indicator bar graph for more information. The graph shows you when, during the selected time frame, there were AWS network issues for the network flows tracked by your monitor.
Traffic Summary: Observe the overall metrics for the flows tracked by this monitor for the period you’ve selected. The monitor shows the average round-trip time and the amount of data transferred for the flows. Be aware that RTT data can be sparse because RTT is not always calculated.

Conclusion

Hopefully, this tutorial gave you a good overview of how to build the custom image with Tetragon from AWS Marketplace, configure and publish network telemetry to Amazon CloudWatch Network Flow Monitor. We encourage you to explore Tetragon on the AWS Marketplace. If you want to learn more before starting, request a demo today. Click the button below to get in touch!

Schedule a demo with our experts