Isovalent Cilium Enterprise – Network Policies

Transcript

Hello everyone, my name is Duffy and I’m the Field CTO at Isovalent. We work on Cilium and generally help people make amazing things with eBPF and networking. Today, we’re going to play with network policy. We’re going to explore some of the gotchas and we’re going to play with how to actually generate rules or how to generate network policy and things like that with Cilium and Hubble. Let’s play around and see what we got. So the first thing I’ve got is I’ve got a couple of folks in tenant A. I’ve got a couple of applications there: a front-end service and a backend service. And it seems like the communication back and forth between the front-end and the backend service is not working. So we’re going to troubleshoot that, get that fixed up. And then also, they are unable to reach from the front-end service to the backend service in tenant B, and they’re unable to reach Google.com in the outside world from the front-end service. So let’s explore that and play with like how to actually generate those network policies and how to be very specific about what access we’re going to allow. So let’s start with our trouble report.

I’m going to jump in here and try to reproduce that connectivity problem that my user was telling me about. So dash n tenant A front-end service, curl backend service dot tenant A. Yeah, looks like it’s not working for sure. So let’s see what the problem is here. I’m going to break out a tool called Hubble and Hubble allows me to enter to query network event data that’s happening within the cluster. This is across the entire cluster, across all pods communicating. I can actually do a lot with Hubble query language, but first, I’m just going to take a look at the flows that are happening within the tenant A namespace and see if I can figure out what’s happening. Some other ways that I’ve tried to solve this in the past are things like tcpdump, which are really focused on a very specific interface, things like that. This is actually, in my opinion, much better. So let’s do Hubble observe dash n tenant A. Ah, look at that. So what we see is that the communication between the front-end service, the identity of this pod tenant A front-end service, is trying to communicate with coreDNS and that policy, and that’s being policy dropped.

So let’s take a look at the policies that are defined for this namespace. Get netpol dash a. There’s only just the one network policy. And if I do kubetl get network policy dash n tenant a yaml. I can see that this is a pretty generic network policy. It matches policy types ingress and egress, and it’s allowing any pod within that namespace to communicate. So this policy is basically just allowing pods within the same namespace to communicate, but nothing in this policy allows this pod to communicate with the DNS server. So that makes sense, right? The way the rules work in network policy is if any rule matches a pod, then any traffic not with it defined within that rule will fall to an implicit deny. So the next thing I want to do is I want to allow that traffic. So let’s take a look at this new policy that I created here. This new policy is going to be a Cilium network policy, and I’m allowing communication between the net pods in the namespace that I’m applying it to to A. I’m allowing egress to kube-system, and I’m allowing egress to any pods that match the k8s app kube-dns labels query. I’m allowing that on port 53 on protocol UDP. And then I’m going to increase the visibility here. I’m actually both creating a rule that says you can send any query to the DNS server, but I’m also creating a rule here that increases the visibility and into those DNS queries themselves. Let’s go ahead and apply this. Tenant A and and if we were on our test, things are working. All right! Let’s try to connect with tenant B real quick. Oh, that’s not working. Let’s try google.com. Also not working. Okay, so next, I’m going to actually use, if I looked at Hubble observe again, I’m pretty sure I would see the same output. Yeah, policy dropped to Google.com. Traffic being forwarded to the DNS server, right? So I was able to resolve Google.com, but I was not able to communicate to it. Um, yeah, all right. So next up, we want to do we want to actually define the network policy that would allow this stuff. So for that, I’m actually going to jump over to Hubble UI. I’m going to click on my tenant-A namespace, and I should be able to see the flows that are moving back and forth between that tenant A namespace. Let’s do a reload here. There we go. So there are the traffic that’s being dropped, and we can see that it’s being dropped by policy, just like we saw before. And if I click on review here, it opens up the network policy editor, which is actually pretty neat. So I’m going to create a new policy. Let’s create a new empty policy here. And I’m going to look at the flows that are being dropped, and I’m going to go ahead and add rules for them to our new policy.

So this policy, we’re gonna name it, allow egress. I’m gonna put this in the tenant A namespace, save. So inside here, we’re going to allow a match on Google.com. We’re also going to add this rule, so we’re going to allow a match on k8’s app back in service and tenant B. So both of these rules should apply in this network name and this policy. I’m going to go ahead and download this policy and we’re going to download it into the demo directory. We’ll hit save. Okay, that’s working. Let’s jump back over here. We’ll do a kubectl apply -f egress allow egress yaml, and that allow egress yaml policy looks like this, just like I did in the UI. So let’s go ahead and run our test again, and that’s working. And this, let’s try hitting that back end, back end service tenant B, also working. All right, so I’ve shown you a couple of different tools for diagnosing and creating network policy and also just for troubleshooting things. I hope they, I hope they were helpful, and I hope you enjoyed it. And I’ll see you all next time.