# Restrict Pod egress to one external domain with Cilium toFQDNs

> A CiliumNetworkPolicy should let payments reach api.stripe.com and nothing else on the internet. It blocks everything. Read the drops with Hubble and fix it.

Level: Advanced · Topic: Networking · About 15 minutes
Try it in the browser: https://k8wiz.com/playground/restrict-pod-egress-to-a-domain-with-cilium-tofqdns

The playground is a simulated Kubernetes cluster that runs in the browser. kubectl output and errors follow the real ones.

## The situation

`payments` in the `shop` namespace calls two things: the payment provider at `https://api.stripe.com` and the internal `ledger` Service in `finance`. Security wants it to reach nothing else outside the cluster.

A colleague applied the `payments-egress` CiliumNetworkPolicy yesterday. Since then `payments` can't reach the provider or the ledger, and the calls time out.

Fix the policy without opening it up. `hubble observe` shows what Cilium forwards and drops, and `payments-egress.yaml` is in the editor. Test with `kubectl exec deploy/payments -- curl -m 5 https://api.stripe.com`.

## Objectives

1. See what Cilium drops for payments (Make a call, then `hubble observe --namespace shop --verdict DROPPED`)
2. payments reaches https://api.stripe.com
3. payments reaches the ledger at http://ledger.finance:8080
4. payments still can't reach any other site, such as https://example.org

## Hints

1. `kubectl exec deploy/payments -- curl -m 5 https://api.stripe.com` times out, then `hubble observe --namespace shop --verdict DROPPED` shows the connection to `api.stripe.com:443` as `EGRESS DENIED`. The name resolved, so DNS itself works.
2. A `toFQDNs` rule only allows addresses Cilium has seen the Pod look up, and it only sees lookups that go through its DNS proxy. The DNS rule needs an L7 block for that: under `toPorts`, add `rules: {dns: [{matchPattern: "*"}]}`. `hubble observe --protocol dns` then shows the queries.
3. `matchName` is an exact match. `stripe.com` doesn't cover `api.stripe.com`. Use `matchName: api.stripe.com`, or `matchPattern: "*.stripe.com"` for every subdomain.
4. Once a policy has egress rules, everything it doesn't list is denied, the ledger included. Add a third rule with `toEndpoints` for `k8s:io.kubernetes.pod.namespace: finance` and `app: ledger` on port 8080. It can't share a rule with `toFQDNs`.
5. Edit `payments-egress.yaml` and `kubectl apply -f payments-egress.yaml`. Don't reach for `toEntities: [world]`: it would allow every site.

## Solution

```bash
kubectl exec deploy/payments -- curl -s -m 5 https://api.stripe.com
```
```bash
hubble observe --namespace shop --verdict DROPPED
```
```bash
cat <<'EOF' | kubectl apply -f -
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: payments-egress
  namespace: shop
spec:
  endpointSelector:
    matchLabels:
      app: payments
  egress:
  - toEndpoints:
    - matchLabels:
        k8s:io.kubernetes.pod.namespace: kube-system
        k8s:k8s-app: kube-dns
    toPorts:
    - ports:
      - port: "53"
        protocol: ANY
      rules:
        dns:
        - matchPattern: "*"
  - toFQDNs:
    - matchName: api.stripe.com
    toPorts:
    - ports:
      - port: "443"
        protocol: TCP
  - toEndpoints:
    - matchLabels:
        k8s:io.kubernetes.pod.namespace: finance
        app: ledger
    toPorts:
    - ports:
      - port: "8080"
        protocol: TCP
EOF
```
```bash
kubectl exec deploy/payments -- curl -s -m 5 https://api.stripe.com
```
```bash
kubectl exec deploy/payments -- curl -s -m 5 http://ledger.finance:8080
```
```bash
hubble observe --namespace shop --protocol dns
```

## What it teaches

- `toFQDNs` turns a name into allowed addresses at run time. Cilium's DNS proxy watches the Pod's lookups and allows the addresses in the answers, so the Pod can only reach what it resolved itself while the policy was in place.
- The proxy only sees DNS when the rule that allows port 53 has a `rules.dns` block. Without it lookups work, nothing is learned, and every connection to the domain is dropped.
- `matchName` matches one exact name. `matchPattern` takes `*` as a wildcard within the name, so `*.stripe.com` covers subdomains and not `stripe.com` itself.
- A policy with egress rules denies all other egress from the Pods it selects. Internal dependencies need their own `toEndpoints` rule.
- `hubble observe --verdict DROPPED` names the source, the destination and the reason for each dropped connection, which beats guessing from a timeout.