# How to expose LoadBalancer Services on bare metal with Cilium LB IPAM and L2 announcements

> Give bare-metal Services real LoadBalancer IPs with Cilium by creating an IP pool, announcing it over ARP and tuning failover.

Published: 2026-11-12
Author: k8wiz Team
URL: https://k8wiz.com/articles/expose-loadbalancer-services-on-bare-metal-with-cilium-l2-announcements

On a cloud provider, a Service of type `LoadBalancer` gets an external IP within a minute. On bare metal or in a lab cluster, the same Service sits at `<pending>` forever, because nothing in the cluster knows how to hand out an address or make the network deliver traffic to it.

Many teams install MetalLB to fill that gap. If you already run Cilium, you don't need a second component. Cilium has two features that together do the same job. LB IPAM allocates addresses from pools you define, and L2 announcements make one node answer ARP requests for each address so the rest of the LAN can reach it. This guide sets both up on Cilium 1.20, explains how failover works and shows how to debug the usual failures. L2 announcements is still marked beta in the Cilium docs, so test it before you depend on it.

## How the two pieces fit together

LB IPAM only does bookkeeping. It watches Services of type `LoadBalancer`, picks a free address from a matching `CiliumLoadBalancerIPPool` and writes it into the Service's `.status.loadBalancer.ingress`. It is always present but dormant, and it wakes up when you create the first pool. On its own it makes the IP show up in `kubectl get svc`, but no packet will ever arrive.

L2 announcements does the delivery. For every selected Service, the Cilium agents on eligible nodes run a leader election through a Kubernetes Lease. The winner answers ARP requests (NDP for IPv6) for the Service IP with its own MAC address. Clients on the same layer 2 segment send traffic to that node, and Cilium's service load balancing forwards it to the backend pods, wherever they run.

This is the same model as a classic VRRP pair or MetalLB in layer 2 mode. One node takes all inbound traffic for a given IP, so you get failover but not load spreading across nodes before the cluster. If you need traffic spread across nodes, or your network team runs BGP, Cilium's BGP control plane is the better tool. For an office network, a homelab or a rack with a flat VLAN, L2 is simpler.

## Step 1: check the prerequisites

L2 announcements requires Cilium's kube-proxy replacement. The announcing node hands traffic to Cilium's own eBPF service load balancer, so the feature doesn't work when kube-proxy handles Services. Check the current setting from any agent pod.

```bash
kubectl -n kube-system exec ds/cilium -- cilium-dbg config --all | grep KubeProxyReplacement
```

```text
KubeProxyReplacement              : true
```

If it says `false`, you have a bigger migration ahead, since switching to kube-proxy replacement means removing kube-proxy and pointing Cilium at the API server directly. Do that first and come back.

You also need a range of free addresses on the node network. Pick addresses that your DHCP server won't hand out and that no other device uses. In this guide the nodes sit on `192.168.10.0/24` and the range `192.168.10.200` to `192.168.10.230` is reserved for Services.

Finally, the network interface that carries this traffic must be one Cilium manages. If you set the `devices` Helm value explicitly, the interface has to be in that list. If you left `devices` empty, Cilium auto-detects the interfaces that have addresses, which usually includes the main NIC.

## Step 2: enable L2 announcements and raise the client rate limit

Leader election is not free. Every node that is a candidate for a Service keeps polling its Lease, and the holder renews it every few seconds. That means steady traffic to the API server from each Cilium agent, and the agent's client rate limit can become the bottleneck.

The Cilium docs give a worst-case formula for the agent's request rate.

```text
QPS = number of announced services / leaseRenewDeadline
```

With the default renew deadline of 5 seconds, 40 announced Services cost about 8 requests per second. The chart's default agent limit in 1.20 is 10 QPS with a burst of 20, which is tight once you add the agent's other API traffic. With 65 Services and a 2 second deadline, you'd need around 32.5 QPS. Set the limit a little above your calculated value and the burst somewhat higher.

Enable the feature with Helm. Replace the API server address and port with your own (they are the values you already use for kube-proxy replacement).

```bash
helm upgrade cilium cilium/cilium \
  --version 1.20.4 \
  --namespace kube-system \
  --reuse-values \
  --set kubeProxyReplacement=true \
  --set k8sServiceHost=192.168.10.10 \
  --set k8sServicePort=6443 \
  --set l2announcements.enabled=true \
  --set k8sClientRateLimit.qps=30 \
  --set k8sClientRateLimit.burst=50
```

The chart doesn't restart the agents when the ConfigMap changes (`rollOutCiliumPods` defaults to `false`), so restart them yourself, along with the operator.

```bash
kubectl -n kube-system rollout restart ds/cilium deployment/cilium-operator
kubectl -n kube-system rollout status ds/cilium
kubectl -n kube-system exec ds/cilium -- cilium-dbg config --all | grep EnableL2Announcements
```

```text
EnableL2Announcements             : true
```

Using the Helm chart matters here. It also grants the agent's ClusterRole permission to create, get, update, list and delete `leases` in `coordination.k8s.io`. Clusters that enable the flag by editing the ConfigMap by hand often miss that, and the agent logs show a `forbidden` error on the lease.

## Step 3: create an IP pool

The pool tells LB IPAM which addresses it may hand out. A block is either a CIDR or a `start` and `stop` pair. The range form is handy on a shared LAN, where you rarely own a clean CIDR.

```yaml
apiVersion: cilium.io/v2
kind: CiliumLoadBalancerIPPool
metadata:
  name: lan-pool
spec:
  blocks:
    - start: "192.168.10.200"
      stop: "192.168.10.230"
```

Watch the dashes. `start` and `stop` together form one block, so they must sit in a single list item. Two separate dashes create two broken blocks.

```bash
kubectl apply -f lan-pool.yaml
kubectl get ippools
```

The output shows whether the pool is disabled, whether it conflicts with another pool and how many addresses are still free. Check the `CONFLICTING` column every time you add a pool. Pools may not overlap, and when they do, the newest one is marked conflicting and stops allocating. That is a soft error, so nothing else warns you.

Two more pool options are worth knowing. With a CIDR block, LB IPAM hands out every address including the first and last; set `spec.allowFirstLastIPs: "No"` if your network treats those as network and broadcast addresses. And `spec.serviceSelector` restricts a pool to matching Services, which lets you give a tenant namespace its own range with the special label `io.kubernetes.service.namespace`.

## Step 4: create an L2 announcement policy

Without a policy, nothing is announced. The policy picks which Services to announce, which nodes may announce them and on which interfaces.

```yaml
apiVersion: cilium.io/v2alpha1
kind: CiliumL2AnnouncementPolicy
metadata:
  name: lan-announce
spec:
  nodeSelector:
    matchExpressions:
      - key: node-role.kubernetes.io/control-plane
        operator: DoesNotExist
  interfaces:
    - ^eth[0-9]+
  loadBalancerIPs: true
  externalIPs: false
```

The `loadBalancerIPs` and `externalIPs` fields both default to `false`. A policy that sets neither is valid, applies cleanly and announces nothing, which is the most common reason a first attempt fails silently. Set `loadBalancerIPs: true` to announce addresses assigned by LB IPAM.

The node selector keeps control plane nodes out of the election. The node that wins a Service's Lease carries all of its inbound traffic, so put it on workers sized for that. The `interfaces` field takes Go regular expressions, ORed together. Adjust the pattern to your NIC names, such as `^ens[0-9]+` or `^bond0$`. Leaving the field out announces on every interface Cilium manages. The interface filter is not a security boundary; the Service still answers on other interfaces if someone points traffic at it.

There is no `serviceSelector` here, so every eligible Service is announced. A Service is eligible when its `loadBalancerClass` is unset or set to `io.cilium/l2-announcer`.

```bash
kubectl apply -f lan-announce.yaml
kubectl get l2announcement
```

## Step 5: expose a Service and test it

Create a test deployment and a `LoadBalancer` Service.

```bash
kubectl create deployment echo --image=nginx --replicas=2
kubectl expose deployment echo --type=LoadBalancer --port=80
kubectl get svc echo
```

```text
NAME   TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)        AGE
echo   LoadBalancer   10.96.41.17    192.168.10.200   80:31544/TCP   6s
```

Now check that a node took the Lease. The Lease lives in the Cilium namespace and is named `cilium-l2announce-<namespace>-<service>`.

```bash
kubectl -n kube-system get lease cilium-l2announce-default-echo
```

```text
NAME                             HOLDER     AGE
cilium-l2announce-default-echo   worker-2   12s
```

From a machine on the same LAN, `curl http://192.168.10.200` should return the nginx welcome page. Running `ip neigh show 192.168.10.200` on that machine shows the MAC address of `worker-2`.

To pin an address, use the `lbipam.cilium.io/ips` annotation instead of the deprecated `spec.loadBalancerIP` field. To put several Services on one IP, give them the same `lbipam.cilium.io/sharing-key` annotation. They share an address as long as their ports don't clash.

## Step 6: understand and tune failover

When the holder node dies, the other candidates notice that the Lease wasn't renewed within `leaseDuration` and race to take it. The first request the API server processes wins. The new holder sends a gratuitous ARP reply on its interfaces so clients update their ARP tables straight away.

Three Helm values control the timing. `l2announcements.leaseDuration` defaults to 15s, `l2announcements.leaseRenewDeadline` to 5s and `l2announcements.leaseRetryPeriod` to 2s. Failover takes between `leaseDuration - leaseRenewDeadline` and `leaseDuration + leaseRenewDeadline`, so 10 to 20 seconds with the defaults.

You can shorten it. With a 3s duration, 1s renew deadline and 200ms retry period, failover drops to 2 to 4 seconds. The cost is more API traffic, since the renew rate goes up fivefold and the QPS formula from Step 2 grows with it. Raise the rate limit at the same time.

```bash
helm upgrade cilium cilium/cilium \
  --version 1.20.4 \
  --namespace kube-system \
  --reuse-values \
  --set l2announcements.leaseDuration=3s \
  --set l2announcements.leaseRenewDeadline=1s \
  --set l2announcements.leaseRetryPeriod=200ms \
  --set k8sClientRateLimit.qps=60 \
  --set k8sClientRateLimit.burst=100
kubectl -n kube-system rollout restart ds/cilium
```

Gratuitous ARP has a catch. Some devices ignore unsolicited ARP replies because they can be used for spoofing. Those clients keep sending to the dead node until their ARP cache entry expires, so their outage lasts longer than the Lease settings suggest. Firewalls and some managed switches are the usual suspects.

## Troubleshooting a Service that doesn't answer

Work from the top down. First, if `EXTERNAL-IP` stays `<pending>`, the problem is LB IPAM, not L2. The Service's status conditions say why.

```bash
kubectl get svc echo -o jsonpath='{.status.conditions}'
```

A `no_pool` reason means no enabled pool matches the Service. Check the pool's `serviceSelector` and the `CONFLICTING` column.

If the Service has an IP but no Lease exists, the policy doesn't select it, or the agent can't write Leases. Search the agent logs for `forbidden` (RBAC) or `client-side throttling` (rate limit too low). Throttling shows up as intermittent renew failures and frequent leader changes.

If the Lease exists, look at the holder's internal state. Find the Cilium pod on the holder node and dump its announcement table.

```bash
kubectl -n kube-system get pod -l app.kubernetes.io/name=cilium-agent -o wide --field-selector spec.nodeName=worker-2
kubectl -n kube-system exec cilium-7xk2p -- cilium-dbg shell -- db/show l2-announce
```

```text
# IP             NetworkInterface
192.168.10.200   eth0
```

A missing entry means the interface regex doesn't match a selected device. Run `cilium-dbg shell -- db/show devices` in the same pod and look at the `Selected` column. Only selected devices can announce.

If the table looks right but clients still can't connect, check `externalTrafficPolicy`. L2 announcements ignores the `Local` policy and may elect a node that has no backend pod. That node drops the traffic, because `Local` forbids forwarding to another node. Keep announced Services on `externalTrafficPolicy: Cluster`. Unless you run Cilium's DSR mode, that means backends see the forwarding node's address instead of the client's, which is the price of this design.