apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: checkout
namespace: shop
spec:
replicas: 6
selector:
matchLabels:
app: checkout
template:
metadata:
labels:
app: checkout
spec:
containers:
- name: checkout
image: ghcr.io/acme/checkout:2.5.0
ports:
- containerPort: 8080
strategy:
canary:
stableService: checkout-stable
canaryService: checkout-canary
trafficRouting:
plugins:
argoproj-labs/gatewayAPI:
httpRoute: checkout
namespace: shop
steps:
- setWeight: 10
- pause: { duration: 5m }
- analysis:
templates:
- templateName: success-rate
- setWeight: 50
- pause: { duration: 10m }
---
apiVersion: v1
kind: Service
metadata:
name: checkout-stable
namespace: shop
spec:
selector:
app: checkout
ports:
- port: 80
targetPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: checkout-canary
namespace: shop
spec:
selector:
app: checkout
ports:
- port: 80
targetPort: 8080
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: checkout
namespace: shop
spec:
parentRefs:
- name: public
namespace: infra
hostnames:
- shop.example.com
rules:
- backendRefs:
- name: checkout-stable
port: 80
weight: 100
- name: checkout-canary
port: 80
weight: 0
---
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
name: success-rate
namespace: shop
spec:
metrics:
- name: success-rate
interval: 1m
count: 5
failureLimit: 1
successCondition: result[0] >= 0.99
provider:
prometheus:
address: http://prometheus-k8s.monitoring.svc:9090
query: |
sum(rate(http_requests_total{service="checkout-canary",code!~"5.."}[2m]))
/ sum(rate(http_requests_total{service="checkout-canary"}[2m]))Argo CD graph
Argo Rollouts canary: traffic weights on an HTTPRoute and analysis from Prometheus
How Argo Rollouts runs a canary: a Rollout owns two ReplicaSets, shifts traffic by editing HTTPRoute weights through the Gateway API plugin, and an AnalysisTemplate queries Prometheus to promote or abort.
A Deployment's rolling update replaces Pods at a fixed pace and stops only when Pods fail their probes. Argo Rollouts replaces the Deployment with a Rollout that moves traffic in steps, measures the new version while it takes real requests, and goes back by itself when the numbers are bad.
Here the traffic split lives in a Gateway API HTTPRoute and the measurement is a Prometheus query. Argo CD syncs the Rollout from Git like any other object.
Interactive graph
Click a resource to see what it does, or follow the setup step by step.
How it works
All 10 resources and what they do
- Rollout checkoutargoproj.io/v1alpha1
- A Deployment-shaped object with a
strategy.canary. The steps set a traffic weight, pause, and run an analysis. It names the two Services and the HTTPRoute it may edit. - ReplicaSet checkout-6d5f8c7b9apps/v1
- The current version, 2.4.1. It keeps serving most of the traffic during the canary and is scaled down only after the new version is promoted.
- ReplicaSet checkout-79c4d6f5bapps/v1
- The new version, 2.5.0, created when the Pod template changes. The controller sizes it to match the traffic weight.
- Service checkout-stablev1
- You write it with the plain
app: checkoutselector. The controller adds the stable ReplicaSet'srollouts-pod-template-hashto the selector, so it only ever reaches the old Pods. - Service checkout-canaryv1
- The same manifest under another name. Its selector gets the canary ReplicaSet's hash, so it reaches only the new Pods.
- Gateway publicgateway.networking.k8s.io/v1
- The platform team's Gateway. Requests for
shop.example.comarrive here and follow the HTTPRoute attached to it. - HTTPRoute checkoutgateway.networking.k8s.io/v1
- Two backends with weights. You write 100 and 0; the Gateway API plugin rewrites the two numbers at each
setWeightstep, and the Gateway splits real requests accordingly. - AnalysisTemplate success-rateargoproj.io/v1alpha1
- How to judge a version: a query, how often to run it, and what counts as success. Here the canary's share of non-5xx responses must stay at 99% or more, and two failed measurements abort.
- AnalysisRun checkout-79c4d6f5b-2argoproj.io/v1alpha1
- One execution of the template for one rollout step. It stores each measurement and ends as
Successful,FailedorInconclusive. - Service prometheus-k8sv1
- Answers the analysis query with the canary's success rate over the last two minutes.
Common mistakes
- Argo CD sees the edited weights as drift and puts 100 and 0 back in the middle of a canary. Add
ignoreDifferencesfor the route'sbackendRefsweights, or leave the weights out of Git. - The Gateway API support is a plugin. The controller only loads it when the
argo-rollouts-configConfigMap lists it undertrafficRouterPlugins. - With too little traffic on the canary the query returns no data, and the measurement is an error, not a pass. Give the step real traffic, or handle the empty result in the condition.
- An aborted Rollout stays aborted with the new image still in its spec. Fix forward with a new image, or revert the commit;
kubectl argo rollouts retry rollout checkoutstarts the same version again. - A HorizontalPodAutoscaler must target the Rollout, not a Deployment.
scaleTargetReftakesapiVersion: argoproj.io/v1alpha1andkind: Rollout.
The manifests you write
Everything above that is marked "You write it", in one file. The rest comes from add-ons and controllers.
Keep going
More graphs
- Deployment, Service and Ingress: how a request reaches your Pods
- Gateway API: GatewayClass, Gateway and HTTPRoute with a canary split
- PVC, StorageClass and CSI: how a cloud disk ends up in your Pod
- RBAC: how a ServiceAccount gets its permissions
- cert-manager with Let's Encrypt: Certificate, CertificateRequest, Order and Challenge
- A private CA with cert-manager and trust-manager
- Istio traffic management: Gateway, VirtualService, DestinationRule and policies
- Flux dependencies: Git and OCI sources, Kustomizations and HelmReleases
- Argo CD: ApplicationSet, AppProject and the Applications it generates
- Inside an OCI image: tag, index, manifests, config and layers
- OCI artifacts: Helm charts, Flux artifacts and image volumes
- A Helm release: chart, values, revisions and the objects Helm owns
- Kustomize: a base, two overlays, a component and generated ConfigMaps
- A StatefulSet database: headless Service, per-Pod volumes, snapshots and restore
- Autoscaling end to end: KEDA, the HPA it creates and Karpenter adding nodes
- Prometheus Operator: ServiceMonitor, PrometheusRule and the path to an alert
- External Secrets Operator: from a cloud secret manager to a Pod, with rotation
- Kyverno CEL policies: ValidatingPolicy, MutatingPolicy, GeneratingPolicy and exceptions
- Cilium egress to a domain: CiliumNetworkPolicy, the DNS proxy and toFQDNs
- Istio ambient mode: ztunnel, waypoint proxies and authorization without sidecars