k8wiz
All graphs

Kubernetes graph

Autoscaling end to end: KEDA, the HPA it creates and Karpenter adding nodes

From queue depth to new machines: a KEDA ScaledObject reads Prometheus, creates an HPA that scales a Deployment from zero, and when the Pods don't fit, Karpenter's NodePool launches a NodeClaim and a node.

Scaling a workload on a business metric takes two layers. KEDA turns an outside signal, here the number of jobs waiting in a queue, into a HorizontalPodAutoscaler. When the new Pods don't fit on the nodes there are, Karpenter reads what they ask for and launches a machine that fits.

Neither knows about the other. The Pending Pod is the handover.

Interactive graph

Click a resource to see what it does, or follow the setup step by step.

Comes with an add-onYou write itCreated by a controllertrafficreferencescreatesselects by label

How it works

All 10 resources and what they do

Service prometheus-k8sv1
Where KEDA sends its query. The answer is one number, the jobs waiting in the render queue.
TriggerAuthentication prometheus-tokenkeda.sh/v1alpha1
Tells KEDA where to find the credentials for the trigger, here a bearer token in a Secret, so they stay out of the ScaledObject.
ScaledObject render-workerkeda.sh/v1alpha1
Names the workload, the bounds and the trigger. One replica per 20 waiting jobs, from 0 to 40. KEDA handles 0 to 1 itself and leaves 1 to 40 to the HPA it creates.
HorizontalPodAutoscaler keda-hpa-render-workerautoscaling/v2
Created and owned by KEDA. Its one metric is of type External, served by KEDA's metrics adapter. Edit the ScaledObject, never this object: KEDA overwrites it.
Deployment render-workerapps/v1
The workers. Their requests are what the scheduler and Karpenter size nodes by. No replicas in the manifest, so applying it doesn't fight the autoscaler.
Pod render-worker-…v1
Some fit on the nodes there are. The rest stay Pending with 0/5 nodes are available: 5 Insufficient cpu, which is the event Karpenter acts on.
NodePool generalkarpenter.sh/v1
The rules for nodes Karpenter may launch: which instance families, zones and capacity types, a ceiling on total CPU, and when to remove or replace nodes.
EC2NodeClass defaultkarpenter.k8s.aws/v1
The cloud side of a node: which image, which subnets and security groups (found by tag) and the IAM role the instance runs as.
NodeClaim general-x7k2pkarpenter.sh/v1
One request for one machine, with the instance types that would fit the Pending Pods. Karpenter creates it, launches the instance and tracks it until the node registers.
Node ip-10-0-42-17v1
The instance once its kubelet has joined, labelled with its NodePool. The scheduler places the Pending Pods on it, usually within a minute of the scale-up.

Common mistakes

  • A Deployment with replicas in Git and a GitOps tool that enforces it resets the count on every sync. Leave the field out, or tell Argo CD or Flux to ignore it.
  • Pods without requests don't make Karpenter add nodes: as far as it can tell they fit anywhere.
  • limits.cpu on the NodePool is a hard ceiling. When it is reached, Pods stay Pending and the NodePool's events say why.
  • Consolidation evicts Pods to pack nodes tighter. Long jobs need a PodDisruptionBudget or the karpenter.sh/do-not-disrupt: "true" annotation, and a grace period long enough to finish.
  • Two HPAs on one Deployment fight. Don't add your own next to the one KEDA creates; add a second trigger to the ScaledObject.

The manifests you write

Everything above that is marked "You write it", in one file. The rest comes from add-ons and controllers.

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: prometheus-token
  namespace: media
spec:
  secretTargetRef:
    - parameter: bearerToken
      name: prometheus-reader
      key: token
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: render-worker
  namespace: media
spec:
  scaleTargetRef:
    name: render-worker
  minReplicaCount: 0
  maxReplicaCount: 40
  cooldownPeriod: 300
  triggers:
    - type: prometheus
      metadata:
        serverAddress: http://prometheus-k8s.monitoring.svc:9090
        query: sum(queue_jobs_waiting{queue="render"})
        threshold: "20"
        activationThreshold: "1"
        authModes: bearer
      authenticationRef:
        name: prometheus-token
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: render-worker
  namespace: media
spec:
  selector:
    matchLabels:
      app: render-worker
  template:
    metadata:
      labels:
        app: render-worker
    spec:
      terminationGracePeriodSeconds: 120
      containers:
        - name: worker
          image: ghcr.io/acme/render-worker:5.2.0
          resources:
            requests:
              cpu: "2"
              memory: 4Gi
---
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general
spec:
  template:
    spec:
      nodeClassRef:
        group: karpenter.k8s.aws
        kind: EC2NodeClass
        name: default
      requirements:
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: karpenter.k8s.aws/instance-category
          operator: In
          values: ["c", "m"]
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64"]
      expireAfter: 720h
  limits:
    cpu: "400"
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 2m
---
apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: default
spec:
  role: KarpenterNodeRole-prod
  amiSelectorTerms:
    - alias: al2023@latest
  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: prod
  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: prod

Keep going

Kubernetes quizTest yourself on Pods, Deployments, Services, scheduling, storage, networking, security and kubectl.Try it in the playgroundAutoscale a Deployment with an HPAArticleHow to debug OOMKilled containers in KubernetesArticleCVE-2026-123158 (*sshole): the OpenSSH password bypass on AlpineArticleHow to drain Kubernetes nodes safely with PodDisruptionBudgets

More graphs