# Cosign: keyless image signing, attestations and verification with Kyverno

> How a CI job signs an image with cosign and no long-lived key: an OIDC token, a 10-minute Fulcio certificate, a Rekor log entry and a Sigstore bundle attached to the image. Kyverno checks all of it before a Pod may run.

Interactive graph: https://k8wiz.com/graphs/cosign-keyless-signing-and-kyverno-image-verification

Cosign signs container images and other OCI artifacts. In keyless mode, the usual one in CI, there is no private key to store or rotate. The job proves who it is with an OIDC token, Sigstore's Fulcio CA turns that identity into a certificate that is valid for 10 minutes, and the Rekor transparency log records that the signature existed while the certificate was valid.

Cosign 3 stores the signature, the certificate and the log proof together in one Sigstore bundle, attached to the image through the OCI referrers API, so the image digest never changes. A Kyverno `ImageValidatingPolicy` reads those referrers at admission and admits a Pod only if a trusted identity signed its image.

## Resources

### CI job release workflow

outside the cluster

Builds and pushes the image, then signs the digest it pushed with `cosign sign --yes`. `id-token: write` lets the job request an OIDC token from GitHub. The job's identity is the workflow file and the ref it ran from.

### OIDC provider token.actions.githubusercontent.com

outside the cluster

Issues the job a short-lived ID token. Its `sub` and `job_workflow_ref` claims name the repository, workflow and ref, which become the identity in the certificate. The issuer URL is the second half of every keyless trust rule.

### Certificate authority Fulcio

outside the cluster

Checks the ID token, then issues a code-signing certificate for the public half of a key pair cosign generated in memory. The certificate binds that key to the workflow identity and the issuer, and it expires after 10 minutes. The private key is thrown away after signing.

### Transparency log Rekor

outside the cluster

An append-only public log. Cosign records the signature and certificate there and gets back a signed entry with a timestamp. That entry proves the signature was made while the 10-minute certificate was valid, so it still verifies years later. Anyone can search the log for signatures made with their identity.

### Trust root Sigstore TUF root

outside the cluster

Distributes, through TUF, the Fulcio root certificates and the Rekor public keys that verifiers trust, plus the service URLs cosign 3 uses when signing. cosign and Kyverno fetch and cache it, so a key rotation doesn't need a new release.

### Image index ghcr.io/acme/shop@sha256:5e1a…

made by a build or publishing tool

What gets signed: the digest, not the tag. A tag can be moved to other content after signing; the digest is the hash of the exact bytes. Signing never changes this manifest, so the digest stays the same.

### Signature Sigstore bundle

made by a build or publishing tool

Pushed by `cosign sign`. One manifest whose single layer is the bundle: the signature over the image digest, the Fulcio certificate and the Rekor entry with its inclusion proof. Its `subject` points at the image, so `GET /v2/acme/shop/referrers/sha256:5e1a…` lists it.

### Attestation SBOM (SPDX)

made by a build or publishing tool

Pushed by `cosign attest --type spdxjson --predicate sbom.spdx.json`. An in-toto statement says "this SBOM describes image sha256:5e1a…" and is signed the same keyless way. It's another referrer of the image, so it travels with the image to every registry that copies it with its referrers.

### ImageValidatingPolicy require-signed-shop

you write it, policies.kyverno.io/v1

Matches images under `ghcr.io/acme/*` and trusts exactly one identity: the release workflow on `main`, as vouched for by GitHub's issuer. `mutateDigest` rewrites tags to the verified digest, so the node runs what was checked.

### Admission controller kyverno-admission-controller

comes with an add-on, namespace kyverno

Kyverno's webhook. For each matching image it resolves the tag to a digest, lists the image's referrers, and checks each bundle: the certificate chains to Fulcio, the identity and issuer match an attestor, the Rekor entry is valid, and the signature covers this digest. It needs registry credentials for private images.

### Deployment shop

you write it, apps/v1, namespace shop

Refers to the image by tag, as most manifests do. The policy checks the Pods its ReplicaSet creates.

### Pod shop-7d9c…

created by a controller, v1, namespace shop

Admitted only after verification. With `mutateDigest`, its image reads `ghcr.io/acme/shop:4f2c8e1@sha256:5e1a…`, so a tag moved after admission can't change what the kubelet pulls.

## How the resources connect

- CI job release workflow → Image index ghcr.io/acme/shop@sha256:5e1a…: docker buildx push (creates)
- CI job release workflow → OIDC provider token.actions.githubusercontent.com: request ID token (traffic)
- CI job release workflow → Certificate authority Fulcio: public key + ID token (traffic)
- Certificate authority Fulcio → OIDC provider token.actions.githubusercontent.com: verifies token (references)
- CI job release workflow → Transparency log Rekor: upload signature + cert (traffic)
- CI job release workflow → Signature Sigstore bundle: cosign sign (creates)
- CI job release workflow → Attestation SBOM (SPDX): cosign attest (creates)
- Certificate authority Fulcio → Signature Sigstore bundle: 10-minute certificate (creates)
- Transparency log Rekor → Signature Sigstore bundle: log entry + proof (creates)
- Signature Sigstore bundle → Image index ghcr.io/acme/shop@sha256:5e1a…: subject (references)
- Attestation SBOM (SPDX) → Image index ghcr.io/acme/shop@sha256:5e1a…: subject (references)
- Trust root Sigstore TUF root → Certificate authority Fulcio: root certificates (references)
- Trust root Sigstore TUF root → Transparency log Rekor: log public keys (references)
- Deployment shop → Pod shop-7d9c…: creates via ReplicaSet (creates)
- Pod shop-7d9c… → Image index ghcr.io/acme/shop@sha256:5e1a…: image (references)
- Admission controller kyverno-admission-controller → ImageValidatingPolicy require-signed-shop: loads (references)
- Admission controller kyverno-admission-controller → Signature Sigstore bundle: GET referrers (traffic)
- Admission controller kyverno-admission-controller → Trust root Sigstore TUF root: trusted roots (traffic)
- Admission controller kyverno-admission-controller → Pod shop-7d9c…: admits, pins digest (traffic)

## How it works

1. **Build and push, then sign the digest.** The workflow pushes `ghcr.io/acme/shop:4f2c8e1`. The registry returns the digest `sha256:5e1a…`, and that digest is what gets signed. Signing a tag would sign whatever the tag points to at that moment, which may not be what you built.
2. **Prove who is signing with an OIDC token.** `cosign sign --yes` asks GitHub for an ID token. The token names the repository, the workflow file and the ref, and is signed by `https://token.actions.githubusercontent.com`. No secret is stored anywhere: the token only exists while the job runs.
3. **Fulcio turns the identity into a short-lived certificate.** Cosign generates a key pair in memory and sends the public key with the token to Fulcio. Fulcio checks the token and issues a certificate for that key, with the workflow identity in its Subject Alternative Name and the issuer in an extension. It expires after 10 minutes.
4. **Sign and record it in Rekor.** Cosign signs the image digest with the private key and uploads the signature and certificate to Rekor. Rekor returns a signed entry with a timestamp and an inclusion proof. The private key is discarded: the certificate expires, but the log proves the signature was made while it was valid.
5. **Attach the bundle to the image.** Cosign 3 packs the signature, certificate and log entry into one Sigstore bundle and pushes it as a manifest whose `subject` is the image. The image is untouched. On a registry without the referrers API, the reference is kept in a tag named after the digest, `sha256-5e1a…`.
6. **Attach an SBOM as a signed attestation.** `cosign attest --yes --type spdxjson --predicate sbom.spdx.json ghcr.io/acme/shop@sha256:5e1a…` wraps the SBOM in an in-toto statement about that digest and signs it the same way. Provenance from the build works the same, with a different `--type`.
7. **Write down which identity you trust.** A keyless signature only proves that some identity signed. Trust comes from naming it: the `subject` is the workflow on `main`, and the `issuer` is GitHub's. A signature from a fork, another workflow or another branch has a different subject and fails.
8. **Kyverno verifies at admission.** The Deployment's ReplicaSet creates a Pod. Kyverno resolves `:4f2c8e1` to `sha256:5e1a…`, fetches the image's referrers, and checks each bundle against the Sigstore trust root and the attestor. If none passes, the Pod is denied and the ReplicaSet reports `FailedCreate`.
9. **The Pod runs the digest that was checked.** With `mutateDigest: true`, Kyverno writes the digest into the Pod's image. The kubelet pulls exactly what was verified, even if someone moves the tag a second later.

## Common mistakes

- Trusting any identity from an issuer, such as `subject: "*"` with GitHub's issuer, trusts every repository on GitHub. Pin the repository, the workflow file and the ref.
- Signing a tag (`cosign sign ghcr.io/acme/shop:latest`) signs whatever the tag points at when cosign resolves it. Use the digest from the build step's output.
- Cosign 3 attaches bundles through the referrers API. A verifier, mirror or registry-copying tool that only knows the old `sha256-….sig` tags won't find them. Copy referrers along with images, and check that every verifier in the path reads bundles.
- Private registries: Kyverno needs pull credentials to read referrers. Without them verification fails even though the kubelet, which has its own pull secret, could pull the image.
- A policy that matches only Pods lets an unsigned Deployment through, and its ReplicaSet then fails to create Pods. `kubectl describe rs` shows the denial in `FailedCreate` events.
- Kyverno's webhook only checks new requests. Pods that were running before the policy existed keep running until they are recreated; policy reports list them.

## The manifests you write

```yaml
apiVersion: policies.kyverno.io/v1
kind: ImageValidatingPolicy
metadata:
  name: require-signed-shop
spec:
  validationActions: [Deny]
  matchConstraints:
    resourceRules:
      - apiGroups: [""]
        apiVersions: ["v1"]
        operations: ["CREATE", "UPDATE"]
        resources: ["pods"]
  matchImageReferences:
    - glob: "ghcr.io/acme/*"
  attestors:
    - name: release
      cosign:
        keyless:
          identities:
            - issuer: https://token.actions.githubusercontent.com
              subject: https://github.com/acme/shop/.github/workflows/release.yaml@refs/heads/main
  validationConfigurations:
    mutateDigest: true
    verifyDigest: true
    required: true
  validations:
    - expression: >-
        images.containers.map(image, verifyImageSignatures(image, [attestors.release])).all(e, e > 0)
      message: shop images must be signed by the release workflow on main
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: shop
  namespace: shop
spec:
  replicas: 2
  selector:
    matchLabels:
      app: shop
  template:
    metadata:
      labels:
        app: shop
    spec:
      containers:
        - name: shop
          image: ghcr.io/acme/shop:4f2c8e1
```
