k8wiz
Back to Articles

Ready to test your cert-manager knowledge? Try our cert-manager quiz!

How to troubleshoot a cert-manager Certificate that never becomes ready

Follow a stuck cert-manager Certificate through its CertificateRequest, Issuer, Order and Challenge to find why it isn't issued, and fix the common causes.

8 min read
By k8wiz Team
  • cert-manager
  • troubleshooting
  • acme
  • lets-encrypt

You create a Certificate, wait a minute, and kubectl get certificate still shows READY False. Ten minutes later it's the same. cert-manager rarely fails loudly. It records what went wrong on a chain of resources it created for you, and the reason sits at whichever link stopped moving.

This guide walks that chain from the top. Each step tells you what to look at and what the common messages mean. Most of it follows the cert-manager troubleshooting docs, with the ACME parts written for Let's Encrypt, where most stuck certificates come from.

The chain of resources

Every issuance starts with a Certificate. cert-manager turns it into a CertificateRequest, which carries the CSR to an Issuer or ClusterIssuer. For an ACME issuer such as Let's Encrypt, the request creates an Order, and the Order creates one Challenge per domain name. Other issuer types, such as CA or Vault, stop at the CertificateRequest.

cert-manager records progress as conditions and events on each of these objects. Its docs say kubectl describe is your best tool here, and that the controller logs are verbose and best left for when the resources don't explain the problem. Work down the chain until you find the object with an error.

Step 1: look at the Certificate

Start with the Certificate itself.

kubectl get certificate -n shop
kubectl describe certificate shop-tls -n shop

The Status section holds the Ready condition, and the events at the bottom tell the story so far. For a new certificate you typically see that cert-manager is issuing because the Secret doesn't exist, that it stored a new private key in a temporary Secret, and that it created a CertificateRequest. The last event names that CertificateRequest, which is your next stop.

If there's no Certificate at all and you expected one from an Ingress annotation, the problem is earlier. Check that the Ingress has the cert-manager.io/cluster-issuer or cert-manager.io/issuer annotation and a tls section that lists the hosts and a secretName.

Step 2: use cmctl to see everything at once

Describing five objects one by one gets tedious. The cmctl CLI has a command that collects the Certificate, its CertificateRequest, the Secret, the Issuer and, for ACME, the Order and its Challenges, with their conditions and events.

brew install cmctl
cmctl status certificate shop-tls -n shop

Without Homebrew, install it with go install github.com/cert-manager/cmctl/v2@latest or download a release binary. The output is long, but it usually points straight at the broken link, and the steps below explain what to do there.

Step 3: check the CertificateRequest

kubectl get certificaterequest -n shop
kubectl describe certificaterequest shop-tls-1 -n shop

A CertificateRequest has to be approved before an issuer signs it. cert-manager's built-in approver handles that automatically, so look for an Approved condition. If it's missing and you've turned off the default approver, for example to use approver-policy, then your approval policy is the thing rejecting or ignoring the request.

For ACME issuers, a healthy request shows Ready False with reason Pending and a message saying it waits on an Order. That's normal while the Order runs. An error here usually comes from the issuer, such as an issuer that doesn't exist or isn't ready. Check the issuerRef for typos and remember that kind: Issuer only looks in the Certificate's own namespace.

Step 4: check the issuer

kubectl get clusterissuer
kubectl describe clusterissuer letsencrypt-prod

An issuer that shows READY False can't sign anything. For ACME issuers the problems are mostly about the account. The troubleshooting docs list a few common messages. An invalidEmail error means Let's Encrypt rejected the contact address, which happens with placeholder domains such as example.com. A message saying the account key Secret already exists means an old account key is left over from a previous issuer; delete that Secret so cert-manager can register again.

While you're testing, point the issuer at the Let's Encrypt staging server, https://acme-staging-v02.api.letsencrypt.org/directory. Its certificates aren't trusted by browsers, but it lets you retry as often as you need without running into the production rate limits.

Step 5: check the Order

kubectl get order -n shop
kubectl describe order shop-tls-1-2745722290 -n shop

The Order's State is pending while challenges run, valid when the certificate is ready to download and invalid when validation failed. The events list one Challenge per domain. If the state is invalid, the ACME server gave up on at least one authorization, and the Challenge tells you why.

You can also ask the ACME server directly. The Order's status lists one authorization URL per domain, and opening it shows the server's view of the validation, including the error it saw.

kubectl get order shop-tls-1-2745722290 -n shop -o jsonpath='{.status.authorizations[*].url}'

Step 6: check the Challenges

This is where most stuck certificates end up.

kubectl get challenges -n shop
kubectl describe challenge shop-tls-1-2745722290-0 -n shop

The REASON column is the most useful field in the whole chain. Before cert-manager asks the ACME server to validate, it runs a self-check of its own. For HTTP-01 it fetches the challenge URL; for DNS-01 it looks for the TXT record. A reason that starts with Waiting for http-01 challenge propagation or Waiting for dns-01 challenge propagation means the self-check keeps failing.

cert-manager retries a failing self-check every 10 seconds, with no limit. A Challenge can sit in pending for days without anything new in the events, which is why it's easy to miss.

HTTP-01 failures

For HTTP-01, the self-check fetches http://<domain>/.well-known/acme-challenge/<token>. Test that URL from outside the cluster first, then from a pod inside it.

curl -sv http://shop.example.com/.well-known/acme-challenge/test
kubectl run acme-probe -n shop --image=curlimages/curl --restart=Never --rm -i -- \
  curl -sv http://shop.example.com/.well-known/acme-challenge/test

With a made-up token you expect a 404, so this only proves the host answers on plain HTTP. A connection error or timeout points at DNS, a firewall or the load balancer. To check that requests really reach the solver, use the real token from the Challenge's spec.token, which should return the key authorization instead of a 404.

If the self-check gets a 404, check that the solver pod cert-manager created is running and describe the solver Ingress it made. If it fails with tls: handshake failure, the ingress controller is probably forcing HTTPS on a host that has no certificate yet. The docs suggest adding cert-manager.io/issue-temporary-certificate: "true" to the Ingress or Certificate, so cert-manager first issues a temporary self-signed certificate. Some ingress controllers also handle a second Ingress for the same host badly. For those, acme.cert-manager.io/http01-edit-in-place: "true" makes cert-manager add the challenge path to your existing Ingress instead.

DNS-01 failures

For DNS-01, check that the _acme-challenge TXT record exists in your provider's console and from the internet. If the Challenge events show an error from the DNS provider, fix the credentials or permissions first; that's the most common cause.

dig +short TXT _acme-challenge.shop.example.com @1.1.1.1

If the record is visible publicly but the self-check never sees it, the cluster's DNS probably answers differently, which some cloud providers do. Tell cert-manager to use public resolvers for the self-check. With the Helm chart, pass the controller flags through extraArgs.

helm upgrade cert-manager jetstack/cert-manager -n cert-manager --reuse-values \
  --set 'extraArgs={--dns01-recursive-nameservers-only,--dns01-recursive-nameservers=8.8.8.8:53\,1.1.1.1:53}'

The same setting fixes a less obvious failure, where cert-manager picks the wrong DNS zone. It finds the zone by looking up SOA records, and some resolvers, such as dnsmasq with its filterwin2k option, drop those.

When the self-check can't see what the ACME server sees

Sometimes the challenge is reachable from the internet but not from inside the cluster. Common causes are a network without NAT loopback, split-horizon DNS, or an ingress setup that splits public and private traffic. The self-check then fails forever, even though Let's Encrypt would succeed.

cert-manager 1.21 added an escape hatch for this. Set waitInsteadOfSelfCheck on the solver, and cert-manager skips its self-check, waits that long after presenting the challenge, then asks the ACME server to validate.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    email: platform@shop.example.com
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-prod-account-key
    solvers:
      - http01:
          ingress:
            ingressClassName: nginx
        waitInsteadOfSelfCheck: 60s

Pick the delay with care. Too short, and the ACME server validates before the solver is reachable and the authorization fails. Too long, and every issuance just waits longer. Use it only after you've confirmed the self-check is wrong and the outside path works.

Step 7: retry after the fix

Once you've fixed the cause, you don't have to wait for cert-manager's own retry schedule. Trigger a new issuance with cmctl.

cmctl renew shop-tls -n shop

For a stuck ACME Order, the cert-manager docs describe another way to retry, which is to delete the Order. cert-manager then creates a new Order with new Challenges.

kubectl delete order shop-tls-1-2745722290 -n shop

Watch the chain again with cmctl status certificate. When the Certificate shows READY True, the signed certificate and key are in the Secret named in secretName.

Test yourself

Ready to test your cert-manager knowledge? Try our cert-manager quiz!