Kubernetes cert-manager Certificates
Build a cert-manager Certificate together with the Issuer or ClusterIssuer it needs, because a Certificate pointing at an issuer that does not resolve is the usual failure. It checks the namespace rules, tells you when a wildcard cannot be issued by the solver you picked, and explains what to read when nothing happens.
certificate.yaml
updates as you typeCommon mistakes
These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.
Using an Issuer where a ClusterIssuer was meant
An Issuer is namespaced and only serves Certificates in its own namespace. A Certificate referencing one from another namespace stays pending with no obvious error.
Instead:ClusterIssuer for anything shared, and check the kind in issuerRef.
Testing against the Let's Encrypt production endpoint
The rate limit is fifty certificates per registered domain per week, and hitting it locks you out for a week with no appeal.
Instead:Use the staging endpoint until the flow works end to end.
Forgetting the secret is created, not read
spec.secretName names a secret cert-manager will CREATE. Pointing it at an existing secret gets it overwritten.
Instead:Use a fresh name, and let cert-manager own it.
Why a cert-manager Certificate sits at Ready: False
The Certificate object is short and almost nothing that goes wrong with it is visible in the object. Every common failure is about something outside it: which namespace the issuer is in, whether the CA can reach port 80, whether the Secret landed somewhere the Ingress can read. The object applies cleanly in every one of those cases, which is why the usual symptom is silence.
issuerRef has no namespace field
This is the mistake. An Issuer is namespaced, and a Certificate referencing kind: Issuer always resolves it in the Certificate's own namespace. There is no field to point anywhere else. So a Certificate copied into a second namespace looks correct, applies without complaint, and waits forever for an Issuer that is not there. A ClusterIssuer is cluster-scoped and resolves from any namespace, which is why most setups use one.
# namespace: staging
issuerRef:
name: letsencrypt-prod
kind: Issuer <- looked up in staging
and nowhere else
# there is no issuerRef.namespace. It was never
# added, on purpose: it would let any namespace
# borrow another namespace's ACME account.
kubectl describe certificate api-tls
Message: issuer.cert-manager.io "letsencrypt-prod"
not foundThe Certificate, the Secret and the Ingress are one namespace
cert-manager writes spec.secretName into the Certificate's namespace, and an Ingress can only reference a Secret in its own namespace. All three have to agree. When they do not, the Ingress serves whatever fallback certificate the controller has, so the browser error names the ingress controller and gives no hint that a certificate two namespaces away is perfectly healthy.
Certificate ns: apps secretName: api-tls
Secret ns: apps (written here)
Ingress ns: web tls.secretName: api-tls
^ reads ns: web, finds nothing
# ingress-nginx then serves:
# CN=Kubernetes Ingress Controller Fake Certificate
# and the Certificate still reads Ready: True.
# Two namespaces needing the same certificate means
# issuing it twice, or replicating the Secret with a
# tool built for that.HTTP01 needs public port 80, DNS01 is the only route to a wildcard
An HTTP01 challenge makes cert-manager create a temporary pod, Service and Ingress, and the CA fetches a token over plain HTTP from the internet. An internal cluster, a load balancer listening only on 443, or a WAF in front of the challenge path all fail identically. A wildcard has no single host to serve a token from, so HTTP01 cannot issue one at all. DNS01 writes a TXT record instead, which is why it works for wildcards and for clusters with no inbound path.
HTTP01
GET http://example.com/.well-known/acme-challenge/<token>
needs: public DNS, port 80 open, no WAF on the path
cannot: *.example.com, internal-only clusters
DNS01
TXT _acme-challenge.example.com = <token>
needs: an API credential for the zone, in a Secret
can: *.example.com
# *.example.com and example.com validate against the
# same _acme-challenge.example.com record, so asking
# for both is two TXT values on one name.The production rate limits cost a week, which is what staging is for
Let's Encrypt production allows 50 certificates per registered domain per week, and 5 duplicate certificates, meaning the same exact set of names, per week. Both are rolling windows with no appeal and no manual reset. A reconcile loop, or a Certificate deleted and recreated a few times while debugging a solver, burns the duplicate limit in an afternoon and then the fix has to wait. The staging endpoint exists precisely so the plumbing gets debugged somewhere the limits do not matter.
staging https://acme-staging-v02.api.letsencrypt.org/directory
production https://acme-v02.api.letsencrypt.org/directory
50 certificates / registered domain / week
5 duplicate certificates / week
# A staging certificate is signed by a root no browser
# trusts. The warning is the expected result, not a
# failure, and it proves the whole path worked.
# The two environments have separate accounts, so
# switching also means a new account key Secret.Renewal at day 60, and a private key that may never change
Left unset, renewBefore puts renewal two thirds of the way through the lifetime, so a 90 day certificate renews at about day 60 with 30 days of margin. The margin is the point: it is how long you have to notice that a DNS credential expired. Separately, privateKey.rotationPolicy decided for years to reuse the same key on every renewal, so one key could sit in a Secret across a dozen certificates. cert-manager 1.18 changed the default, which means the behaviour now depends on the installed version and the only way to know is to set it.
duration 2160h (90 days, and Let's Encrypt
ignores what you ask for)
renewBefore unset -> 1/3 of 2160h = 720h
-> renewal at day 60
privateKey:
rotationPolicy: Never # reuse the existing key
rotationPolicy: Always # new key every renewal
# Never was the default until 1.18. It is also the
# answer nobody wants to give when a review asks how
# often TLS keys are rotated.The annotation and the object both create certificates
The ingress-shim turns a cert-manager.io/cluster-issuer annotation on an Ingress into a Certificate object of its own. That is convenient, and it is why a hand-written Certificate plus the annotation gives you two objects issuing for the same names: they take turns writing the Secret, both spend the duplicate certificate limit, and removing one appears to fix things until the other renews. Pick one route. The annotation is less work; the object is what lets you set the key algorithm, the rotation policy and the renewal window.
# Route A: the shim writes the Certificate for you
kind: Ingress
metadata:
annotations:
cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
tls:
- hosts: [example.com]
secretName: example-com-tls
# Route B: a Certificate object you control
# Both at once: two Certificates, one Secret.When it does not issue, follow the chain
The status is spread across four objects and each one only names the next. The Certificate holds a condition and the name of a CertificateRequest, the request holds an Order, the Order lists Challenges, and the Challenge is where the real reason lives: a 404 from the token URL, or a TXT record the CA could not see. Stopping at the Certificate is why this so often reads as no error at all.
kubectl describe certificate example-com-tls
kubectl describe certificaterequest example-com-tls-1
kubectl describe order example-com-tls-1-2748
kubectl describe challenge example-com-tls-1-2748-...
# the last one says things like:
# Waiting for HTTP-01 challenge propagation:
# wrong status code '404', expected '200'
# Waiting for DNS-01 challenge propagation:
# NS record not found for domain
# Also: the Secret existing does not mean it issued.
# cert-manager can write a temporary self-signed
# certificate so the Ingress has something to serve.