Kubernetes Probe Generator
Build startup, readiness and liveness probes on a Deployment that applies as it stands, and find out how many seconds your container actually gets to boot before something restarts it. Tell it how long your app takes to come up and it draws the two against each other.
probes.yaml
updates as you typeCommon mistakes
These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.
Using a liveness probe with a short initial delay
The container is killed before it finishes starting, which produces a restart loop that looks like a crash.
Instead:Use a startup probe for slow starters, and let liveness begin after it passes.
Pointing liveness at a dependency
A liveness probe that checks the database restarts the application when the database is slow, which adds load and helps nothing.
Instead:Liveness checks whether THIS process is wedged. Dependencies belong in readiness.
Setting the same endpoint for readiness and liveness
They answer different questions, and a shared endpoint means a temporary dependency failure kills the container instead of removing it from the load balancer.
Instead:Separate them.
What the three probes actually do
They look interchangeable in the YAML and they are not. A startupProbe holds the other two off while the container boots. A readinessProbe decides whether the pod is in the Service endpoints. A livenessProbe restarts the container. None of them is validated against the others at apply time, so the wrong one in the wrong place applies cleanly and shows up later as a restart loop or an outage that spreads.
Liveness restarts, readiness only stops traffic
This is the mistake with the largest blast radius. A liveness endpoint that checks a database means that when the database has a blip, every replica fails the probe within the same few seconds and Kubernetes restarts all of them at once. The replicas that were still serving cached reads are now gone too, and the app cannot come back until the dependency does. Readiness is the probe that is allowed to know about dependencies, because failing it removes one pod from rotation and leaves the process alive.
livenessProbe -> SIGTERM, then a restart
readinessProbe -> removed from the Service endpoints
startupProbe -> holds the other two off, then restarts
# The endpoint each one should call:
/healthz liveness "my event loop is not deadlocked"
no database, no cache, no downstream call
/readyz readiness "I can serve a request right now"
connection pool filled, migrations doneThe startup budget is failureThreshold times periodSeconds
That product is the whole answer to how long a container gets to boot, and it is the number nobody computes. Setting initialDelaySeconds on the liveness probe instead is the common workaround and a worse one: it is dead time in which the probe learns nothing, it is paid in full on every restart even when the app was ready in two seconds, and once it expires the app still only gets failureThreshold periods. A startupProbe stops at the first success, so a generous threshold costs nothing on a fast boot.
startupProbe:
periodSeconds: 5
failureThreshold: 30 -> 150s to boot
# Same intent, written the wrong way:
livenessProbe:
initialDelaySeconds: 150 -> 150s of not looking,
periodSeconds: 10 then 30s of patience
failureThreshold: 3 and 150s lost on every
restart of a crash loopRemoving a pod from a Service is not instant
Failing a readiness probe updates the EndpointSlice, and that change then has to reach kube-proxy, or the equivalent dataplane, on every node in the cluster before those nodes stop forwarding to the pod. Until they do, requests keep arriving at a pod that Kubernetes already considers unready. The same gap exists on shutdown, in the other direction, which is why a preStop sleep keeps the container serving for a few seconds after it has been told to go.
t+0s readiness fails for the third time
t+0s kubelet marks the pod NotReady
t+0s endpoints controller updates the EndpointSlice
t+?s every node's kube-proxy observes the change
^ this is the window where traffic still lands
lifecycle:
preStop:
exec:
command: ["sleep", "5"] # cover the windowThe defaults are tighter than they read
timeoutSeconds is 1 second unless you say otherwise, and a timeout is a failure exactly like a 500 is. One second is generous for a handler that returns a constant and short for a JVM in a GC pause, or for any container against a CPU limit it is being throttled at. successThreshold must be 1 on the liveness and startup probes, and the API server rejects anything else, so only readiness may raise it.
initialDelaySeconds: 0
periodSeconds: 10
timeoutSeconds: 1 <- the one that bites
failureThreshold: 3
successThreshold: 1 <- must be 1 for
liveness and startup
# probe-level grace period, Kubernetes 1.25+:
livenessProbe:
terminationGracePeriodSeconds: 10
# kill a hung process fast without shortening
# the grace period for normal shutdownsWhere the probe connects from, and what counts as healthy
An httpGet probe is made by the kubelet on the node, to the pod IP. It is not localhost and it does not go through the Service, so an app bound to 127.0.0.1 fails every probe while working perfectly from inside the container. Any 2xx or 3xx is a success, so an endpoint that redirects to a login page passes. An exec probe is the opposite: it runs inside the container, where 127.0.0.1 is the app, and it forks a process on every period in every replica.
httpGet kubelet -> 10.42.3.17:8080/healthz
pod IP, not the Service, not localhost
200-399 = healthy, including a 302
tcpSocket connect, then close
proves a listener is bound, nothing more
most frameworks bind before they can serve
exec forks in the container every period
period 5s x 3 probes x 20 replicas
= 720 processes a minute