Kubernetes Rollout & Probe Timing

Work out what your probe, rollout and shutdown settings actually do: how long a container has to start before it is killed, how much capacity a deploy removes, and whether a node drain can complete at all.

Paste below, or drop a file anywhere on this panel

Or drop a file anywhere on this panel. Nothing is uploaded: the analysis runs in this tab.

The answer appears here

Paste on the left and press Work it out. Nothing leaves this tab.

Examples

Real input you can load into the tool above. Each one shows a different thing going wrong, because that is what the tool is for.

A slow rollout

maxUnavailable zero with a long readiness delay, which makes twenty replicas take a while

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 20
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    spec:
      containers:
        - name: app
          readinessProbe:
            initialDelaySeconds: 30
            periodSeconds: 10

A faster strategy

The same rollout with surge raised, and what it costs in peak capacity

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 20
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
  template:
    spec:
      containers:
        - name: app

Common mistakes

These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.

  1. Setting maxUnavailable to zero without surge

    With both at zero nothing can move and the rollout stalls. With surge one it proceeds one pod at a time.

    Instead:Allow either surge or unavailability, and expect the rollout to take replicas times readiness time.

  2. Forgetting the preStop hook

    Without one, a pod stops accepting traffic and is killed before endpoints propagate, dropping in-flight requests.

    Instead:Add a preStop sleep of a few seconds.

  3. Ignoring initialDelaySeconds in the estimate

    Every replica waits it out. Thirty seconds times twenty replicas at maxSurge one is ten minutes.

    Instead:Lower the delay and use a startup probe for slow-starting apps.

What it works out

None of these settings produce an error when they are wrong. They produce a restart loop, a capacity dip, or an upgrade that never finishes.

How long the container has to start

initialDelaySeconds plus failureThreshold times periodSeconds is the budget before the liveness probe kills it. An app needing 45 seconds under a probe that allows 20 restarts forever, and the logs show a clean startup every single time, which is what makes it so hard to diagnose.

maxSurge rounds up, maxUnavailable rounds down

That asymmetry is Kubernetes' own and it catches people. 25% of 3 replicas is 0.75, which becomes 1 extra pod of surge and 0 unavailable. So the setting you thought limited disruption often does nothing at all until the replica count grows.

PodDisruptionBudgets that block a drain forever

minAvailable equal to the replica count means no pod may ever be evicted. kubectl drain does not fail, it retries indefinitely, so a cluster upgrade or an autoscaler scale-down hangs with nothing explaining why. The arithmetic is resolved against the actual replica count, including percentages.

The requests dropped on every deploy

Endpoint removal and SIGTERM happen in parallel, not in order, so traffic keeps arriving at a pod that has already started shutting down until every kube-proxy notices. Without a preStop sleep those requests fail, on every rollout, and it looks like random flakiness.

The three clocks in a pod lifecycle

Startup, rollout and shutdown each have their own timers, spread across fields that do not sit near each other in the manifest. None of them produce an error when set wrongly. They produce restart loops, capacity dips and dropped connections instead.

Startup: the probe that should be doing the waiting

A liveness probe answers whether a running container is still healthy. It is not for waiting on a slow boot, but with no startupProbe present its clock starts when the container does, so it ends up doing that job badly. The startupProbe exists precisely to separate the two: give it a generous failureThreshold, and the liveness probe does not begin until startup has succeeded, so you can keep its threshold tight without punishing a slow start.

startupProbe:
  failureThreshold: 30
  periodSeconds: 5      allows 150s to boot

livenessProbe:
  failureThreshold: 3
  periodSeconds: 5      then kills after 15s of failures

without the startupProbe, that liveness probe kills at 15s
whatever the application is doing

Rollout: two numbers, rounded in opposite directions

maxSurge is how many extra pods may exist above the replica count, and it rounds up. maxUnavailable is how many may be missing, and it rounds down. Both default to 25%. The rounding is what makes small replica counts behave unexpectedly, and it is why maxSurge 1 with maxUnavailable 0 is the usual answer for anything that matters: it costs one extra pod during the rollout and loses no capacity at all.

3 replicas, both at 25%

  maxSurge        0.75 -> rounds UP   -> 1
  maxUnavailable  0.75 -> rounds DOWN -> 0

  between 3 and 4 pods exist during the rollout

Disruption budgets protect drains, not rollouts

A rollout is already governed by maxUnavailable. A PodDisruptionBudget governs voluntary eviction, which is node drains, cluster upgrades and autoscaler scale-downs. They are separate mechanisms and a workload wants both. The failure mode is specific: set minAvailable to the replica count and the eviction API refuses every request, drain retries in a loop, and the upgrade never finishes.

Shutdown is not ordered, which is the whole problem

When a pod is deleted, Kubernetes sends SIGTERM and removes the pod from the Service endpoints at the same time. Endpoint removal then has to propagate to every node's kube-proxy, which takes a moment. During that moment traffic is still being sent to a pod that has begun shutting down. A preStop hook that sleeps for a few seconds fixes it, because the container keeps serving normally while the sleep runs and the endpoint removal catches up.

lifecycle:
  preStop:
    exec:
      command: ["sleep", "5"]

terminationGracePeriodSeconds: 45   covers the sleep AND the drain