Paste your Nodes and your pods together and find out which pods can land on which nodes, which taint blocked the rest, and the exact toleration that would let them through. A toleration only ever permits: getting a pod onto a node is nodeSelector or nodeAffinity, and both gates are checked here.
Or drop a file anywhere on this panel. Nothing is uploaded: the analysis runs in this tab.
The answer appears here
Paste on the left and press Check placement. Nothing leaves this tab.
Nothing else to flag.
No formatting problems, and nothing the rules object to. Worth remembering what that covers: this reads the file you pasted, not the account or cluster it will be applied to.
No finding matches that filter.
Real input you can load into the tool above. Each one shows a different thing going wrong, because that is what the tool is for.
A tainted node and a pod that cannot land on it, which presents as Pending
apiVersion: v1
kind: Node
metadata:
name: gpu-1
spec:
taints:
- key: dedicated
value: gpu
effect: NoSchedule
---
apiVersion: v1
kind: Pod
metadata:
name: app
spec:
containers:
- name: app
image: app:1The same pair with a toleration that matches key, value and effect
apiVersion: v1
kind: Node
metadata:
name: gpu-1
spec:
taints:
- key: dedicated
value: gpu
effect: NoSchedule
---
apiVersion: v1
kind: Pod
metadata:
name: app
spec:
tolerations:
- key: dedicated
operator: Equal
value: gpu
effect: NoScheduleThese are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.
A toleration only permits a pod on a tainted node. It does not prefer that node.
Instead:Pair it with nodeAffinity or a nodeSelector to actually place the pod.
A toleration for NoSchedule does not cover NoExecute. An existing pod is evicted by the second.
Instead:Match the effect, or leave it empty to tolerate all.
key empty with operator Exists tolerates EVERY taint, including the unreachable ones that exist to evict.
Instead:Name the key unless you truly mean all.
Taints come from any object with kind: Node, tolerations from a Pod, Deployment, StatefulSet, DaemonSet, ReplicaSet, Job or CronJob. The matching follows the Kubernetes source: Toleration.ToleratesTaint for the taints, and the label selector matcher for nodeSelector and node affinity.
This is the misunderstanding the page exists for. A pod that tolerates the taint on your GPU nodes is allowed there, and it is equally allowed on every untainted node, so the scheduler puts it wherever its scoring lands and the dedicated nodes sit idle. When a pod tolerates a tainted node, has no nodeSelector and no nodeAffinity of either kind, and could equally land on an untainted node in the same input, that is reported with a nodeSelector suggested from the tainted node's own labels. DaemonSet pods are exempt, because the controller places them.
Tolerating every taint on a node is half the question. nodeSelector and required nodeAffinity are separate gates evaluated against the node's labels, and a pod that clears the taints and fails the labels is just as Pending. Each node is reported as SCHEDULES or BLOCKED with the reason, and when a pod has nowhere to go the report names the node it came closest on and says which of the two gates closed.
NoSchedule keeps new pods off and leaves running ones alone, which is why tainting a node does not drain it. PreferNoSchedule makes a node a last resort rather than a refusal, so it never blocks. NoExecute is the only effect that removes pods that are already running, so a pod pinned with spec.nodeName to a node whose NoExecute taint it does not tolerate is reported as evicted now, with no grace period, and the same pod against a NoSchedule taint is reported as staying put.
Kubernetes adds tolerations behind your back and they change the answer. Every pod gets not-ready and unreachable at NoExecute with tolerationSeconds: 300 from the DefaultTolerationSeconds plugin, which is the five minute wait before a pod moves off a dead node. A DaemonSet gets six more from its controller, plus network-unavailable when hostNetwork is set. spec.unschedulable: true is counted as the node.kubernetes.io/unschedulable taint that a cordon really adds. Each one is labelled in the report as added rather than written.
operator Exists with a value set. An empty key with the default Equal operator, when an empty key is only legal with Exists. tolerationSeconds on anything but NoExecute. A taint with no effect, which has no default and is not ignored. An operator other than Equal or Exists. Keys are checked as qualified names on both sides. These fail at apply time, and on a Deployment the failure surfaces as an event on the ReplicaSet rather than anywhere obvious.
When a pod cannot schedule anywhere, the fix carries the exact YAML for the closest node's blocking taints, with operator Exists for a valueless taint and Equal with the value for the rest. When the label constraints are what failed instead, the fix says so and says a toleration will not help. A blanket toleration and a control plane toleration are both called out, including the case where only one of the two spellings of the control plane taint is tolerated.
What a green verdict does not say: this answers taints and label constraints against the nodes you pasted, and nothing else. A pod can still stay Pending because the node has no CPU or memory left, because a pod affinity or anti-affinity rule fails, or because a topology spread constraint is unsatisfiable, and none of that is read here. Preferred node affinity is not scored, since it cannot make a node ineligible. A matchFields key other than metadata.name is left undecided rather than guessed at, and a toleration that no pasted node needs is reported as unused at low severity, because the nodes in a paste are rarely the whole cluster.
A taint is a refusal that lives on the node. A toleration is a waiver that lives on the pod. Together they answer one question: is this pod allowed here. They never answer the other question, which is where should this pod go, and the gap between the two is where most confusion about node dedication comes from. Adding a toleration to a pod does not move it anywhere. It only stops one particular node from turning it away.
A taint is a key, an optional value and one of three effects. The effect is required and has no default. A toleration is a key, an operator, a value, an effect and optionally tolerationSeconds, and almost every field is optional in a way that widens what it covers rather than narrowing it.
node:
spec:
taints:
- key: dedicated
value: gpu
effect: NoSchedule
pod:
spec:
tolerations:
- key: dedicated
operator: Equal
value: gpu
effect: NoScheduleThis is the whole point. A taint keeps other pods off a node. It does nothing to bring your pod on. The pod that should run on those nodes needs a toleration so it is allowed, and a nodeSelector or nodeAffinity so it is sent. With only the toleration the pod is allowed on the tainted nodes and on all the ordinary ones, and the ordinary ones usually win because there are more of them.
taint on the node keeps everything else off
toleration on the pod lets this pod through
nodeSelector on the pod sends this pod there
the dedicated node pattern needs all three:
spec:
tolerations:
- key: dedicated
operator: Equal
value: gpu
effect: NoSchedule
nodeSelector:
node-pool: gpuThe distinction that matters is placement against eviction. NoSchedule and PreferNoSchedule are consulted when the scheduler is choosing a node and never again, so a pod already running on the node stays running. NoExecute is checked continuously, so applying it removes the pods that do not tolerate it. This is why cordoning a node, which adds a NoSchedule taint, does not empty it, and why drain is a separate command that deletes the pods itself.
NoSchedule no new pods, existing pods stay
PreferNoSchedule last resort, never a refusal
NoExecute no new pods, and existing pods
without a toleration are evicted
kubectl cordon -> NoSchedule taint, node keeps
running what it has
kubectl drain -> cordon, then evict the podsThree of them, and each one widens the toleration. An empty effect matches all three effects for that key, not none of them. An empty key with operator Exists matches every taint that exists or ever will. An absent operator is Equal rather than Exists, so a toleration written with just a key and a value still compares the value, and a toleration with just a key and no value only tolerates a taint whose value is also empty.
key: dedicated
operator: Exists
any value, any effect,
for that one key
operator: Exists (no key at all)
every taint on every node
key: dedicated
value: gpu operator defaults to Equal,
so the value must be "gpu"
key: dedicated value defaults to "",
so this matches only a taint
with no valuetolerationSeconds only exists for NoExecute, because it counts down an eviction, and the API server rejects it on any other effect. Omitting it means tolerate forever. Setting it to zero means evict immediately, which is not the same thing as omitting it. The reason it matters on pods that never mention it: the DefaultTolerationSeconds admission plugin adds two tolerations with a 300 second timer to every pod that does not already cover those keys.
added to your pod on admission:
- key: node.kubernetes.io/not-ready
operator: Exists
effect: NoExecute
tolerationSeconds: 300
- key: node.kubernetes.io/unreachable
operator: Exists
effect: NoExecute
tolerationSeconds: 300
a node stops answering at t=0
the pod is evicted at t=300
nothing is stuck, that is the default holding itMost taints in a real cluster were not written by anyone. The node lifecycle controller and the kubelet add them from node conditions, a cordon adds one, and the control plane carries one from installation. A pod that cannot schedule anywhere because of one of these is a different problem from one blocked by a taint you chose, so it helps to know which is which.
node.kubernetes.io/not-ready
node.kubernetes.io/unreachable
node.kubernetes.io/memory-pressure
node.kubernetes.io/disk-pressure
node.kubernetes.io/pid-pressure
node.kubernetes.io/unschedulable cordon
node.kubernetes.io/network-unavailable
node-role.kubernetes.io/control-planeIt was node-role.kubernetes.io/master before 1.24 and node-role.kubernetes.io/control-plane after it. A cluster built before that release and upgraded through it can have nodes carrying either one, so a toleration for a single spelling silently stops applying on half the nodes and the symptom is a pod that schedules on some control plane nodes and not others. Tolerate both until every node is known to carry the new key.
- key: node-role.kubernetes.io/control-plane
operator: Exists
effect: NoSchedule
- key: node-role.kubernetes.io/master
operator: Exists
effect: NoScheduleAn empty key with operator Exists and no effect is three widenings at once and it covers every taint that will ever be invented. It is the normal way to run a node agent as a DaemonSet, and it is a placement policy with no boundary anywhere else. Two consequences are easy to miss: it puts the pod on the control plane nodes, and it cancels the 300 second not-ready toleration, so the pod is never evicted from a node that has stopped answering.
tolerations:
- operator: "Exists"
covers control-plane, not-ready, unreachable,
every pressure taint, a cordon, and anything
a future version adds