Kubernetes Node Capacity Planner
Work out what a node can actually schedule. Give it the advertised size, the reservation formula your provider uses and the pod you want to pack onto it, and it subtracts kube-reserved, system-reserved, the eviction threshold and your DaemonSets, then bin packs what is left and shows the stranded remainder. It is arithmetic on the numbers you type, not a measurement of a cluster.
node-capacity.txt
updates as you typeCommon mistakes
These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.
Planning against capacity rather than allocatable
The kubelet reserves memory and CPU for the system and for itself, plus an eviction threshold. Allocatable is meaningfully smaller and is what the scheduler uses.
Instead:Plan against allocatable.
Forgetting the eviction threshold
The node begins evicting before memory is exhausted, so usable memory is lower again than allocatable suggests.
Instead:Leave headroom above the threshold.
Assuming reservations scale with instance size
They are configured, not derived, and a large node with default reservations wastes proportionally less but still wastes.
Instead:Check the actual kube-reserved and system-reserved values on your nodes.
Why a node hands the scheduler less than it advertises
The size on the instance type page is a billing figure. The number the scheduler places pods against is allocatable, which is that figure less what the kubelet holds back for itself, less what it holds back for the operating system, less the hard eviction threshold. Then the DaemonSets take their share on every node before your workload gets any of it. Every step is published arithmetic, and it is invisible until you put the instance type and kubectl describe node side by side.
Capacity is what you pay for, allocatable is what can be scheduled
kubectl describe node prints both blocks, one under the other, and they are different numbers. Allocatable is capacity less kube-reserved, less system-reserved, less the hard eviction threshold. Nothing in that subtraction is visible on the instance type page, so a sizing exercise done from the instance list is wrong before it starts. The share grows as the node shrinks: on the regressive schedule GKE publishes, the first 4Gi of any node is reserved at 25%, so a 4Gi node hands over a quarter of its memory before the eviction threshold comes off as well.
16Gi node, EKS, --max-pods=110
capacity 16384Mi
kube-reserved -1465Mi 255 + 11 x 110
hard eviction -100Mi memory.available<100Mi
--------
allocatable 14819Mi what the scheduler sees
kubectl describe node prints Capacity and
Allocatable as two separate blocks. The instance
type page prints only the first one.The reservation formula is per provider, and on EKS it moves with --max-pods
EKS reserves memory per pod slot rather than per gibibyte: the AMI bootstrap script uses 255MiB plus 11MiB for every pod in --max-pods. That couples the two questions, so lowering the pod ceiling gives memory back to your workload and raising it with prefix delegation takes memory away. GKE reserves on a regressive schedule and AKS moved to one, which means small nodes lose a much larger share than big ones. A self-managed kubelet with none of the flags set reserves nothing at all, which is how a node ends up with sshd competing with pods for memory. CPU is the same regressive schedule on all three and is rarely what the reservation costs you.
EKS 255MiB + 11MiB per pod slot
110 pods -> 1465Mi
58 pods -> 893Mi lower ceiling,
more for pods
250 pods -> 3005Mi prefix delegation,
paid for in memory
GKE, AKS 25% of the first 4Gi
20% of the next 4Gi
10% of the next 8Gi
6% of the next 112Gi
2% of everything above 128Gi
CPU 6% of core 1, 1% of core 2, 0.5% of
cores 3 and 4, 0.25% above four
4 vCPU -> 80m
The published tables are written in decimal units
while a kubelet measures in binary, so a boundary
value can land a percent or two either side.The eviction threshold is capacity that can never hold a pod
The hard eviction threshold is subtracted from allocatable, not held in reserve behind it. The kubelet default is memory.available<100Mi, so 100Mi of every node is paid for and unschedulable by design. Raising it to protect the node against a spike costs allocatable one for one. The other default, nodefs.available<10%, is about disk: it does not appear in the memory arithmetic at all, and it is what evicts pods on a node whose image cache has filled up.
--eviction-hard=memory.available<100Mi default
--eviction-hard=memory.available<500Mi -400Mi
of
allocatable
nodefs.available<10% disk, not memory. Evicts
pods on a node with a full
image cache, and does not
change the numbers above.DaemonSets come off every node, and they take pod slots too
A DaemonSet is subtracted once per node, not once per cluster, and it is scheduled before your workload and tolerates most taints, so the capacity is gone whether or not anything else lands there. The usual set is a CNI agent, kube-proxy, a log shipper, a node exporter and a CSI node plugin. Sum their requests rather than their usage, because requests are what the scheduler subtracts. Leaving them out is the single most common reason a sizing exercise comes out one node short.
per node 300m 500Mi 6 pods
3 nodes 900m 1500Mi 18 of the 330 pod slots,
gone before your workload
gets any of it
They are also the reason a node that looks empty
in kubectl top has no room left in
kubectl describe node.Pods per node is the lowest of three ceilings
The answer is the smallest of three floor divisions: what fits by CPU, what fits by memory, and what is left of --max-pods after the DaemonSet pods. A request of zero does not constrain anything, so a BestEffort pod is limited only by the slot count. Which of the three binds is the useful part, because it names the lever: CPU or memory means a different instance shape, and pod slots means the node is too large for the pods on it.
allocatable less DaemonSets 3620m 14319Mi
one pod requests 500m 1024Mi
by CPU floor(3620 / 500) = 7
by memory floor(14319 / 1024) = 13
by pod slots 110 - 6 = 104
---
pods per node 7
bound by CPU
stranded 120m and 7151Mi: schedulable,
billed, and too small to hold
another pod of this shapeThe scheduler packs by requests, and this packing is an upper bound
The scheduler never looks at what a container consumes. A cluster can report every node as full while the metrics show 20% CPU, and both are true at once, because the requests are the reservation and the reservation is what gets subtracted. Everything above is also an upper bound on placement: pod anti-affinity, topology spread constraints, taints, node affinity and PVC zone binding all reduce the set of nodes a given pod may actually land on. A cluster that fits on paper and not in practice is usually a spread constraint rather than a capacity problem. And none of this is a measurement: it is arithmetic on numbers typed into a form, so the node itself is the only authority on the node.
allocatable 3920m 14819Mi
kubectl describe node ip-10-0-1-23
Allocated resources:
Resource Requests Limits
cpu 3800m (96%) 6 (153%)
memory 14336Mi (96%) 20Gi
kubectl top node ip-10-0-1-23
CPU 21% MEMORY 34%
Both are correct at the same time. Only the first
one decides whether the next pod is scheduled.