Kubernetes ResourceQuota Generator
Build a ResourceQuota with the LimitRange that keeps it from rejecting your next deploy. A quota on requests or limits makes those fields mandatory for every pod in the namespace, and the LimitRange is what fills them in.
resourcequota.yaml
updates as you typeCommon mistakes
These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.
Adding a quota with no LimitRange
Once a quota on requests or limits exists, every pod in the namespace MUST specify them. Existing deployments without them stop scheduling immediately.
Instead:Add a LimitRange with defaults at the same time. This is the change that breaks deploys.
Setting a quota on pods without one on resources
Pod count says nothing about consumption, so one namespace can still exhaust the cluster with a handful of huge pods.
Instead:Quota the resources, not the object count.
Forgetting the quota counts terminating pods
Pods still terminating hold their quota, so a rollout can fail on quota that appears to have room.
Instead:Leave headroom for the rollout's surge.
What a ResourceQuota does, and why it needs a LimitRange next to it
A ResourceQuota caps what one namespace may consume in total: CPU, memory, storage, and how many of each kind of object exist. A LimitRange works on one container at a time, supplying the values a pod did not set and rejecting the ones that are out of range. They are separate objects with separate jobs, and applying the first without the second is the single most common way a quota takes a team's deploys down.
A quota makes requests and limits mandatory
The moment requests.cpu or limits.memory appears in a namespace's quota, the API server needs to know what each new pod will consume before it can decide whether there is room. A pod that does not say is rejected. Nothing that is already running is touched, so the namespace looks fine for days, and then the next deploy fails with a message about a field the author never set. It reads like the deployment is malformed rather than like the quota did it.
kubectl apply -f deployment.yaml
deployment.apps/api configured
kubectl get pods -n team-a
NAME READY STATUS RESTARTS AGE
# nothing new appears
kubectl describe rs api-6d4b
Error creating: pods "api-6d4b-" is forbidden:
failed quota: compute-quota: must specify limits.memory
# The Deployment applied. The ReplicaSet cannot
# create pods. Only the events say why.default is the limit, defaultRequest is the request
These two fields in a LimitRange look like a pair and are not symmetrical. default supplies the limit for a container that gave none, defaultRequest supplies the request. Set default alone and Kubernetes sets the request equal to the limit, because a container with a limit and no request gets its request from the limit. Every container then reserves its full ceiling on a node, the namespace's requests quota fills at the limit value, and the cluster reports itself full while the nodes sit idle.
# defaults only, no defaultRequest:
limits:
- type: Container
default:
memory: 512Mi
# a container that sets nothing ends up with:
resources:
limits:
memory: 512Mi
requests:
memory: 512Mi # <- copied from the limit
# 50 pods now reserve 25Gi of requests.memory
# rather than the 6.25Gi that 128Mi each impliesmin and max reject, defaults fill in
Both mechanisms live in the same LimitRange item, and they do unrelated things. min and max are admission checks: a container outside the range is refused, with a message naming the resource and the bound. default and defaultRequest are mutations: they only ever apply to a container that left the field empty. A LimitRange with only min and max rejects unset containers rather than fixing them, and a max with no matching default is worse, because the max then becomes the default limit and every unset container is handed the largest allocation in the namespace.
limits:
- type: Container
max:
cpu: "2"
min:
cpu: 10m
defaultRequest:
cpu: 100m
default:
cpu: 500m
# container asks for 4 CPU -> rejected:
# maximum cpu usage per Container is 2, but limit is 4
# container asks for 5m -> rejected:
# minimum cpu usage per Container is 10m
# container asks for nothing -> amended to 100m / 500mObject counts, and the two that spend real money
Compute is not the only thing a namespace can exhaust. services.loadbalancers and services.nodeports exist as their own keys because each one costs something outside the cluster: a LoadBalancer Service provisions a billed cloud load balancer, and a NodePort takes one of the 2,768 ports the entire cluster shares. Resources in the core API group have canonical names; everything else uses count/<resource>.<group>, and leaving the group off makes the key match nothing at all.
hard:
pods: "50"
services: "20"
services.loadbalancers: "2" # each one is billed
services.nodeports: "0" # 30000-32767, cluster-wide
persistentvolumeclaims: "20"
requests.storage: 500Gi
gp3.storageclass.storage.k8s.io/requests.storage: 100Gi
count/deployments.apps: "30"
# count/deployments (no .apps) resolves to the core
# group, where deployments do not exist. It is stored,
# it counts nothing, and it never reports an error.Scopes narrow which pods count, and which keys are allowed
A scope makes the quota apply to a subset of pods. The four named ones also restrict what the quota may track: only cpu, memory, requests.*, limits.* and pods, with BestEffort narrower still at pods alone. One unsupported key fails validation for the whole object, not for that line. A PriorityClass scopeSelector has no such restriction, which is why it is the one to reach for when a scoped quota also needs to count objects.
# every pod, every resource type
spec:
hard: {...}
# only pods with activeDeadlineSeconds set
spec:
scopes:
- Terminating # compute keys and pods only
# only pods with priorityClassName: high
spec:
scopeSelector:
matchExpressions:
- scopeName: PriorityClass
operator: In
values: ["high"]
# a pod with any other priority class is not
# counted against this quota at allWhere usage is visible, and why it can exceed the limit
Quota is checked at admission, and the check is not a transaction. Two creates arriving together can both see room and both be admitted, so used can sit briefly above hard. The quota controller recalculates from the objects that exist and the next create is refused, so the overshoot resolves itself. The same is true of a quota applied to a namespace that is already over it: nothing is deleted, used simply reads higher than hard until somebody removes something.
kubectl describe quota compute-quota -n team-a
Name: compute-quota
Namespace: team-a
Resource Used Hard
-------- ---- ----
limits.cpu 6 8
limits.memory 13Gi 16Gi
pods 41 50
requests.cpu 3200m 4
requests.memory 9Gi 8Gi <- over, nothing evicted
# kubectl get quota packs all of this into two
# columns and is the wrong command for the question.