Kafka Quota Generator
The kafka-configs.sh commands that set a quota,
and what the per-broker limit adds up to across the cluster.
set-quotas.sh
updates as you typeCommon mistakes
These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.
Reading a byte-rate quota as a cluster limit
Quotas are per broker. A producer_byte_rate of 10 MB/s across six brokers permits 60 MB/s from that client.
Instead:Divide the cluster budget by the broker count.
Applying a quota to the wrong entity
Quotas resolve by a precedence over user, client-id and the defaults, and the most specific match wins. A user-level quota is silently overridden by a more specific user plus client-id one.
Instead:Check which level you are setting and what else exists at a more specific level.
Setting request_percentage without understanding the unit
It is a percentage of one thread's time multiplied by the thread count, so the ceiling is well above 100.
Instead:Read it against the number of network and IO threads.
Why a Kafka quota is looser than the number you set
Two things about quotas surprise almost everyone, and both make the limit weaker than intended.
Byte rates are enforced per broker
producer_byte_rate and consumer_byte_rate are limits per broker, not per cluster. A client producing to partitions spread over six brokers can move six times the number you set, and there is no cluster-wide byte rate quota at all. Sizing a quota from a total throughput figure and dividing by nothing is the usual mistake, and the error is always in the permissive direction.
request_percentage is not out of 100
It is a percentage of one broker's total request handler capacity, which is (num.io.threads + num.network.threads) multiplied by 100. On a broker with 8 io threads and 3 network threads that is 1100, so a value of 200 means two threads' worth. Treating 100 as the ceiling throttles a client to a single thread, which is a very effective way to make it appear broken.
Throttling delays, it does not reject
A throttled client is not told it was throttled. The broker holds the response back, so a producer sees latency and eventually a timeout and a consumer sees reduced throughput. Nothing in either client's error handling mentions a quota. The broker's throttle-time metrics are the only signal, which is why an unexplained slow client is worth checking against them.
Precedence replaces rather than combines
Quotas resolve from the most specific match to the least: the exact user and client-id pair, then user alone, then client-id alone, then the defaults. A more specific match replaces a broader one entirely rather than tightening it, so adding a user-and-client quota does not narrow an existing user quota, it overrides it for that client-id. A default user quota is the safest thing to set first, because it bounds every client nobody has thought about yet.
What this cannot see
It does not know your actual traffic, your broker count beyond what you typed, or what quotas already exist. A client-id quota is also not a boundary: client.id is set by the client, so anything that wants more capacity can change its own. Quota by user for anything that matters, and read back what is in force with the describe command in the output, because quotas live in cluster metadata rather than in a file in your repository.