LLM API Rate Limit Planner
Enter the throughput you need, the shape of a request, and your own limits. Get the axis that binds first, the rate it actually permits, and why cache reads do not help.
rate-limit-plan.txt
updates as you typeCommon mistakes
These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.
Checking the request limit and assuming you fit
Three limits are enforced separately and only the smallest matters. A generous request limit means nothing if the token limit permits a fraction of that throughput.
Instead:Divide each token limit by the tokens a request uses. The smallest resulting rate is your real ceiling.
Expecting prompt caching to relieve the rate limit
Caching cuts the price of a cached read to a tenth. It does not shrink the prompt, so those tokens count in full against the input limit. A workload caching made affordable can still be throttled.
Instead:Budget cost on the cached split and rate limits on total prompt size. They are different numbers.
Treating a per-minute limit as a per-minute budget
Enforcement is continuous, not reset on the minute. Sending a minute's traffic in ten seconds hits the limit even though the average is fine, and that shape comes from concurrency rather than throughput.
Instead:Cap concurrency, throttle on `x-ratelimit-remaining`, and honour `retry-after` rather than a fixed backoff.
Three limits, and only one of them binds
Requests per minute, input tokens per minute and output tokens per minute are enforced separately. Which one stops you depends on the shape of your requests, not their number, and a tier upgrade raises all three whether or not you needed all three.
Work out the throughput each limit permits, then take the smallest
An input limit of 400,000 tokens a minute against a 10,000 token prompt permits forty requests a minute regardless of what the request limit says. Comparing your totals against each limit separately tells you whether you fit; dividing tells you which one binds and by how much.
Cache reads are cheap and are not free here
Caching cuts the price of a cached read to a tenth. It does not shrink the prompt. Those tokens were still processed, so they count in full against the input limit. A workload that caching made affordable can still be rate-limited on precisely the tokens it stopped paying full price for, which is a genuinely surprising place to end up.
A per-minute limit is not a per-minute budget
Enforcement is continuous rather than reset on the minute, so sending a minute's traffic in the first ten seconds hits the limit even though the average is fine. That shape comes from concurrency, not throughput. Cap concurrency at roughly your permitted rate divided by sixty, times the average request duration in seconds.
The headers tell you before the 429 does
Every response carries the limit and the remaining allowance per axis. Throttling on x-ratelimit-remaining is how you avoid the wall; honouring retry-after is how you recover from it. Backing off on a fixed schedule instead means hammering a limit you are already over. The official SDKs retry 429 and 5xx with backoff; on raw HTTP that is yours to write.
A lopsided workload wastes a tier upgrade
If input tokens bind five times sooner than requests, moving up a tier buys headroom on an axis you were not using to get it on the one you were. Reshaping the request is usually cheaper: fewer, larger calls move you along the request axis, shorter prompts along the input axis.
What this cannot see
Your actual limits, which vary by organisation, tier, model and sometimes by feature: some capabilities draw on their own separate pool rather than the shared one. Read them from your own rate limit page and enter them here. It also cannot see burst behaviour, retry storms, or traffic from anything else in your organisation sharing the same quota. Everything is computed in your browser from what you type.