Work out what a tool-calling agent costs per run and per month, with the quadratic growth that a cost-per-turn estimate misses. Then see what prompt caching does to it, and where the context window stops the loop.
Worked setups you can load into the form above. Each one is a decision the generator makes differently, and the reason it makes it.
566,000 input tokens to produce 8,000 of output, because turn 20 carries turns 1 to 19 with it. A cost-per-turn estimate says 110,000.
The resent prefix is charged at a tenth and only the increment is written, which takes the same run to under a quarter of the price. The token counts do not change at all.
Ten minutes between turns is longer than the five minute cache lives, so every request writes its whole prefix at the premium and reads none of it back. Caching here costs more than not caching.
Two hundred turns against a 200,000 token window. The loop does not slow down as it approaches the limit, it is rejected at about turn 82 with every turn before it already paid for.
These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.
Every request carries the whole conversation, so the input over T turns is the fixed prefix T times plus the increment times T(T-1)/2. At twenty turns the resent history is around 90% of the input tokens, and a linear estimate is off by a factor of five.
Instead:Cap the loop, and attack the increment rather than the turn count, because the increment is the term that is squared.
The system prompt is a constant charged once per turn. A tool result is charged once for every turn that follows it, so a token added at turn 3 of 30 is paid for 27 more times. The tool results are usually ten to a hundred times larger to begin with.
Instead:Truncate and paginate tool output first, with an explicit marker so the model knows there is more. Halving it roughly halves the quadratic term.
The default cache lives five minutes. A loop with a human between turns writes the whole prefix at the write premium on every request and reads none of it back, which costs more than not caching at all.
Instead:Use the one hour cache for an interactive agent, or turn caching off. Which one wins depends on how many turns follow the pause.
It is not one. The request is rejected outright, partway through whatever the agent was doing, after the money for every earlier turn has been spent. Long runs are disproportionately the ones already going badly.
Instead:Track the running total and compact before the limit: summarise older turns, or drop tool results while keeping the calls that produced them.
Each tool's schema and description is resent on every request whether or not it is called. Forty tools where the run uses four is forty definitions paid for on every turn.
Instead:Cache the definitions, then narrow the set to what this agent needs. It usually improves tool selection as well as the bill.
Cost per turn times turns is the wrong shape. Each request carries the whole conversation, so turn 20 pays for the nineteen before it, and the total grows with the square of the turn count.
Every request sends the system prompt, the tool definitions and the whole conversation so far. With a uniform turn that is a fixed part resent T times, plus an increment resent once for every turn that follows it. The second term is the one that matters and the one that gets left out, and it is why a loop that runs twice as long costs roughly four times as much rather than twice.
input over T turns
= T x (system + tools + user)
+ (output + tool result) x T x (T - 1) / 2
T=20, fixed 5,500, increment 2,400
= 110,000 + 456,000 = 566,000 input tokens
for 8,000 tokens of outputA file read, a search result or a page of logs is routinely ten to a hundred times the assistant message that asked for it, and it stays in the history for the rest of the run. A token added at turn 3 of 30 is paid for 27 more times. That makes truncating tool results the highest-leverage change available and almost never the first one people reach for, because the tool looks like it is called rarely and the cost does not.
A cache read is a tenth of the input rate, and the resent prefix is exactly what caching is for. The write premium is paid once per increment and is linear, so the saving grows with the length of the run rather than shrinking. It does not change the shape of the curve; it divides the coefficient on the dominant term by ten, which is often the difference between viable and not.
Five minutes by default. A loop that runs to completion in one go hits it every turn. A loop that waits for a person to read the output and reply does not, and then every request writes its whole prefix at the write premium and reads none of it back. That is the one case where caching costs more than not caching, and it is exactly the shape of a chat agent.
turns seconds apart every read hits, roughly 4x cheaper
turns minutes apart every write is paid for, nothing is read
back: 1.25x the uncached priceThe loop does not get slower as it approaches the limit. The request is rejected, partway through whatever the agent was doing, having already spent the money for every turn before it. Agent runs have a long tail, and the runs that need the most turns are the ones already going badly, so the wall is hit disproportionately by the cases you most want to finish.
Every tool's schema and description is sent on every request whether or not it is called. A registry of forty tools where the run uses four pays for all forty, forty times. Caching them is the cheapest fix; narrowing the set the agent can see is the better one, and it usually improves tool selection as well.
It models a uniform turn, and real runs are not uniform: one tool call returns a hundred tokens and the next returns forty thousand. Use averages and treat the result as a shape rather than an invoice. It also cannot see retries, which resend the whole context again for nothing, or a sub-agent whose own loop is a separate quadratic inside this one. Both make the real number larger. Count the real prompt with the token counter on this site rather than estimating, because the fixed part is multiplied by every turn.