Prompt caching made our bill go up: the write premium nobody budgets for

You turned on prompt caching, and the bill went up 25%.

Nothing is broken. Nothing errored. The requests succeeded, the responses were correct, and cache_read_input_tokens came back as zero on every single one. You are now paying a premium to write a cache that nothing ever reads.

This is the most common way prompt caching goes wrong, and it is worth understanding the economics before the invalidation rules, because the economics are what decide whether any of it was a good idea.

The three multipliers, and the one that surprises people

Caching is described as a discount. It is two prices, and only one of them is a discount:

Operation	Cost, relative to the base input rate
Cache read	0.1x
Cache write, 5 minute TTL	1.25x
Cache write, 1 hour TTL	2x
No caching at all	1x

A read is a tenth of the price. That is the headline and it is real.

A write costs more than not caching. Not the same, more. Putting 20,000 tokens into a five minute cache costs 25% above sending those tokens normally, and putting them into a one hour cache costs double. If nothing reads that entry before it expires, you have simply paid a premium for nothing.

Where the break-even actually falls

One write plus N reads has to beat N+1 uncached requests:

write + (N x 0.1)  <  (N + 1) x 1

At the five minute TTL that is 1.25 + 0.1N < N + 1, which crosses at one read. Two requests in total. At one hour the write is 2x, so you need two reads, three requests in total.

Those thresholds are low, which is why caching is usually worth it. They are not zero, which is why the situations below lose money:

• A per-request document that is never seen again.
• A per-user context assembled fresh on every call.
• A retrieved RAG chunk that differs each time.
• Any prefix on a one hour TTL that is read once.

Each of those writes an entry that nothing reads. You are paying 1.25x or 2x for something that would have cost 1x.

How it shows up in production

The failure is quiet, which is what makes it expensive. Three shapes to recognise:

The bill went up and nothing else changed. Caching enabled, hit rate zero, 25% premium on every request. Usually a prefix that is not actually stable.

The bill did not move. Caching enabled, working, and the cached portion was a small fraction of the prompt. Correct behaviour, no benefit. Worth measuring before concluding caching “does not work here”.

The dashboard says the prompt shrank. It did not. input_tokens in the usage block is the uncached remainder, and a dashboard that reads that field alone under-reports by exactly as much as the cache is working. The real prompt is:

input_tokens + cache_creation_input_tokens + cache_read_input_tokens

This one is genuinely perverse: the better your caching gets, the more wrong your cost dashboard becomes. An agent that ran for four hours across two hundred turns can report a four thousand token prompt.

The wrong instinct: raise the TTL

When hit rates are poor the reflex is to reach for the one hour cache. It feels like it should help. Traffic is bursty, entries are expiring, so keep them alive longer.

It usually makes things worse, and the arithmetic says why. The longer TTL doubles the write cost, from 1.25x to 2x. It only pays when it rescues reads that would genuinely have missed at five minutes, which means your traffic has gaps longer than five minutes and enough volume afterwards to repay a doubled write.

If the hit rate is zero because the prefix is unstable, a longer TTL changes nothing except the size of the premium. You are now paying 2x instead of 1.25x for the same zero reads.

Diagnose before you tune. cache_read_input_tokens on the response is the only thing that settles it. Zero across repeated requests means the prefix is moving, and no TTL fixes that.

Caching is a prefix match, and that is the whole game

The cache key is the exact bytes of the rendered prompt up to each breakpoint. One changed byte anywhere invalidates everything after it.

The render order is fixed and worth memorising:

tools  ->  system  ->  messages

Anything volatile placed early poisons everything downstream. The usual culprits, in rough order of how often they turn up:

• datetime.now() interpolated into the system prompt
• a session id, request id or user id near the front
• json.dumps(d) without sort_keys=True, so key order varies between runs
• a tool list built per user, so no two users share a prefix
• conditional system sections, where every flag combination is its own prefix
• a fork operation (summarisation, compaction, a sub-agent) that rebuilds system or tools slightly differently from its parent

That last one catches teams who have already done everything else right. A sub-agent that reconstructs the system prompt from the same template but in a different order misses the parent’s cache entirely, and it looks like a sub-agent problem rather than a caching one.

Tool definitions are the sharpest edge

Tools render at position zero. Adding or removing a single tool mid-conversation changes the first byte, which invalidates every breakpoint after it, including the entire conversation history. Not just the tool section.

This is why a server that registers tools dynamically based on context is expensive in a way that never appears in the tool list’s own size.

What does not invalidate everything

Worth knowing, because over-caution here costs real flexibility. The API has three cache tiers, and a change only invalidates its own tier and below:

Change	Tools	System	Messages
Tool definitions	invalid	invalid	invalid
Model switch	invalid	invalid	invalid
System prompt content	kept	invalid	invalid
tool_choice, images, thinking on/off	kept	kept	invalid
Message content	kept	kept	invalid

So you can toggle tool_choice per request, or turn thinking on and off, without losing the tools and system cache. Only tool-definition and model changes force a full rebuild.

The minimum, and why it is not what you would guess

A prefix shorter than the model’s minimum is silently not cached. No error, no warning, cache_creation_input_tokens comes back zero. You pay full price and believe caching is on.

The figure ranges from 512 to 4,096 tokens depending on the model. And here is the part worth writing down somewhere:

It is not monotonic by generation. Some current models cache from 512 tokens. Some older ones require 4,096, eight times as much. So moving a workload to a different model can turn caching off entirely with no code change and no error, and moving it back turns it on again. A team that migrates to an older model for cost reasons can lose more to a silently disabled cache than they saved on the rate.

Check the minimum for the specific model. Do not assume newer means lower.

What changed recently

Three things worth knowing if your caching strategy was designed a while ago:

The minimum dropped on the newest models. Claude Opus 5 caches from 512 tokens, half what Opus 4.8 requires. Prompts you previously wrote off as too short to cache may now create entries with no code change at all. Worth re-auditing anything under about 1,000 tokens.

Mid-conversation system messages exist. Editing the top-level system field mid-session invalidates the entire cached history, because system renders ahead of messages. On Claude Opus 5, Opus 4.8, Fable 5 and Mythos 5 you can instead append a {"role": "system", ...} entry to the messages array. It sits after the cached prefix, so the history survives, and it carries operator authority rather than being user text. Not available on Sonnet 5; catch the 400 and fall back to a <system-reminder> block in the user turn.

Tools can change without discarding the cache. Behind a beta flag on Claude Opus 5, tool_addition and tool_removal blocks let you change the tool set between turns without invalidating the prefix. The tool has to be declared up front with defer_loading: true. This is the escape hatch for the position-zero problem above, and it is the difference between a dynamic tool set being unaffordable and being routine.

Fixing it on a system already running

You cannot usually rewrite the prompt architecture of a live product. In rough order of effort:

1. Measure first. Log cache_read_input_tokens, cache_creation_input_tokens and input_tokens separately for a day. The ratio tells you which of the three failure shapes you have.

2. Diff the bytes, not the code. If the hit rate is zero, capture the rendered prompt from two consecutive requests and diff them. Reading the code to find the volatile field is slower and less reliable, because the thing that changed is usually not where you would look. Sorted JSON serialisation and a timestamp moved out of the system prompt fix the majority of real cases.

3. Move volatile content after the last breakpoint. Dynamic context does not have to leave the prompt, it has to leave the prefix. A fact injected as a message at turn five invalidates nothing before turn five.

4. Freeze the tool set per conversation. Serialise deterministically, sorted by name. If the set genuinely must vary, look at whether the deferred-loading route above is available to you.

5. Only then tune the TTL. With a stable prefix and a measured hit rate, the choice between five minutes and one hour is a straightforward comparison of your inter-request gap against the doubled write cost.

Two operational details that are easy to miss

Concurrent requests all miss. A cache entry becomes readable only after the first response begins streaming. Fire ten parallel requests with an identical prefix and all ten pay full price, because none of them can read what the others are still writing. For fan-out, send one request, await the first streamed token, then fire the rest.

The 20-block lookback. Each breakpoint walks backward at most 20 content blocks looking for a prior entry. An agentic turn that adds more than 20 blocks, which is easy with several tool_use and tool_result pairs, pushes the previous cache out of range and the next request silently misses. Place an intermediate breakpoint every 15 blocks or so in long turns. You get four breakpoints per request; spending one on this is usually worth it.

When this advice is wrong

Some workloads should not cache, and it is worth being able to say which:

Genuinely unique prompts. If the first 1,000 tokens differ per request there is no reusable prefix. Adding cache_control only buys the write premium.

Very low volume. A prefix read once an hour on a five minute TTL never gets a hit. Either accept full price or pre-warm on a schedule, and pre-warming has its own cost.

Output-dominated bills. Caching only touches input. If output is 80% of your spend, a perfect caching implementation moves the total by a few percent. Shorter answers and a lower effort setting are the lever there.

Rate limits rather than cost. This one catches people who have just finished a caching project. A cached read is cheap and it is not smaller. Those tokens were still processed, so they count in full against your input tokens-per-minute limit. A workload that caching made affordable can still be throttled on precisely the tokens it stopped paying full price for. Budget cost on the cached split and rate limits on total prompt size; they are different numbers.

The short version

• A write costs 1.25x at five minutes and 2x at one hour. It is not free.
• Break-even is two requests, or three on the long TTL.
• Zero hit rate means the prefix is moving. Diff the bytes; no TTL fixes it.
• The minimum prefix is 512 to 4,096 tokens and is not lower on newer models as a rule.
• input_tokens is the uncached remainder, not the prompt.
• Cached tokens are cheap and still count against your rate limit.

If you want the arithmetic against your own numbers, the prompt caching planner takes your prefix size, reuse, TTL and rates and reports whether the write is repaid and by how much. It checks the prefix against the specific model’s minimum too, which is the check most worth running before concluding that caching does not work for you.