LLM Prompt Caching Planner
Work out whether caching a prefix pays, and by how much. A write costs 1.25x the input rate at five minutes and 2x at one hour; a read costs 0.1x. Below the break-even, caching is more expensive than not caching.
caching-plan.txt
updates as you typeCommon mistakes
These are the ones that fail silently. The config is accepted, nothing raises an error, and the consequence arrives later.
Caching a prefix that is read once
A write costs 1.25x the input rate at five minutes and 2x at one hour, against 1x for sending it uncached. Below the break-even, caching is a cost.
Instead:Cache what is read at least twice on the five minute TTL, three times on the hour.
Assuming a prompt caches because you set `cache_control`
A prefix under the model's minimum is silently not cached. No error, no warning, and the cache-creation count comes back zero. The minimum ranges from 512 to 4096 tokens and is not lower on newer models as a rule.
Instead:Check the minimum for your specific model, and verify with `cache_read_input_tokens` on a real response.
Interpolating anything dynamic into the system prompt
Caching is a prefix match, and the render order is tools, then system, then messages. A timestamp, a user id or an unsorted JSON serialization near the front invalidates everything after it, on every request.
Instead:Stable content first, volatile content after the last breakpoint. Inject per-request context as a later message.
A cache write costs more than not caching, which is the part people skip
Reads are a tenth of the input rate. Writes are 1.25x at five minutes and 2x at one hour. Caching something read once is strictly more expensive than sending it uncached, and nothing warns you.
The break-even is two requests, or three
One write plus N reads has to beat N+1 uncached requests. At the five minute TTL that is 1.25 + 0.1N against N+1, which crosses after a single read: two requests in total. At one hour the write is 2x, so it takes two reads and three requests. Below that line the cache is a cost, not a saving.
The minimum cacheable prefix is not the same on every model, and it does not decrease with age
It ranges from 512 to 4096 tokens depending on the model, and newer does not reliably mean lower: some older models require eight times the prefix of the current ones. Below the minimum the request succeeds, nothing errors, and the cache-creation count comes back zero. Moving a workload between models can therefore turn caching off with no code change at all.
Caching is a prefix match
One changed byte anywhere invalidates everything after it. The rendered order is tools, then system, then messages, so anything volatile placed early poisons the whole prefix: a timestamp in the system prompt, a per-user id, an unsorted JSON serialisation, or a tool set that varies by request. The request still succeeds; the only symptom is the bill.
Changing tools discards everything
Tool definitions are at position zero. Adding or removing one mid-conversation changes the first byte, which invalidates every breakpoint after it including the entire conversation history, not just the tool section.
Verify with the numbers, not by inspection
cache_read_input_tokens on the response is the ground truth. If it is zero across repeated requests with what you believe is an identical prefix, something is moving. Diff the rendered prompt bytes between two calls rather than reading the code.
A gap longer than the TTL means every read is a miss
The entry expires between requests, so each call writes a fresh cache and none of them reads one. You pay the write premium on every single request and get nothing back, which is worse than not caching at all. Either use the longer TTL, pre-warm on a schedule, or stop.
What this cannot see
Your prefix. It cannot tell you whether the thing you plan to cache is genuinely stable, which is the question that decides whether any of this arithmetic applies. Rates come from a table with a date on it and are overridden by any input-rate you supply; use your own, because published pricing changes and a static page cannot notice. Everything runs in your browser.