Your LLM cost dashboard is wrong, and it gets worse as caching improves

An agent ran for four hours across two hundred turns. You check the usage and the prompt was four thousand tokens.

It was not. It was closer to two hundred thousand. The field you read counts only the part that was not served from cache, and on a well-cached agent loop that is a small tail.

This is the most common LLM cost reporting error, and it has an unusual property: it gets worse as the system gets better.

Three fields, one prompt

The usage object splits input into three, and they are priced differently:

Field	What it counts	Priced at
input_tokens	The uncached remainder	1x
cache_creation_input_tokens	Tokens written to cache this request	1.25x or 2x
cache_read_input_tokens	Tokens served from cache	0.1x

The prompt is the sum of all three:

total_prompt = (
    usage.input_tokens
    + usage.cache_creation_input_tokens
    + usage.cache_read_input_tokens
)

input_tokens on its own is whatever was left over after the cached prefix was accounted for. It is not “the prompt minus caching savings” or “the billable input”. It is one of three line items.

The inverted failure mode

A dashboard that multiplies input_tokens by the input rate under-reports by exactly the amount the cache is carrying:

Cache working	Reported cost	Reality
Badly	Roughly right	Roughly right
Well	Substantially under	Actual bill
Nearly perfectly	Close to zero	Actual bill

So the metric degrades as the system improves, which produces two specific organisational failures.

The day someone fixes an invalidation bug, the cost graph falls off a cliff and the invoice does not move. Somebody gets credited with a saving that did not happen, and the number that was wrong before is now wrong by more.

A suspiciously cheap agent gets read as a caching success. It is a reporting artefact. The genuinely useful signal, the ratio between the three fields, was never plotted.

How it shows up

Finance and engineering disagree and neither is lying. Engineering’s dashboard reads one field; the invoice sums three. The gap is exactly the cached volume, which is why it grows over time rather than staying a constant percentage.

Per-customer cost attribution is wrong in a specific direction. Customers whose prompts share a cached prefix, typically your highest-volume ones, appear cheapest by the widest margin. Pricing decisions built on that are wrong in the expensive direction.

A “cost per request” metric that drifts down without any change. As a conversation grows, more of it is cached, so input_tokens becomes a smaller share. The metric improves because the measurement is decaying.

The wrong instinct: multiply by an average

The usual patch is to apply a correction factor: measure the real ratio once, multiply the reported figure, move on.

It fails because the ratio is not stable. It varies with conversation length, because later turns have more cached history. It varies by endpoint. It varies between a cold start and a warm one. And it changes silently whenever anyone touches the prompt, which is the exact moment your correction factor becomes wrong with nothing to signal it.

Log all three fields. They are already in the response. There is no sampling, no estimation and no factor to maintain:

resp = client.messages.create(...)
u = resp.usage
metrics.emit({
    "input_uncached": u.input_tokens,
    "cache_write":    u.cache_creation_input_tokens,
    "cache_read":     u.cache_read_input_tokens,
    "output":         u.output_tokens,
})

Four numbers, exact, and the cost calculation becomes arithmetic rather than inference.

The other half: output is where the money usually is

Output is priced several times higher than input on essentially every model, and teams optimising cost reliably start on the wrong side of the ledger.

Take a workload at 20,000 input tokens and 1,000 output tokens per request on a model at $5 input and $25 output per million:

input:   20,000 x $5  / 1M = $0.100
output:   1,000 x $25 / 1M = $0.025

Input dominates here, and caching is the right lever. Now the same model on an agentic workload with long reasoning, 20,000 in and 8,000 out:

input:   20,000 x $5  / 1M = $0.100
output:   8,000 x $25 / 1M = $0.200

Output is now double the input cost, and prompt caching cannot touch it. A perfect caching implementation moves the smaller number.

The lever for output is asking for less: a conciseness instruction in the system prompt, or a lower effort setting. Both move this figure where trimming the prompt does not.

Note also that thinking tokens are output tokens. They are billed as output and they count against max_tokens. A task that reasons at length is expensive in a way the visible answer does not explain, and it is the usual cause of an agentic workload whose cost profile does not match a chat workload’s.

Cache writes with no reads

The other line item worth watching:

cache_creation_input_tokens: 22000
cache_read_input_tokens: 0

Consistently. That is a prefix which changes slightly on every request, writing a fresh entry each time and reading none. A write costs 1.25x the input rate at the five minute TTL and 2x at one hour, against 1x for not caching at all, so this configuration is actively more expensive than having caching off.

The usual causes are a timestamp in the system prompt, an unsorted JSON serialisation, or a tool list built per user. All three are invisible in code review and obvious in a byte-level diff of two rendered prompts.

What changed recently

Cache minimums dropped on the newest models, to 512 tokens on Claude Opus 5 against 1,024 on Opus 4.8 and 4,096 on some older ones. If you concluded a prompt was too short to cache, that conclusion may no longer hold. The figure is not monotonic by generation, so it is worth checking per model rather than assuming newer means lower.

Mid-conversation system messages let you inject operator context without editing the top-level system field, which would invalidate the entire cached history. Available on Claude Opus 5, Opus 4.8, Fable 5 and Mythos 5; not on Sonnet 5. This is the difference between dynamic context being affordable and being a cache reset every turn.

Intro pricing expires. Claude Sonnet 5 carries an introductory rate through 2026-08-31. Any forecast built on it has a cliff in it, and a hardcoded rate table will not notice.

Fixing it on a running system

1. Log all four numbers today. Before changing anything else. You cannot reason about a ratio you are not recording, and a week of history makes every subsequent decision easier.

2. Reconcile one day against the invoice. Sum the four, apply the published rates, compare. If it does not match within a percent or two, something else is wrong: a model you forgot about, a batch job, a retry storm.

3. Plot the split, not just the total. Cached versus uncached versus output, over time. This is the graph that tells you when a prompt change broke invalidation, usually within hours rather than at the end of the month.

4. Alert on cache_read_input_tokens hitting zero on an endpoint where it was previously non-zero. That single alert catches the entire class of prefix-invalidation regressions, and it is one line.

5. Only then optimise. With the split visible, the question of whether to work on the input or the output side answers itself.

Cheap to buy is not the same as small

Worth stating separately because it catches people who have just finished a caching project.

A cached read costs a tenth. It is not a tenth of a token. Those tokens were still processed, so they count in full against your input tokens-per-minute rate limit. A workload that caching made affordable can still be throttled on exactly the tokens it stopped paying full price for.

Budget cost on the cached split. Budget rate limits on total prompt size. They are different numbers derived from the same three fields, and conflating them produces a system that is cheap and throttled.

When this does not matter

If you are not using prompt caching, input_tokens is the whole prompt and everything above is theoretical. Worth knowing before you enable caching rather than after.

At low volume, the absolute error may be a few dollars a month and not worth instrumenting. The threshold is roughly where somebody starts asking why the bill is what it is.

If you only need relative comparisons between two versions of the same prompt, a consistently-wrong metric is still directionally useful. It stops being useful the moment you compare across prompts with different caching profiles, which is usually the comparison you actually wanted.

The short version

• input_tokens is the uncached remainder. The prompt is the sum of three fields.
• The error grows as caching improves, so the metric decays as the system gets better.
• Log all four numbers. They are already in the response; there is nothing to estimate.
• Output is priced several times higher and caching cannot touch it. Check which side dominates before optimising.
• Writes with no reads cost more than no caching at all.
• Alert on cache_read_input_tokens going to zero. It catches the whole class.
• Cached tokens are cheap and still count in full against your rate limit.

The LLM API cost calculator prices cache reads and writes as separate lines rather than folding them into input, and takes your own rates rather than a published table, because rates change and a static page cannot notice. It will not count tokens for you: that needs the provider’s tokenizer, and a character-count estimate is wrong by a wide margin on code and non-English text.