The prompt fits and the request still fails: max_tokens comes out of the same window

A 190,000 token prompt against a 200,000 token window. Comfortable, with ten thousand to spare.

Set max_tokens: 20000 and the request is rejected.

Three limits, all called a token limit

The confusion is that three different numbers get described the same way:

Limit	What it bounds	Typical size
Context window	Prompt and response together	200K to 1M
max_tokens	The response alone, this request	You choose
Output ceiling	The most the model will produce in one response	64K to 128K

So the arithmetic is not prompt ≤ window. It is:

prompt + max_tokens ≤ context window
max_tokens ≤ output ceiling

190,000 plus 20,000 is 210,000, which does not fit in 200,000. The prompt was never the problem.

The output ceiling is a separate hard limit, and it is much smaller than the window. Being able to hold a million tokens does not mean being able to emit a million: a request whose max_tokens exceeds the ceiling is rejected regardless of how much window is free.

Thinking spends the answer’s budget

Reasoning tokens are output tokens. They are billed as output and they count against max_tokens, and this produces a failure much stranger than a rejected request.

You set max_tokens: 4000 because the answer should be about two thousand tokens. The model works through a hard problem for 4,000 tokens of reasoning, and the response comes back having produced nothing readable, with a max_tokens stop reason.

The API call succeeded. The response is a valid response object. It just does not contain an answer.

This has become the dominant version of the problem as adaptive thinking has become the default rather than an opt-in. A budget sized against the visible answer was fine when reasoning was off; it is now routinely consumed before the answer starts.

The fix is headroom, not precision. On demanding work at higher effort, several times the expected answer length is not unreasonable, and over-reserving costs nothing: you are billed for tokens produced, not tokens reserved. max_tokens is a ceiling, not a target.

Read the stop reason. Always.

This is the through-line, and it is the single most consequential habit in this whole area.

A truncated response is textually indistinguishable from a complete one. The only place that fact appears is stop_reason:

Value	Complete?	What to do
end_turn	Yes	Use it
tool_use	Yes	Execute the tool, continue the loop
stop_sequence	Yes	Use it
max_tokens	No	Raise the budget, or continue
refusal	No	Handle it; do not retry the same prompt
pause_turn	No	Resend to resume a server-tool turn
model_context_window_exceeded	No	Compact or split the conversation

Code that reads response.content[0].text without checking stop_reason first ships fragments to users, and it does so in exactly the cases where the answer mattered enough to be long. That is not a coincidence, it is selection bias: short answers never truncate.

resp = client.messages.create(...)
if resp.stop_reason != "end_turn":
    handle_incomplete(resp.stop_reason)   # do not fall through

Worth knowing: stop_details is populated only when stop_reason is refusal, and is null for everything else. Branch on stop_reason; treat stop_details as extra colour.

The wrong instinct: trim the prompt

When a request overflows, the reflex is to cut the prompt. Sometimes correct, often the harder path, and occasionally it makes things worse.

The prompt has four parts and they behave completely differently:

Part	Grows?	Cheap to cut?
System prompt	No, chosen once	Yes, and usually already small
Tool definitions	Only when you change them	Yes, and often the largest surprise
Conversation history	Every turn, without a decision	Hard: it is the context
Documents	Per request	Yes, if retrieval can be narrowed

Conversation history is the part that grows without anyone deciding it should, and it is the part that is hardest to cut because it is what makes the conversation work. A margin that looks comfortable at turn three is gone by turn thirty.

Tool definitions are the surprise. They render at the very front of the prompt, ahead of the system prompt, and they are paid on every request. A team that has accumulated forty tools is often carrying more tokens in definitions than in the conversation, and trimming descriptions there is both easier and more effective than summarising history.

Before trimming anything, get the split. The proportions are almost never what people guess.

What changed recently

Adaptive thinking is on by default on the newest models. On Claude Opus 5, a request that omits the thinking parameter entirely now thinks, where on Opus 4.8 and 4.7 the same request did not. This is a silent cost and truncation change: a workload that ran thinking-off by omission and sized max_tokens tightly around its answer can now truncate mid-response with no code change. Revisit max_tokens on any route that never set thinking.

Disabling thinking is now capped by effort. thinking: {type: "disabled"} is accepted only at effort high or below on Claude Opus 5; pairing it with xhigh or max returns a 400, and the check is per request rather than per conversation.

1M context windows are widely available, which changes the calculus: overflow is much rarer and cost becomes the binding constraint rather than capacity. A million-token prompt fits and is expensive. The planning question has shifted from “will this fit” to “what is this costing me every turn”.

Compaction and context editing are real features now rather than something you hand-roll. Compaction summarises earlier context server-side; context editing clears stale tool results outright. They solve different problems and long-running agents often use both.

Fixing it on a running system

1. Measure the split first. System, tools, history, documents. Use the provider’s token counting endpoint against a real prompt rather than estimating; a character-count approximation is wrong by a wide and unpredictable margin on code and non-English text, and a wrong budget is worse than no budget.

2. Handle the stop reason before anything else. This is a small change that converts silent truncation into a visible, handleable event. Do it before optimising, because until it is in place you cannot tell how often you are truncating.

3. Give max_tokens real headroom, particularly on any route with thinking enabled. Reserving more costs nothing.

4. Trim tool definitions. Usually the largest easy win, and it compounds: they are paid every request and they sit at the front of the cache prefix, so changing them mid-conversation also discards the cache.

5. Choose an eviction strategy before you need one. Compaction, context editing, or your own truncation. All three work. Choosing between them during an incident is the version that goes badly.

When the window is not the constraint

If you are on a 1M-context model, overflow is unlikely and cost is the real limit. A prompt that fits comfortably and costs a fortune every turn is the more common failure now, and the answer is caching and trimming rather than window management.

If your conversations are short by design, single-turn extraction or classification, history never grows and none of this applies. Size max_tokens to the answer and move on.

If you are hitting the output ceiling rather than the window, no amount of prompt trimming helps. That is a different problem with a different fix: split the work into several requests, or stream and stitch. Asking for a 200,000 token document in one response is not going to start working.

The short version

• The window holds prompt plus response. max_tokens is reserved from it.
• The per-response output ceiling is a separate, much smaller limit.
• Thinking tokens are output tokens and spend max_tokens before the answer does.
• A truncated answer looks exactly like a complete one. Read stop_reason.
• Tool definitions render first and are usually the biggest easy saving.
• On a 1M window the binding constraint is cost, not capacity.
• Adaptive thinking now defaults on for some models. Re-check max_tokens on routes that never set it.

The context window planner breaks a request down across system prompt, tools, history, documents and reserved output, and reports the headroom against both the window and the model’s separate output ceiling. It will not count tokens for you, for the reason above.