Does the prompt cache survive if I remove a tool mid-conversation?

With the beta header mid-conversation-tool-changes-2026-07-01 it does, provided the removal runs through a tool_removal block in a message with role system. The tools array in the request stays untouched, which is why the hashed prefix does not change. Delete the tool from the tools array instead and, according to Anthropic's documentation, that invalidates tools, system prompt and messages in one go.

Which changes invalidate the prompt cache at all?

Changing tool definitions costs the entire cache. Toggling web search or citations, and the speed setting, invalidate the system prompt and messages. tool_choice, disable_parallel_tool_use and adding or removing images affect the messages only. Thinking and effort parameters act on the messages, and on some models additionally on tools and system prompt.

Does the feature work with Claude Sonnet 5 as well?

No. For mid-conversation system messages and the associated tool change blocks Anthropic explicitly lists Fable 5, Mythos 5, Opus 4.8 and Opus 5, and excludes Sonnet 5. If you are on Sonnet 5 and only need to load tools in rather than withdraw them, Tool Search with defer_loading gets you to the same place.

Why does usage show nothing but zeros despite cache_control?

The prefix is probably too short. Claude Opus 5 caches from 512 tokens, Sonnet 5 from 1,024, Haiku 4.5 from 4,096. Below that threshold nothing is cached silently, no error is raised, and cache_creation_input_tokens and cache_read_input_tokens simply stay at 0.

Swapping tools mid-conversation without losing the prompt cache

An agent has been running for twenty turns, the context is nicely filled, and now a tool needs to go. Write access to the repository, for example, because the pull request is out and from here on it is review only. Delete that entry from the tools array and the tool takes the entire prompt cache with it. The next request writes the whole conversation so far back into the cache, at 1.25 times the input price instead of 0.1 times for a read.

Since the release of Claude Opus 5 there is a way around exactly that. It is called mid-conversation-tool-changes-2026-07-01 and it is a beta header. What it does is unspectacular, and that is precisely why it works.

Why a single tool costs the whole conversation

Prompt caching works off a hash of the request prefix. Anthropic hashes strictly in the order tools, then system, then messages. A change at one level invalidates that level and everything after it.

That makes the tool definition the most expensive spot in the entire request, because it sits right at the front. Change a name, a description or a schema there and tools, system prompt and all messages are invalid in one stroke. The invalidation table in the documentation shows the gradient clearly. Toggling web search or citations costs the system prompt and messages. The speed setting likewise. tool_choice and disable_parallel_tool_use cost only the messages, exactly like adding or removing images. Thinking and effort parameters act on the messages, and on some models additionally on tools and system prompt.

A workspace with several monitors, one showing a code editor Photo: davidschultz / Unsplash

How much money is riding on this depends on the prefix size. In its Tool Search documentation, Anthropic cites around 55,000 tokens of tool definitions alone for a typical multi-MCP setup with GitHub, Slack, Sentry, Grafana and Splunk, before anything has even happened. Run against the list prices, a five-minute write on Claude Opus 5 costs 6.25 USD per million tokens, so roughly 0.34 USD for 55,000 tokens. Read from the cache at 0.50 USD per million it would be around 0.03 USD. That difference falls due on every unnecessary rebuild. On top comes the latency, which the user notices at exactly the moment the agent should be carrying on smoothly.

What the cache costs and what it saves

The multipliers on the base input price are manageable. A write with a five-minute TTL costs 1.25 times, a write with a one-hour TTL 2 times, every read and every refresh 0.1 times. For Claude Opus 5 that means 5 USD input, 6.25 USD for the five-minute write, 10 USD for the one-hour write, 0.50 USD for a read and 25 USD output, each per million tokens.

A few boundary conditions are worth knowing before you start debugging cache behaviour. A maximum of four cache breakpoints is allowed per request, and the lookback window per breakpoint covers 20 content blocks. And there is a minimum length. Claude Opus 5 caches from 512 tokens, whereas Opus 4.8 still needed 1,024. Fable 5 and Mythos 5 also sit at 512, Mythos Preview at 2,048, Opus 4.7 at 2,048, Opus 4.6 and 4.5 at 4,096, Sonnet 5, 4.6 and 4.5 at 1,024 each, and Haiku 4.5 at 4,096.

Stay below that and you get no error. The request runs, cache_control is silently ignored and the response contains nothing but zeros. Which is why it pays to look into the usage object before speculating about prices.

print(response.usage.cache_creation_input_tokens)  # newly written, 1.25x or 2x
print(response.usage.cache_read_input_tokens)      # read from cache, 0.1x
print(response.usage.input_tokens)                 # only the tokens AFTER the last breakpoint
# total_input_tokens = cache_read + cache_creation + input_tokens

The input_tokens value is the most common stumbling block here, because it does not mean the whole input but exclusively the tokens after the last breakpoint.

Several people working at monitors with open code in a bright open-plan office Photo: sigmund / Unsplash

The new route via tool_addition and tool_removal

The trick behind the beta feature is to leave the tools array alone. Visibility is controlled instead through content blocks inside a message with "role": "system". The hashed prefix stays byte-identical, so the cache keeps hitting.

curl https://api.anthropic.com/v1/messages \
  -H "content-type: application/json" \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "anthropic-beta: mid-conversation-tool-changes-2026-07-01" \
  -d '{
    "model": "claude-opus-5",
    "max_tokens": 1024,
    "tools": [ { "name": "get_weather", "description": "Get the current weather for a location.", "input_schema": { "type": "object", "properties": { "location": {"type": "string"} }, "required": ["location"] } } ],
    "messages": [
      { "role": "user", "content": "Say OK." },
      { "role": "system", "content": [ { "type": "tool_removal", "tool": {"type": "tool_reference", "name": "get_weather"} } ] }
    ]
  }'

Through the SDKs the same logic runs on the beta endpoint.

response = client.beta.messages.create(
    model="claude-opus-5",
    max_tokens=1024,
    betas=["mid-conversation-tool-changes-2026-07-01"],
    tools=[{
        "name": "get_weather",
        "description": "Get the current weather for a location.",
        "input_schema": {"type": "object", "properties": {"location": {"type": "string"}}, "required": ["location"]},
    }],
    messages=[
        {"role": "user", "content": "Say OK."},
        {"role": "system", "content": [
            {"type": "tool_removal", "tool": {"type": "tool_reference", "name": "get_weather"}},
        ]},
    ],
)

References go through {"type": "tool_reference", "name": "..."}. For tools from the MCP connector there are two dedicated forms, mcp_tool_reference with server_name and name for a single tool, and mcp_toolset_reference with server_name for an entire server.

{"type": "tool_addition", "tool": {"type": "mcp_tool_reference", "server_name": "github", "name": "create_pull_request"}}
{"type": "tool_removal", "tool": {"type": "mcp_toolset_reference", "server_name": "github"}}

The second case is the more interesting one. An entire MCP server can be shut down in a single block without anything in the request prefix changing.

The other direction needs some preparation. Every tool in the tools array is available from the start of the conversation unless it carries defer_loading: true. Then it stays held back until a tool_addition block unlocks it. The same block also reactivates a tool that was previously withdrawn via tool_removal.

{
  "name": "get_forecast",
  "description": "Get the 5-day forecast.",
  "input_schema": {"type": "object", "properties": {"city": {"type": "string"}}},
  "defer_loading": true
}

The rules that trip it up

The role: "system" message is not allowed everywhere. It must not be the first entry in messages. It has to follow immediately after a user turn, where a user turn with tool_result blocks counts, or after an assistant turn that ends with a server tool result. And it has to sit before an assistant turn or close out the array. Push it between a tool_use and its matching tool_result and you get a 400.

Several consecutive system messages are allowed and are treated as a single system section. What does not work is editing a mid-conversation system message after it has been sent. That invalidates the cache from that point on, which would defeat the purpose of the whole approach. These blocks are history, not configuration.

Source code of an API documentation page on a screen Photo: afgprogrammer / Unsplash

A name that is not declared in the tools array also returns 400. No declaration, no reference.

With the SDKs it is worth a quick look at your own version. The documentation shows typed examples for Go, Java and C#, while the Python and TypeScript examples pass the blocks as plain dicts and object literals. Anthropic’s bundled skill documentation adds that the typings lag behind and that you should fall back to plain dicts. Depending on the state of your SDK, check first rather than relying on clean types.

The sibling function without the beta header is mid-conversation system messages themselves. They are GA and run on Fable 5, Mythos 5, Opus 4.8 and Opus 5, explicitly not on Claude Sonnet 5. The same model list applies to the tool change blocks, available through the Claude API, Amazon Bedrock and Google Cloud.

If you are only loading tools in, Tool Search is enough

For the more common case the simpler solution has been around for a while. Tool Search is generally available on the Claude API with tool_search_tool_regex_20251119 and tool_search_tool_bm25_20251119 and needs no beta header. Deferred tools do not appear in the system prompt prefix, and tools that are found get attached inline into the conversation as tool_reference blocks. The prefix stays untouched.

tools=[
    {"type": "tool_search_tool_regex_20251119", "name": "tool_search_tool_regex"},
    {"name": "get_weather", "description": "...", "input_schema": {...}, "defer_loading": True},
]

Two sources of error are baked in. At least one tool has to keep defer_loading: false, otherwise you get a 400 telling you that not all tools may be deferred. And a tool with defer_loading: true must not carry cache_control, again a 400. The breakpoint belongs on a non-deferred tool. Further limits are 10,000 deferred tools per request, up to five hits per search, 200 characters for a regex pattern and 500 characters for a BM25 query.

The effect goes beyond money. Anthropic puts the reduction in definition overhead typically at over 85 per cent, because only the three to five tools actually needed get loaded. I find the other note from the same documentation more important, namely that tool selection accuracy drops once 30 to 50 tools are available. A tidy tool set is therefore not purely a cost topic but a quality question. Anyone building agents for code reviews or AI-assisted audits notices it in the hit rate before noticing it on the bill.

What this means for a setup with many MCP servers

The practical consequence is a division of labour. Loading tools in runs through Tool Search and defer_loading, without a beta header, on Sonnet 5 as well. Withdrawing runs through tool_removal with the beta header, and that is the only thing it is really worth having. Typical cases are phase changes in an agent, for instance when writing is done and only reading should follow, or an entire MCP server that has to come off the table once its work is finished.

Where the old rule of thumb continues to apply unchanged is the tools array itself. Touch names, descriptions or schemas there and you pay the full price. There is no documented way to change a tool definition mid-conversation without losing the cache. tool_addition and tool_removal switch visibility, nothing more. A changed schema has to go into the array, and that invalidates. So the rule is not dead, it has merely acquired a workaround for one particular case.

A half-open laptop in a dark room, its screen glowing orange and blue Photo: alesnesetril / Unsplash

Two more details that cause puzzlement on invoices. The pure tool use overhead in the system prompt is 286 tokens on Claude Opus 5 with tool_choice auto or none, and 406 tokens with any or tool. On Opus 4.7 it was still 675 and 804 respectively. And as soon as prompt caching is active at all, the API automatically sets an additional breakpoint after server tool results with a fixed five-minute TTL, visible as cache_creation.ephemeral_5m_input_tokens. That happens even when all of your own markers are set to one hour.

When the beta header pays off

It pays off when a long conversation with a large prefix is running and tools regularly need to disappear from it. It does not pay off when tools are only ever loaded in, because Tool Search is the cleaner route there without a beta flag. And it does not pay off on Sonnet 5, because that model is not included. What remains are exactly the setups already running on Fable 5, Mythos 5, Opus 4.8 or Opus 5 with long agent sessions on top.

If you run agents or MCP integrations and do not know how much cache rebuilding your own setup is carrying, cache_read_input_tokens and cache_creation_input_tokens will tell you within a few minutes. If you want a second opinion on an analysis like that, or on restructuring your tool architecture, feel free to get in touch.