Cache diagnostics: finding exactly why your prompt cache missed
TL;DR
Cache diagnostics compares consecutive requests and names the first divergence behind a cache miss: model, system prompt, tools or messages. Enable it on every turn, fix per the reported type, and remember the usual culprit: timestamps in system prompts.
You've turned on prompt caching, the bill is still high, and cache_read_input_tokens keeps dropping to zero for no clear reason. It could be a timestamp in the system prompt, a reordered tool, or edited conversation history. Until now, all you could do was guess. The Claude API's cache diagnostics compares two consecutive requests and tells you exactly where they diverged, so you fix the root cause.
How to enable it
- Add a
diagnosticsobject on every turn. First turn:{"previous_message_id": null}. Later turns: pass the previous response'sid. - No beta header is needed any more (the old header still works).
- When streaming, the result arrives on the
message_startevent.
r2 = client.beta.messages.create(
model="claude-opus-5-5", max_tokens=1024,
cache_control={"type": "ephemeral"}, system=SYSTEM,
messages=[...],
diagnostics={"previous_message_id": r1.id},
)
if r2.diagnostics and r2.diagnostics.cache_miss_reason:
print(r2.diagnostics.cache_miss_reason.type)Reading the result
cache_miss_reason.type | Meaning | Fix |
|---|---|---|
model_changed | The model differs from last time (a router, A/B test, fallback) | Hold the model constant within a cached conversation |
system_changed | The system prompt differs, usually a timestamp or request ID | Keep the system prompt byte-stable; move dynamic data into a user message after the breakpoint |
tools_changed | Tools added, removed or reordered, or schemas serialised non-deterministically | Same tools, same order, schemas with sorted keys |
messages_changed | An earlier message was edited, removed or reordered rather than appended to | Treat history as append-only; echo assistant content and tool results back verbatim |
previous_message_not_found | No fingerprint for that ID (the previous request didn't opt in, another workspace, too long ago) | Include diagnostics on every turn and keep turns close together |
unavailable | No information, e.g. another parameter such as tool_choice or thinking changed | Keep prompt-affecting parameters constant |
The API reports the first divergence only; fix it first, since later ones may be hidden behind it. The *_changed types include cache_missed_input_tokens, an estimate of how many tokens lost the cache.
Combining it with usage
| Diagnostics | Cache reads | Interpretation |
|---|---|---|
null | High | Fine, the prefix is stable |
null | Low/zero | Requests match but the cache expired: shorten gaps or use the 1-hour TTL |
A *_changed type | Low/zero | The request changed; fix per the type |
Limitations
- Only on the direct Claude API, not on Bedrock or Google Cloud (Google managed).
- Fingerprints are kept briefly, scoped to the same organisation and workspace, and contain only hashes and token estimates, never prompt content.
- Very long conversations with a change deep in history may return
unavailable. - It never fails your request; it's purely extra information.
How to apply it
- Dev: Turn diagnostics on in staging and log
cache_miss_reasoneach turn to find cache-breaking bugs. - Tech Lead: Alert when the
*_changedrate rises after each prompt deployment. - PM: When API costs jump unexpectedly, this is the first tool to reach for before switching models.
Sources & references
- Cache diagnostics — Claude Docs