Monitor prompt caching to optimize your token usage
Prompt caching reuses identical prompt prefixes by storing their intermediate attention states on the model provider, reducing recomputation and cost on cache hits. Datadog’s 2026 AI Engineering report shows 69% of LLM input tokens are system prompts, making caching optimization critical for agentic systems. Anthropic supports automatic caching and explicit cache breakpoints (up to four per request); OpenAI automatically caches prompts ≥1,024 tokens with optional explicit breakpoints on GPT-5.6+. Cost economics: cache writes incur 1.25–2× premium depending on TTL, but cache reads cost ~90% less, so even a single hit within the write window breaks even. The article provides concrete guidance on cache breakpoint placement—keeping stable system instructions, tool definitions, and conversation history above the cache line while excluding dynamic user messages below it—with a production JSON example showing how to structure an incident-response agent for optimal cache utilization.