When teams try to cut their LLM bill, the first move is almost always the same: switch to a cheaper model. That's a real lever, and we covered the size of that gap across every major vendor this week. But there's a second lever that doesn't cost you any model quality at all, and most teams simply aren't using it: prompt caching.
The Problem It Solves
Every request to an LLM re-sends the entire context — system prompt, tool definitions, conversation history, any documents in play — even when most of that content is identical to the request before it. An agent that calls three tools to answer one question sends that same unchanged system prompt and tool list four times in a row. A chatbot with a long, carefully engineered system prompt pays full price for it on every single message a user sends, all day, from every user. None of that repetition is doing new work — it's the model re-reading the same boilerplate it already processed seconds ago.
What Caching Actually Charges You
Anthropic's published pricing makes the mechanics concrete. Marking a prompt with a cache breakpoint creates three distinct charges instead of one flat input rate:
| Cache operation | Price vs base input | When it applies |
|---|---|---|
| 5-minute cache write | 1.25x | First time content is cached |
| 1-hour cache write | 2x | Longer-lived cache for slower-moving sessions |
| Cache read (hit) | 0.1x | Every subsequent request that reuses it |
That 0.1x figure is the headline: a cache hit costs 90% less than sending the same content fresh. OpenAI's own pricing shows the identical shape from a different angle — GPT-5.5's cached input rate is $0.50/M against a $5.00/M base rate, which is exactly a 90% discount on the reused portion of the prompt. This isn't a rounding-error optimisation; it's the single largest per-request saving available without changing which model you use.
The break-even is fast, too. A 5-minute cache write costs 1.25x once — so it pays for itself after a single cache read (0.1x) on top of the write, and every read after that is close to pure savings. For a multi-step agent loop hitting the same system prompt and tool list 4, 10, or 30 times in one turn, that first write is a rounding error against the reads that follow.
Why Agents Feel This More Than Chatbots Do
This compounds hardest exactly where token spend is already highest: tool-calling agents. We covered the tool-calling loop the way most production agent platforms run it — an agent doesn't make one call per user turn, it makes one call per step, resending the full tool catalogue and system context at every step until it has an answer. That's not a handful of extra tokens; on a connector-heavy setup with a long tool list, the tool definitions alone can dwarf the actual user question in every single request. Cache that block once, and every step after the first pays 10% of what it would have.
The Catch: It Only Pays Off on What Actually Repeats
Caching isn't free complexity-wise. The cached portion has to be an exact prefix match — content, ordering, and all — or the cache misses and you're back to full price plus the write overhead for nothing. That means anything volatile (today's date interpolated into the system prompt, a per-request timestamp, content that changes every turn) has to sit after the cache breakpoint, not mixed through it, or it silently defeats the whole mechanism. Getting the structure right is a one-time engineering cost; getting it wrong means paying the 1.25x write premium on every request and never collecting the 0.1x read discount that's supposed to offset it.
How ApiSpi Implements It
We shipped prompt caching as an opt-in control on the ApiSpi platform rather than switching it on silently for everyone. It lives under Profile → Governance → Sovereignty, off by default: cached content briefly resides on the provider's cache infrastructure for the TTL window, which for governance-conscious organisations is a data-handling decision worth making deliberately rather than inheriting by default. Turn it on, and Aria's system prompt — built from your governance policy, personalisation, deployed agents, and connector tool list — along with your active connectors' tool definitions are marked as a single cache breakpoint on every Anthropic request. The gain applies automatically across the dashboard chat, the LLM Gateway, and scheduled prompts, since all three run through the same underlying request engine.
It's a genuinely rare kind of lever in this market: no quality trade-off, no model downgrade, no prompt rewrite — just a large, structural discount on the part of every request that was never new information in the first place.