Prompt Caching Mistakes Can Cost Your Organization Nearly 6x More on Every AI Turn

I have since come to understand part of the reason. A person can return to a conversation the next day, and whatever time the conversation was given at the start has gone stale. Something has to supply it again. A harness, the program around a model that assembles each request and runs the loop, can do that by writing the current time into the model's instructions before every request. The agent runner we use in production did exactly that.

I have known about prompt caching for a long time. Providers store the start of a request and bill it at a fraction of the price when the next request begins the same way. I did not know its mechanics well enough to see what a line like the time does to it. We stumbled onto the answer in our own production costs, and I directed a set of experiments to measure it properly, across one current model from each of 4 labs and against OpenRouter's public figures for what everyone else is paying.

The answer is expensive. On GPT-6 Luna, the time written in the wrong place made every turn of a conversation cost 5.7× what it needed to, and nothing raised an error. Across the traffic organizations send these models, input is 96–99% of the tokens and 66–87% of the dollars actually paid, so the cache decides most of the bill. For an organization building its own harness, running an internal AI platform or shipping AI products, a mistake like this one multiplies the cost of every turn, silently, for as long as it runs.

Every per-turn number here was measured on September 23, 2026, through OpenRouter. The market figures are OpenRouter's own, read the same day.

Every call to a model sends the whole conversation again: the tool definitions, the system prompt (the standing instructions the harness writes), every earlier turn, and the new message.

Providers keep the processed beginning of recent requests. When the next request starts with the same text, the provider reads that stored prefix instead of processing it again. On GPT-6 Luna, OpenRouter billed a read at 0.10× the listed input price and the first write at 1.25×. A prompt that is written again on every turn therefore costs more than a prompt with no cache at all.

Only the start of a request can be reused. A change at any point voids everything after it:

graph LR
    A[Tool definitions] --> B[System prompt]
    B --> C[Earlier turns]
    C --> D[New user message]
    B -. a changed line here voids B, C and D .-> D

Anything a harness writes near the start and changes on every turn puts the whole conversation behind it at risk. The current time is the plainest example.

The discovery started in an agent we run in production. Its cost ledger priced every call at the list price and showed no caching at all. I raised the alarm and started weighing a switch to a different model.

The agent I was working with found first that the ledger itself was wrong. Over 33 calls it estimated $0.064; the provider had billed $0.023, 2.7× less. Pricing each call from token counts had erased every discount the provider applied. Reading the provider-reported cost instead showed caching working inside each turn and failing at the start of every new one.

Its first explanation was a missing session key that would keep requests on the same server. Its own probe refuted that: identical prompts already read from cache across turns on GPT-6 Luna, and adding a session key, a cache key or an explicit cache marker changed nothing. The cause sat in the system prompt itself. About 95% of the way in, the runner wrote two lines that changed every turn: a per-turn working directory and a clock to the minute. Each first call of a turn wrote the whole prompt again at 1.25×.

Moving both lines to the end of the user's message fixed it in production. Turns 2 and 3 now open at 0.47–0.51× instead of 1.25×. The remainder is a block of per-turn context the runner still sends in the user message.

My guess, before any of this was measured, was that a deterministic check could supply the time only when it had gone stale, as a separate message, and leave the rest of the prompt cached. It holds, with one condition, and that condition is the rule the rest of the article turns on.

To test it directly, we built a probe that runs the same conversation with the clock in different places, turns 65 seconds apart so the minute changes between every pair. GPT-6 Luna, an 8.5K-token system prompt, 3 replicates:

My separate message kept the cache in both forms, every turn and only when stale. The condition is where the message sits. After the stable instructions, the provider reads everything before it. As a second system message it sits among the instructions, and GPT-6 Luna wrote the whole prompt again on 12 of 12 turns.

The rule that follows is short. Anything that changes between turns goes after everything that does not.

The cache compares text, so a clock is harmless for as long as its text stays the same. In the same probe with turns 4 seconds apart, a minute clock in the system prompt read from cache every time. With turns 65 seconds apart, it forced a full rewrite on every turn.

Resolution sets how often that happens. An hour clock and a date line in the system prompt both read from cache on all 12 turn openings of the 65-second runs, because neither rolled over. They break too, once an hour and once a day. A minute clock breaks on every turn that starts in a later minute than the one before it.

The cache itself also expires. In single timed runs, GPT-6 Luna still read its stored prefix after 30 minutes idle and wrote the whole prompt again at 1.25× after 60. DeepSeek V4.1 Flash read after 10 minutes and missed after 60. Claude Opus 5.5 had already missed after 10, since Anthropic's default cache lasts 5 minutes and its 1-hour cache costs 2× to write. The person who returns to a conversation the next day, the case this article began with, finds both a stale time and a cold cache on all three. That first turn back pays for the whole prompt whatever the harness does. Placement decides whether the turns after it do too.

GPT-6 Luna is one provider's answer. To see how general the lesson is, we ran the same placements on one current model from each of 4 labs, with an identical 2.5K-token prompt, each pinned to a single provider:

GPT-6 Luna caches whole messages. One changed line voids the message that holds it and everything after.

Claude Opus 5.5 caches where the caller marks it. Anthropic's models cache only up to a cache_control marker the harness places. With no marker, the same prompt billed full price on every call. With the marker on the system prompt, a clock inside that block cost 1.24×, while a clock in a second system message after it read at 0.08×. The placement that GPT-6 Luna punishes is safe on Opus. At Opus prices the difference per turn opening was $0.0214 against $0.0023. Opus also counted the identical prompt as 4,056 tokens where GPT-6 Luna counted 2,395, so the same text costs 1.69× more tokens before any price applies.

DeepSeek V4.1 Flash caches in 256-token blocks and charges nothing extra to write. Reads came back in exact multiples of 256: 2,560 tokens for the unchanged prompt, 2,304 with the clock at the end of the system prompt. A changed line costs only the block that holds it. First calls billed at the plain input price, and reads at 0.03–0.04×. Across 8 runs per placement, 2 single calls missed the cache on a turn that the same run then read, which reads as the request landing on a different server rather than as an effect of placement. DeepSeek's own endpoint is the only one OpenRouter marks as caching automatically, and my account's privacy setting excludes it because that endpoint trains on paid traffic. The runs went through DeepInfra, which cached, as did Fireworks and Morph in a 2-call check.

Muse Spark 1.3 reads its cache inside a tool loop and missed at the start of each new turn. Our first probes, which ask a fresh question against the same system prompt, served at most 113 tokens from cache in 10 calls, at 2.9K, 8.9K and 19.4K tokens. When a follow-up call extended the previous request, as an agent's tool loop does, Muse read 2,801 of 2,966 tokens and billed 0.17×. The next user turn, with the whole history in front of it, read nothing. OpenRouter reports an 86.1% cache hit rate for Muse on live traffic, which fits a workload dominated by tool loops. At the start of a turn the clock makes no difference on Muse, since that call misses either way.

The same harness decision costs a different amount on each model. Knowing which mechanism your provider uses comes before tuning anything.

We expected cached turns to answer faster. On 10 sequential pairs of GPT-6 Luna calls at 8.5K tokens, the median first call took 437 ms from cache and 490 ms without it. That gap sits within the spread of the samples. At this size, caching changes what a turn costs and leaves how long it takes about the same.

So the pause I remember when asking for the time has some other cause. What a clock in the wrong place costs is money.

A clock in the system prompt sits ahead of the entire conversation, so each rewrite covers the growing history behind it as well as the instructions. Here is the measured cost of one 12-turn GPT-6 Luna conversation, with the full provider-reported bill, including output and the calls within each turn:

The two lines share their first turn, when both write the cache. From then on, every turn with the clock in the system prompt cost 5.7× what the same turn cost with the clock at the end of the user message. By turn 12 the conversation had cost 4.1× as much, and the gap widens with every turn.

Fractions of a cent become a line item at scale. This projection prices the first call of each turn for a product serving 10,000 conversations a day, each with an 8.5K-token system prompt, 20 turns, and history growing 1,500 tokens a turn. Each model uses its listed price and the cache behaviour it showed above:

DeepSeek's blocks keep the system prompt when the clock changes, and the model still comes out 13× more expensive, because the history behind the clock outgrows the instructions within a few turns. Muse Spark shows the other side: it missed at the start of every turn in our runs, so placement saved nothing there and every turn opening paid full price, $2.1M a year for the same traffic.

Our measurements use a controlled prompt. OpenRouter publishes what the same models cost on everyone's traffic, and the same mechanism shows up there at scale. Almost everything sent to these models is input: 96.3% of GPT-6 Luna's tokens on its first day, 98.5% of Claude Opus 5.5's and DeepSeek V4.1 Flash's, and 98.9% of Muse Spark 1.3's. Input is where caching applies, so the hit rate sets most of what a business pays.

Customers paid $0.87 per million input tokens for Claude Opus 5.5 against a listed $4.00, $0.30 against $1.25 for Muse Spark 1.3, and $0.036 against $0.10 for GPT-6 Luna. Across endpoints serving the same model on the same day, the price paid follows the hit rate:

Pricing every miss as a cache write and every hit as a read predicts each of the 5 main endpoints' posted prices to within 2–13%. The hit rate alone explains the spread. On GPT-6 Luna the same pattern runs from $0.025 at an 88.1% hit rate to $0.075 at 49.4%, a factor of 3.

The same placement decision lands differently on each kind of organization that builds on these APIs.

Harness builders set the hit rate for every user at once. The 5 apps that sent Claude Opus 5.5 the most traffic on its first day were all agents, between 2.9B and 16.3B tokens each. For a harness at 10B input tokens a day on that model, each point of cache hit rate is worth $175K a year. The gap between the 92.2% and 77.4% endpoints above comes to $2.07M a year at that volume. A clock in the system prompt that rewrites every turn opening costs between $1.75M and $8.76M a year, depending on whether turn openings are 1 call in 10 or 1 in 2. The decision also travels: every product built on a harness inherits its placement. While writing this we found the same system-prompt clock in a second harness of ours, which runs an older build of the same runner.

Companies running an internal AI platform set it for every team behind them. A gateway that stamps each request's system prompt with a timestamp or request ID for auditing turns every application's turn openings into writes, whatever the teams build on top. Accounting is the second exposure. Charged back at list price, input costs look 2.8× larger than the bill on GPT-6 Luna, 4.1× on Muse Spark 1.3 and 4.6× on Claude Opus 5.5. Our own ledger overstated 33 calls by 2.7×, and I came close to switching models on the strength of it.

Companies shipping AI products pay it on every conversation. Inside one conversation, the misplaced clock cost 5.7× per turn. Across a market, the stakes run larger. On September 21, Muse Spark 1.3 took 88.5B input tokens through OpenRouter. At list price that is $110.6K; customers paid $26.9K, and every point of hit rate on that traffic is worth $355K a year. DeepSeek V4.1 Flash took 2.68T input tokens the same day, and at DeepInfra's prices each point is worth $1.33M a year. Those figures cover one router. Traffic sent to the providers directly does not appear in them.

Every one of those costs traces back to the same mechanism, and so does the fix. The conventions of well-built agent harnesses read as its consequences:

The fix above is a single edit. The conditions around it keep moving. A harness gains a new hook, a gateway starts stamping requests with an ID, a team moves to a model with a different cache mechanism, a provider changes how long an idle prefix stays warm. The four models here cache in four different ways, and Claude Opus 5.5 let a cache go cold within 10 minutes of idle time. The second harness where we found the same clock had simply stayed on an older build. None of this raises an error.

Every call already returns what it takes to see it: prompt tokens, cached tokens, cache-write tokens and the billed cost. Recorded per call, together with whether the call opened a turn or continued one, those fields give three numbers worth watching for every route and model:

A threshold on those numbers catches a sudden drop. Naming the cause across weeks of data is a classification problem, and a newer class of model suits it. Jev, from TypeSafe, is what its maker calls a System One model. It does not generate text. It takes a state and typed questions and returns typed answers with a probability distribution and a confidence: a choice among options, a probability of yes, or a position on an ordered scale. Given a route's daily cache profile, it can name the pattern the day matches (healthy, turn openings rewriting, cache expiring between turns, traffic reaching an endpoint that does not cache) and say how sure it is, so a change of category becomes the alert. We use it in production to classify documents and, in observation, to score finished agent episodes. OpenRouter lists it at $0.042 per million input tokens with no charge for output. At that price, classifying a 2K-token daily profile for each of 1,000 routes costs about $31 a year, against $175K a year for a single point of hit rate at the harness scale above.

A system that keeps rewriting its cache pays the premium on every turn of every conversation, for as long as it runs, and the bill rarely says why. The fix can be as small as where the time is written. Keeping it fixed takes a number someone watches.

Every measured figure above comes from OpenRouter's own per-call fields (cached tokens, cache-write tokens and billed cost) on September 23, 2026. The market figures come from OpenRouter's public model pages read that day: the weighted input price actually paid, each endpoint's effective price and cache hit rate, and one day of token activity per model. Claude Opus 5.5 and GPT-6 Luna launched on September 22, so their activity figures cover a partial first day, and the article uses them only as shares. The experiments cost $0.63 in total, and the probe refused to start once the account's usage approached a $1 cap. Prices are OpenRouter's listed prices read the same day. OpenRouter's caching guide lists OpenAI cache reads at 0.25–0.50×; GPT-6 Luna billed its reads at 0.10×, and this article reports what was billed.

Still open: the exact point at which each model's idle cache expires, which single timed runs only bracket; how often an hour or date clock breaks over real conversation gaps; why Muse Spark missed at the start of each turn when its history was unchanged; and DeepSeek's behaviour on its own endpoint. The per-turn fix is live in our production agent, and the same finding has been queued for a second harness that runs an older build of the same runner.

What I brought. The question behind the article: why AI never seemed to know the date and time well, and took seconds to say it when it should be instant. The decision to measure it: I knew of prompt caching but not its mechanics, and I directed the experiments. The idea that the time goes stale across a session and has to be supplied again, and my guess that a deterministic, only-when-stale message would leave the prompt cached. The direction of the piece: the cost opens and closes it, quantified for the businesses that build on these models, from harness builders to companies running their own internal AI platforms, as it stands today and as it compounds. The close on ongoing observation, with a System One model such as Jev classifying the cache profile over time, because the fix holds only as long as someone keeps watching.

What the AI supplied. It found that our cost ledger hid the caching, 2.7× on 33 calls. It proposed a missing session key as the cause and refuted that with its own probe. It found GPT-6 Luna's whole-message behaviour, built the runner fix that is now in production, and built the probe, the analysis and the cost model behind every number here, and matched the mechanism against OpenRouter's public market figures. Across the four models it identified three different cache behaviours and a fourth model with no usable cache.