If a reusable prefix expires after a few idle minutes, should the harness keep sending requests to preserve it?
That sounds like a scheduling problem. It is also a matching, accounting and product-policy problem. Before adding a timer, I want to know which entry a request can refresh and whether keeping it useful costs less than recreating it.
What has a lifetime?
Active state needed to generate a response and retained state available to a later request have different jobs. A long-running response can still be using KV while a separate cross-request retention clock advances.
A session in our application also has its own lifetime. Saving the transcript in PostgreSQL allows us to reconstruct an input. It does not make the provider retain the corresponding numerical state indefinitely.
The clock in the figure is a deliberately simple five-minute policy measured from a successful write/read request start. Other policies need their own model of time.
Which request refreshes which prefix?
Illustrative five-minute policy. Initial write at t=0; optional matching warmer at t=180s.
At a four-minute generation time, a follow-up thirty seconds later starts at 270 seconds. A ninety-second gap starts at 330 seconds. Under the illustrated policy, the latter misses the original five-minute window. Waiting until exactly expiry leaves no scheduling margin.
Does any traffic help?
Only traffic that actually accesses the eligible cached prefix can refresh that entry. Ten requests with different initial instructions do not keep an eleventh request’s prefix warm merely because they use the same application.
Several sessions can share an identical beginning while having different suffixes. This makes common tool definitions and instructions interesting: traffic from one session may benefit another within the provider’s permitted scope. It does not establish retention of each session’s private conversation.
Stable serialization, model/settings, cache lookup boundaries, isolation and retention all matter. The prefix comparison article explains how to inspect those inputs before interpreting a read count.
What do providers expose?
As checked in November 2026, Claude’s documentation describes five-minute and one-hour caching, automatic or explicit breakpoints, and refresh from request start. It also documents max_tokens: 0 pre-warming with restrictions. A warm-up must target the real shared prefix and matching configuration; adding a changing placeholder to the cached span can defeat the purpose. Claude’s pre-warming guide
OpenAI’s retention and breakpoint options depend on model/API generation. Its documentation also distinguishes cache-read pricing eligibility from guarantees about physical KV reuse or lower latency. Do not apply a universal five-minute policy across those products. OpenAI caching
These are contract descriptions. This project has not yet run a controlled personal-provider warming experiment.
A warmer is another operation
A normal agent turn changes the conversation. A background cache operation should not append invented user messages or fake assistant answers to that session.
I would give it its own operation identity and record the intended prefix/configuration, time, usage, result and cost. It should be observable, cancellable and bounded. A deployment restart should not accidentally create several independent warmers for the same entry.
The existing normal provider path expects a usable response. An output-free operation therefore needs its own handling rather than assuming the normal turn code accepts empty output. Endpoint validation errors should remain visible instead of falling back to an unexpectedly expensive generation.
What would make warming worthwhile?
Let W be all warm-up charges during an idle interval. Let C be the extra charge of the eventual cold continuation relative to its warm alternative. If the continuation happens, warming saves billing only when W is less than C. If it never happens, W was spent without that benefit.
That simple comparison omits latency value. A product may reasonably pay more to avoid a cold first interaction. State that objective directly rather than presenting the policy as an automatic cost saving.
I would compare immediate continuation, several idle gaps, supported warm-up requests and longer retention. Hold the serialized prefix, model, configuration and workload fixed. Record total charges, first-content latency, actual usage categories and failures. Include every warm-up and retry.
The decision belongs to the workload
An active tool loop is likely to reuse its recent output soon. An abandoned interactive session may never return. Shared stable instructions can have a different reuse pattern from a large private suffix. One timer for every session ignores those differences.
The useful question is not whether we can send a request every few minutes. It is whether a specific retention policy improves the cost or latency objective for the traffic we actually have. Until the controlled run exists, the figure is a teaching simulation and the policy remains a hypothesis to test.