OpenAI adds GPT-6 prompt caching controls for persistent agents

GPT-6 caching update targets long-running agents

OpenAI has introduced an upgraded prompt caching system for the GPT-6 family, aimed at persistent agents that work for hours on tasks such as codebase refactoring, document research and presentation creation. The company says the system raises cache hit rates by default and offers discounts of up to 90% on cached input tokens.

The update makes eligible shared prompt prefixes reusable within a 30-minute window. This is relevant when an application sends a sequence of API requests carrying forward the same instructions, tool definitions and earlier context. Reusing that context avoids repeating computation, which OpenAI says reduces response times and input costs.

OpenAI is also adding controls and monitoring tools intended to help developers understand cache performance and decide which parts of a prompt should remain reusable.

Dashboard and diagnostics expose cache misses

The new Prompt Caching Dashboard shows the proportion of an application's input served from cache over time. Its input-composition view compares cached and uncached tokens, allowing teams to identify declines in hit rates and assess the effect of application changes.

For unexpected misses, OpenAI's diagnostics tool compares a request with a recent response and identifies differences in the model, tools, settings or input that stopped reuse. It also estimates the number of affected tokens. In OpenAI's example, a tools_changed miss affected 5,629 reusable tokens and missed the same number.

GitHub Copilot has used OpenAI prompt caching at scale. Chief Product Officer Mario Rodriguez said the service reduced by more than 50% the share of prompt tokens requiring fresh processing across billions of requests, compared with its prior baseline. He said the result was a more efficient inference stack and faster time to first response for developers.

Controls for context, tools and reasoning effort

Developers can use explicit cache breakpoints to select prompt prefixes for reuse. OpenAI recommends keeping tool definitions, schemas and their ordering stable as an agent's needs change. Rather than removing definitions, teams can use allowed_tools to limit callable tools or set tool_choice to none when tools are unnecessary.

New developer messages can append updated instructions near the end of the context, overriding older instructions while retaining the reusable prefix. GPT-6 also permits reasoning effort to change between responses without breaking cache when developers append a configuration_update and leave request-level reasoning effort unchanged.

OpenAI also offers cache prewarming for known shared material, including instructions, tool definitions and reference content. Processing that material during application startup can move work out of a user's waiting period when the first request arrives.

What the update means for application teams

OpenAI cited reported production gains from explicit cache breakpoints. Manus said its cache hit rate rose from roughly 85% to consistently above 90% in less than a week. Another customer, Wordsmith, said evaluation hit rates moved from 83% to 91%, while cache writes fell by roughly two-thirds and inference costs declined by 36%.

For businesses operating GPT-6 agents, the practical implication is to measure cache behaviour, preserve stable prompt structure, and isolate frequently changing instructions or content at the end of the request so reusable context can be retained.