In agentic and multi-turn applications, our models work best when thinking is preserved across API requests, where new actions and instructions are always appended to the model’s context. Keeping the prefix intact has long been a best practice for prompt caching. The cache matches requests from the beginning, so a change to the prefix can cut cache reuse from that point onward. Appending changes keeps more of the conversation eligible for cache reuse, which can substantially lower input costs and speed up responses.
Starting with newer frontier models, the API will drop reasoning items that follow a prefix change as part of our anti-distillation safeguards. When reasoning items are dropped, the model may have to redo some reasoning, which can increase token usage, cost, and latency. If the model can’t recover the lost reasoning, output quality may also drop.
To get the best performance from our models, we recommend avoiding changes to the prefix and adopting an append-only integration pattern as far as possible.
What is a prefix change?
A reasoning item’s prefix is the conversation history that comes before it. A prefix change modifies that earlier history—for example, by shortening a tool result, rewriting a developer message, or replacing older turns with a summary.
| Original conversation | Next request and API handling |
|---|---|
| User messageCheck the ledger total. | Same message |
| Reasoning before the tool call | Reusable |
Tool calllookup_ledger() | Same call |
Tool result[{"amount": 40}, {"amount": 50}] | Same rows |
| Reasoning after the tool result | Reusable |
How to update integrations to avoid prefix changes
You can change the prefix unintentionally by updating Responses API parameters such as tools, instructions, or text.format, or by editing earlier input items. Audit your integration for these changes.
The table below shows common ways integrations change the prefix and how to achieve the same goals by appending to the context where possible.
| Common reason to change prefix | How to avoid prefix change |
|---|---|
| Changing reasoning effort | Instead of changing the reasoning.effort parameter in the Responses API, append a configuration_update item with the new reasoning effort to the input items list. Keep the original request-level setting the same. |
| Changing structured output schemas | Instead of changing the text.format parameter in the Responses API, append a configuration_update item with the new schema to the input items list. Keep the original request-level setting the same. |
| Changing the tools available to the agent | Instead of changing the tools parameter in the Responses API, use an additional_tools item to add tools, or tool_choice with allowed_tools to restrict which tools the model can call on a turn. Keep existing tool definitions in order. Append additional_tools with the developer role and a tools array, and replay it in its original position. |
| Updating developer instructions | Instead of editing the original developer message, append a new developer message asking the model to ignore older instructions and follow the new ones you provide. For instructions scoped to a single turn, place an explicit cache breakpoint immediately before the latest developer message so the reusable prefix ends before that message. |
| Compaction | Compaction changes the prefix, so cache misses are difficult to avoid in these cases. OpenAI compaction: If you use OpenAI compaction via /responses/compact or context_management, our systems retain subsequent reasoning items despite the prefix being changed. An OpenAI compaction item can appear anywhere in context. When calling /responses/compact, omit instructions and pass the returned window to the next request in its original order. If your history contains configuration_update items, follow the configuration-update compaction requirements.Custom compaction: If you use a different or home-grown compaction strategy, place the compacted item at the end of the model’s context, instead of before recent turns. Alternatively, you can accept the trade-off and have the reasoning items from the original history that follow it dropped. |
| Shortening past tool outputs or downsizing past files / images | Some integrations truncate past items, such as tool outputs, files, or images, instead of using compaction. These modifications change the prefix. We strongly recommend either using compaction, or shortening long tool outputs and downsizing images or files before they’re sent to the model for the first time. |
Auditing your integration for prefix changes
Use Prompt Cache Diagnostics to see when cache misses occur and investigate prefix changes. Compare a request with an earlier response by setting prompt_cache_options.comparison_response_id to the earlier response ID, then inspect the diagnostic reason and cached token count. Use this information to reduce prefix changes in your integration. Diagnostics report the first identified difference on a best-effort basis; they can’t verify whether the model reused reasoning.