> ## Documentation Index
> Fetch the complete documentation index at: https://docs.aihubmix.com/llms.txt
> Use this file to discover all available pages before exploring further.
# GPT Prompt Caching
> GPT prompt caching usage and billing: GPT-5.6 models gpt-5.6-sol / terra / luna bill cache writes at 1.25x input, reads at 0.1x; prompt_cache_key, breakpoints.
Prompt caching for GPT models (gpt-4o and later) is applied automatically: when the prompt prefix reaches 1,024 tokens and exactly matches a recent request, the matched portion is billed at the cached-input rate and time-to-first-token latency is reduced. The GPT-5.6 family (`gpt-5.6-sol` / `gpt-5.6-terra` / `gpt-5.6-luna`) upgrades the caching mechanism: cache writes are now billed separately (1.25x the input rate), cache reads are billed at 0.1x the input rate, caches are retained for at least 30 minutes, and the family adds `prompt_cache_key` for more reliable cache matching along with explicit cache breakpoint parameters.
Caching behavior across the two generations at a glance:
| | Before GPT-5.6 | GPT-5.6 and later |
| :------------------------- | :----------------------------------------------------- | :------------------------------------------------------------- |
| Caching mode | Automatic | Automatic + explicit breakpoints |
| Minimum cacheable length | 1,024 tokens | 1,024 tokens |
| Cache write billing | No additional fee | 1.25x base input rate |
| Cache read billing | Model's cached-input rate | 0.1x base input rate |
| Cache retention | Cleared after 5–10 minutes of inactivity, up to 1 hour | Retained for at least 30 minutes |
| `prompt_cache_key` | Optional, improves hit rates | Required by OpenAI to enable more reliable cache matching |
| 24-hour extended retention | Supported on some models (`prompt_cache_retention`) | Replaced by `prompt_cache_options.ttl`, currently only `"30m"` |
## Quick start
Prompt caching requires no extra configuration: send two consecutive requests with the same long prefix, and a `usage.prompt_tokens_details.cached_tokens` value greater than 0 in the second response indicates a cache hit. For GPT-5.6 models, also set `prompt_cache_key`:
```shell Curl theme={null}
curl https://aihubmix.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $AIHUBMIX_API_KEY" \
-d '{
"model": "gpt-5.6-sol",
"prompt_cache_key": "my-app-report-assistant-v1",
"messages": [
{
"role": "system",
"content": "You are a meticulous assistant for analyzing quarterly financial reports... [Place the static long instructions or reference material here, >=1024 tokens]"
},
{
"role": "user",
"content": "Summarize the key figures in one sentence."
}
]
}'
```
```py Python (OpenAI SDK) theme={null}
import os
from openai import OpenAI
client = OpenAI(
api_key=os.environ["AIHUBMIX_API_KEY"], # Read the key from an environment variable
base_url="https://aihubmix.com/v1",
)
long_context = "You are a meticulous assistant for analyzing quarterly financial reports... [Static long instructions or reference material, >=1024 tokens]"
# Send two consecutive requests with the same prefix; the second one hits the cache
for i in range(2):
completion = client.chat.completions.create(
model="gpt-5.6-sol",
prompt_cache_key="my-app-report-assistant-v1",
messages=[
{"role": "system", "content": long_context},
{"role": "user", "content": "Summarize the key figures in one sentence."},
],
)
print(completion.usage.prompt_tokens_details)
```
Measured usage from the two calls (2026-07-10, `gpt-5.6-sol`):
```json theme={null}
// 1st call: no cache hit
"prompt_tokens_details": {"audio_tokens": 0, "cached_tokens": 0}
// 2nd call: prefix hits the cache
"prompt_tokens_details": {"audio_tokens": 0, "cached_tokens": 2816}
```
## Cache pricing
Cache billing rules for the GPT-5.6 family:
| Billing item | Rate |
| :------------------- | :-------------------- |
| Regular input tokens | Platform pricing |
| Cache write tokens | 1.25x base input rate |
| Cache read tokens | 0.1x base input rate |
| Output tokens | Platform pricing |
OpenAI's official statement of this rule (from the [GPT-5.6 announcement](https://openai.com/index/gpt-5-6/)): "For GPT‑5.6 and later models, cache writes are billed at 1.25x the model's uncached input rate, while cache reads continue to receive the 90% cached-input discount." The [official prompt caching guide](https://developers.openai.com/api/docs/guides/prompt-caching) states the scope of these rates as "GPT-5.6 models and later model families". Official list prices for each model are on [OpenAI Pricing](https://developers.openai.com/api/docs/pricing); actual AIHubMix prices are listed in the [model gallery](https://aihubmix.com/models).
The break-even math follows directly from the official rates: writing a prefix to the cache costs 0.25x the input rate more than not caching, and every subsequent hit saves 0.9x the input rate. A prefix that is reused even once produces a net saving; the more reuse, the larger the saving. One-off requests whose prefix is never reused pay the write fee with no return, and can disable caching with explicit mode (see "GPT-5.6 caching parameters" below).
On models before GPT-5.6, cache writes have no additional fee, and cache reads are billed at the model's cached-input rate; see the [model gallery](https://aihubmix.com/models) for per-model prices.
The GPT-5.6 family has separate short- and long-context pricing tiers: when a single request's input exceeds 272K tokens, the entire request is billed at the long-context tier (2x input, 1.5x output). The 1.25x cache write and 0.1x cache read multipliers still apply in the long-context tier, based on the long-context input rate.
## How caching is applied automatically
When you send a request, the system checks whether the prompt prefix (in serialized order of messages, tools, and so on) exactly matches the prefix of a recent request:
1. If the prefix reaches 1,024 tokens and a matching cached prefix is found, the matched portion is billed at the cache read rate and time-to-first-token latency is reduced;
2. Otherwise the prefix is processed as regular input and written to the cache (billed at 1.25x on GPT-5.6 and later);
3. A cache hit requires the prefix to be byte-for-byte identical; any change within the prefix invalidates all cached content from that point onward.
Caching provides the most benefit in these scenarios:
* Static long system instructions or large sets of few-shot examples
* Long reference material repeatedly included in RAG workloads
* Agent workflows carrying large tool definitions (tools)
* Long multi-turn conversations that only append messages
Cache retention: on models before GPT-5.6, caches are cleared after 5–10 minutes of inactivity, up to 1 hour; on GPT-5.6 and later, caches are retained for at least 30 minutes and may persist longer. Caches are not shared across organizations, and caching has no effect on output content.
## GPT-5.6 caching parameters
The GPT-5.6 family adds three caching-related parameters (shared by the Chat Completions and Responses APIs):
| Parameter | Type / location | Values | Default |
| :------------------------ | :----------------------------- | :------------------------------------------------------------------------------------------------------------------------------ | :------------------------------- |
| `prompt_cache_key` | string, request body top level | A stable custom identifier, ideally scoped per workload or tenant; keep total traffic per key to roughly 15 requests per minute | None |
| `prompt_cache_options` | object, request body top level | `mode`: `"implicit"` / `"explicit"`; `ttl`: only `"30m"` supported | `mode: "implicit"`, `ttl: "30m"` |
| `prompt_cache_breakpoint` | object, inside a content block | `{"mode": "explicit"}`, marks the end of the cached prefix | No breakpoint |
Models before GPT-5.6 do not support `prompt_cache_options` or `prompt_cache_breakpoint`, and such requests are rejected; the older 24-hour extended retention parameter `prompt_cache_retention` (`"24h"` / `"in_memory"`) is replaced by `prompt_cache_options.ttl` on GPT-5.6 and later.
How the three caching controls relate:
1. **Default (implicit mode)**: caching happens automatically even with no caching parameters — the system places an automatic breakpoint at the latest message. On GPT-5.6 and later, these automatic cache writes are also billed at 1.25x.
2. **Implicit mode + explicit breakpoints**: in addition to the automatic breakpoint, you can set `prompt_cache_breakpoint` on a content block to pin the cache boundary at the end of stable content; changes after the breakpoint do not invalidate the cached prefix before it.
3. **Explicit mode**: with `prompt_cache_options.mode` set to `"explicit"`, only manual breakpoints are used; if no breakpoints are set at all, the request neither uses the cache nor incurs cache write charges. [Official wording](https://developers.openai.com/api/docs/guides/prompt-caching): "If the conversation contains no explicit breakpoints, the request does not use prompt caching or incur cache-write charges."
Use explicit mode to avoid cache write charges on a one-off long request:
```json theme={null}
{
"model": "gpt-5.6-sol",
"prompt_cache_options": {"mode": "explicit"},
"messages": [
{"role": "user", "content": "[One-off long content that will not be reused]"}
]
}
```
The official pattern for explicit breakpoints (breakpoint at the end of the static long content block):
```json theme={null}
{
"model": "gpt-5.6-sol",
"prompt_cache_key": "my-app-report-assistant-v1",
"messages": [
{
"role": "system",
"content": [
{
"type": "text",
"text": "[Static long instructions or reference material, >=1024 tokens]",
"prompt_cache_breakpoint": {"mode": "explicit"}
}
]
},
{"role": "user", "content": "Summarize the key figures in one sentence."}
]
}
```
Hard constraints (official):
* Each request can create at most 4 new cache writes; in implicit mode, the automatic breakpoint uses 1 of them;
* The prefix before a breakpoint must still reach 1,024 tokens to be cached;
* On reads, the longest matching prefix is selected from the 50 most recent breakpoints;
* Setting a breakpoint on an unsupported content block returns `400 invalid_request_error`. Chat Completions supports `text` / `image_url` / `input_audio` / `file` / `refusal` blocks; the Responses API supports `input_text` / `input_image` / `input_file` blocks.
AIHubMix support for `prompt_cache_breakpoint` content-block breakpoints and for Responses API cache hits is being finalized. For now, we recommend using automatic caching through Chat Completions with `prompt_cache_key` set (the quick start example on this page, verified to hit the cache); the explicit mode of `prompt_cache_options` works as expected for disabling caching. This page will be updated as support progresses.
## Why the cache is not being hit
A hit requires everything before the breakpoint position to be byte-for-byte identical. If `cached_tokens` is still 0 on the second request, work through this checklist:
* **Prefix under 1,024 tokens**: requests below the minimum cacheable length are processed as regular input;
* **Changing content mixed into the prefix**: timestamps, session IDs, per-user variables, and similar values should go after the static content; any change within the prefix invalidates all cached content after that point;
* **Tool definitions or ordering changed**: the tool list is part of the prefix; definitions and their order must match exactly;
* **Inconsistent image detail parameter**: `detail` affects how images are tokenized and must stay the same;
* **Structured output schema changed**: the `response_format` JSON Schema is part of the system message prefix for caching purposes; a schema change is a prefix change;
* **`reasoning_effort` changed**: officially listed as a common cause of lower cache hit rates ("Changes to reasoning effort");
* **Cache retention expired**: before GPT-5.6, caches are cleared after 5–10 minutes of inactivity; GPT-5.6 and later retain caches for at least 30 minutes;
* **`prompt_cache_key` not set (GPT-5.6)**: automatic hits are still possible without it, but the more reliable matching mechanism is not used.
## Best practices
* Put static content (system instructions, examples, reference material, tool definitions) at the very front of the request, and content that changes each turn at the end;
* Use one stable `prompt_cache_key` for traffic sharing the same prefix, keep total traffic per key to roughly 15 requests per minute, and split into more keys per workload when exceeding that;
* In multi-turn conversations, only append messages and avoid editing earlier messages;
* Keep steady traffic on requests with the same prefix to reduce cache eviction;
* For one-off long requests whose prefix will not be reused, use explicit mode to avoid cache write fees (GPT-5.6 and later);
* Monitor hits continuously via `usage.prompt_tokens_details.cached_tokens`.
## FAQ
### Does GPT prompt caching need to be enabled manually?
No manual setup is required: prefixes of 1,024 tokens or more are cached automatically. On GPT-5.6 and later, also set `prompt_cache_key` for more reliable cache matching.
### How are GPT-5.6 cache write fees calculated, and how do I avoid unnecessary write fees?
Cache writes are billed at 1.25x the base input rate and reads at 0.1x; a prefix reused even once produces a net saving. For one-off long requests whose prefix will not be reused, set `prompt_cache_options.mode` to `"explicit"` without setting any breakpoints, and the request neither uses the cache nor incurs write fees.
### How long is the cache retained?
GPT-5.6 and later retain caches for at least 30 minutes (`ttl` currently only supports `"30m"`; actual retention may be longer). Models before GPT-5.6 clear caches after 5–10 minutes of inactivity, up to 1 hour, and some older models support extended retention via `prompt_cache_retention: "24h"`.
### How do GPT-5.6 explicit breakpoints differ from Claude's cache\_control?
Both pin the cache boundary at the end of stable content. The main differences: GPT-5.6 caches automatically with no parameters required, and breakpoints are optional fine-grained control, while Claude requires caching to be enabled in the request (a top-level `cache_control` automatic breakpoint or content-block-level explicit breakpoints); GPT-5.6 retains caches for at least 30 minutes, while Claude defaults to 5 minutes with an optional 1 hour; cache reads on both are billed at 0.1x the input rate. For Claude usage, see [Claude Prompt Caching](/en/api/Claude-Cache).
### Does caching affect output content?
No. Official position: prompt caching only affects input-side processing and billing; the model generates output exactly as it would without caching.
## Official references
The mechanisms, rates, and parameter definitions on this page come from these official OpenAI sources:
* [GPT-5.6 announcement](https://openai.com/index/gpt-5-6/): the 1.25x cache write / 90% cache read discount billing rule
* [Prompt caching guide](https://developers.openai.com/api/docs/guides/prompt-caching): mechanism, parameters, usage fields, and limits
* [OpenAI Pricing](https://developers.openai.com/api/docs/pricing): official list prices per model
* [GPT-5.6 model documentation](https://platform.openai.com/docs/models/gpt-5.6-sol): context window and long-context billing threshold
Actual AIHubMix prices for each model are listed in the [model gallery](https://aihubmix.com/models).
***
Last updated: 2026-07-10