Skip to main content
Prompt caching for GPT models (gpt-4o and later) is applied automatically: when the prompt prefix reaches 1,024 tokens and exactly matches a recent request, the matched portion is billed at the cached-input rate and time-to-first-token latency is reduced. The GPT-5.6 family (gpt-5.6-sol / gpt-5.6-terra / gpt-5.6-luna) upgrades the caching mechanism: cache writes are now billed separately (1.25x the input rate), cache reads are billed at 0.1x the input rate, caches are retained for at least 30 minutes, and the family adds prompt_cache_key for more reliable cache matching along with explicit cache breakpoint parameters. Caching behavior across the two generations at a glance:
Before GPT-5.6GPT-5.6 and later
Caching modeAutomaticAutomatic + explicit breakpoints
Minimum cacheable length1,024 tokens1,024 tokens
Cache write billingNo additional fee1.25x base input rate
Cache read billingModel’s cached-input rate0.1x base input rate
Cache retentionCleared after 5–10 minutes of inactivity, up to 1 hourRetained for at least 30 minutes
prompt_cache_keyOptional, improves hit ratesRequired by OpenAI to enable more reliable cache matching
24-hour extended retentionSupported on some models (prompt_cache_retention)Replaced by prompt_cache_options.ttl, currently only "30m"

​
Quick start

Prompt caching requires no extra configuration: send two consecutive requests with the same long prefix, and a usage.prompt_tokens_details.cached_tokens value greater than 0 in the second response indicates a cache hit. For GPT-5.6 models, also set prompt_cache_key: 
curl https://aihubmix.com/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AIHUBMIX_API_KEY" \
  -d '{
    "model": "gpt-5.6-sol",
    "prompt_cache_key": "my-app-report-assistant-v1",
    "messages": [
      {
        "role": "system",
        "content": "You are a meticulous assistant for analyzing quarterly financial reports... [Place the static long instructions or reference material here, >=1024 tokens]"
      },
      {
        "role": "user",
        "content": "Summarize the key figures in one sentence."
      }
    ]
  }'

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AIHUBMIX_API_KEY"],  # Read the key from an environment variable
    base_url="https://aihubmix.com/v1",
)

long_context = "You are a meticulous assistant for analyzing quarterly financial reports... [Static long instructions or reference material, >=1024 tokens]"

# Send two consecutive requests with the same prefix; the second one hits the cache
for i in range(2):
    completion = client.chat.completions.create(
        model="gpt-5.6-sol",
        prompt_cache_key="my-app-report-assistant-v1",
        messages=[
            {"role": "system", "content": long_context},
            {"role": "user", "content": "Summarize the key figures in one sentence."},
        ],
    )
    print(completion.usage.prompt_tokens_details)
Measured usage from the two calls (2026-07-10, gpt-5.6-sol): 
// 1st call: no cache hit
"prompt_tokens_details": {"audio_tokens": 0, "cached_tokens": 0}

// 2nd call: prefix hits the cache
"prompt_tokens_details": {"audio_tokens": 0, "cached_tokens": 2816}

​
Cache pricing

Cache billing rules for the GPT-5.6 family:
Billing itemRate
Regular input tokensPlatform pricing
Cache write tokens1.25x base input rate
Cache read tokens0.1x base input rate
Output tokensPlatform pricing
OpenAI’s official statement of this rule (from the GPT-5.6 announcement): “For GPT‑5.6 and later models, cache writes are billed at 1.25x the model’s uncached input rate, while cache reads continue to receive the 90% cached-input discount.” The official prompt caching guide states the scope of these rates as “GPT-5.6 models and later model families”. Official list prices for each model are on OpenAI Pricing; actual AIHubMix prices are listed in the model gallery. The break-even math follows directly from the official rates: writing a prefix to the cache costs 0.25x the input rate more than not caching, and every subsequent hit saves 0.9x the input rate. A prefix that is reused even once produces a net saving; the more reuse, the larger the saving. One-off requests whose prefix is never reused pay the write fee with no return, and can disable caching with explicit mode (see “GPT-5.6 caching parameters” below). On models before GPT-5.6, cache writes have no additional fee, and cache reads are billed at the model’s cached-input rate; see the model gallery for per-model prices.
The GPT-5.6 family has separate short- and long-context pricing tiers: when a single request’s input exceeds 272K tokens, the entire request is billed at the long-context tier (2x input, 1.5x output). The 1.25x cache write and 0.1x cache read multipliers still apply in the long-context tier, based on the long-context input rate.

​
How caching is applied automatically

When you send a request, the system checks whether the prompt prefix (in serialized order of messages, tools, and so on) exactly matches the prefix of a recent request:
  1. If the prefix reaches 1,024 tokens and a matching cached prefix is found, the matched portion is billed at the cache read rate and time-to-first-token latency is reduced;
  2. Otherwise the prefix is processed as regular input and written to the cache (billed at 1.25x on GPT-5.6 and later);
  3. A cache hit requires the prefix to be byte-for-byte identical; any change within the prefix invalidates all cached content from that point onward.
Caching provides the most benefit in these scenarios:
  • Static long system instructions or large sets of few-shot examples
  • Long reference material repeatedly included in RAG workloads
  • Agent workflows carrying large tool definitions (tools)
  • Long multi-turn conversations that only append messages
Cache retention: on models before GPT-5.6, caches are cleared after 5–10 minutes of inactivity, up to 1 hour; on GPT-5.6 and later, caches are retained for at least 30 minutes and may persist longer. Caches are not shared across organizations, and caching has no effect on output content.

​
GPT-5.6 caching parameters

The GPT-5.6 family adds three caching-related parameters (shared by the Chat Completions and Responses APIs):
ParameterType / locationValuesDefault
prompt_cache_keystring, request body top levelA stable custom identifier, ideally scoped per workload or tenant; keep total traffic per key to roughly 15 requests per minuteNone
prompt_cache_optionsobject, request body top levelmode: "implicit" / "explicit"; ttl: only "30m" supportedmode: "implicit", ttl: "30m"
prompt_cache_breakpointobject, inside a content block{"mode": "explicit"}, marks the end of the cached prefixNo breakpoint
Models before GPT-5.6 do not support prompt_cache_options or prompt_cache_breakpoint, and such requests are rejected; the older 24-hour extended retention parameter prompt_cache_retention ("24h" / "in_memory") is replaced by prompt_cache_options.ttl on GPT-5.6 and later. How the three caching controls relate:
  1. Default (implicit mode): caching happens automatically even with no caching parameters — the system places an automatic breakpoint at the latest message. On GPT-5.6 and later, these automatic cache writes are also billed at 1.25x.
  2. Implicit mode + explicit breakpoints: in addition to the automatic breakpoint, you can set prompt_cache_breakpoint on a content block to pin the cache boundary at the end of stable content; changes after the breakpoint do not invalidate the cached prefix before it.
  3. Explicit mode: with prompt_cache_options.mode set to "explicit", only manual breakpoints are used; if no breakpoints are set at all, the request neither uses the cache nor incurs cache write charges. Official wording: “If the conversation contains no explicit breakpoints, the request does not use prompt caching or incur cache-write charges.”
Use explicit mode to avoid cache write charges on a one-off long request: 
{
  "model": "gpt-5.6-sol",
  "prompt_cache_options": {"mode": "explicit"},
  "messages": [
    {"role": "user", "content": "[One-off long content that will not be reused]"}
  ]
}
The official pattern for explicit breakpoints (breakpoint at the end of the static long content block): 
{
  "model": "gpt-5.6-sol",
  "prompt_cache_key": "my-app-report-assistant-v1",
  "messages": [
    {
      "role": "system",
      "content": [
        {
          "type": "text",
          "text": "[Static long instructions or reference material, >=1024 tokens]",
          "prompt_cache_breakpoint": {"mode": "explicit"}
        }
      ]
    },
    {"role": "user", "content": "Summarize the key figures in one sentence."}
  ]
}
Hard constraints (official):
  • Each request can create at most 4 new cache writes; in implicit mode, the automatic breakpoint uses 1 of them;
  • The prefix before a breakpoint must still reach 1,024 tokens to be cached;
  • On reads, the longest matching prefix is selected from the 50 most recent breakpoints;
  • Setting a breakpoint on an unsupported content block returns 400 invalid_request_error. Chat Completions supports text / image_url / input_audio / file / refusal blocks; the Responses API supports input_text / input_image / input_file blocks.
AIHubMix support for prompt_cache_breakpoint content-block breakpoints and for Responses API cache hits is being finalized. For now, we recommend using automatic caching through Chat Completions with prompt_cache_key set (the quick start example on this page, verified to hit the cache); the explicit mode of prompt_cache_options works as expected for disabling caching. This page will be updated as support progresses.

​
Why the cache is not being hit

A hit requires everything before the breakpoint position to be byte-for-byte identical. If cached_tokens is still 0 on the second request, work through this checklist:
  • Prefix under 1,024 tokens: requests below the minimum cacheable length are processed as regular input;
  • Changing content mixed into the prefix: timestamps, session IDs, per-user variables, and similar values should go after the static content; any change within the prefix invalidates all cached content after that point;
  • Tool definitions or ordering changed: the tool list is part of the prefix; definitions and their order must match exactly;
  • Inconsistent image detail parameter: detail affects how images are tokenized and must stay the same;
  • Structured output schema changed: the response_format JSON Schema is part of the system message prefix for caching purposes; a schema change is a prefix change;
  • reasoning_effort changed: officially listed as a common cause of lower cache hit rates (“Changes to reasoning effort”);
  • Cache retention expired: before GPT-5.6, caches are cleared after 5–10 minutes of inactivity; GPT-5.6 and later retain caches for at least 30 minutes;
  • prompt_cache_key not set (GPT-5.6): automatic hits are still possible without it, but the more reliable matching mechanism is not used.

​
Best practices

  • Put static content (system instructions, examples, reference material, tool definitions) at the very front of the request, and content that changes each turn at the end;
  • Use one stable prompt_cache_key for traffic sharing the same prefix, keep total traffic per key to roughly 15 requests per minute, and split into more keys per workload when exceeding that;
  • In multi-turn conversations, only append messages and avoid editing earlier messages;
  • Keep steady traffic on requests with the same prefix to reduce cache eviction;
  • For one-off long requests whose prefix will not be reused, use explicit mode to avoid cache write fees (GPT-5.6 and later);
  • Monitor hits continuously via usage.prompt_tokens_details.cached_tokens.

​
FAQ

​
Does GPT prompt caching need to be enabled manually?

No manual setup is required: prefixes of 1,024 tokens or more are cached automatically. On GPT-5.6 and later, also set prompt_cache_key for more reliable cache matching.

​
How are GPT-5.6 cache write fees calculated, and how do I avoid unnecessary write fees?

Cache writes are billed at 1.25x the base input rate and reads at 0.1x; a prefix reused even once produces a net saving. For one-off long requests whose prefix will not be reused, set prompt_cache_options.mode to "explicit" without setting any breakpoints, and the request neither uses the cache nor incurs write fees.

​
How long is the cache retained?

GPT-5.6 and later retain caches for at least 30 minutes (ttl currently only supports "30m"; actual retention may be longer). Models before GPT-5.6 clear caches after 5–10 minutes of inactivity, up to 1 hour, and some older models support extended retention via prompt_cache_retention: "24h".

​
How do GPT-5.6 explicit breakpoints differ from Claude’s cache_control?

Both pin the cache boundary at the end of stable content. The main differences: GPT-5.6 caches automatically with no parameters required, and breakpoints are optional fine-grained control, while Claude requires caching to be enabled in the request (a top-level cache_control automatic breakpoint or content-block-level explicit breakpoints); GPT-5.6 retains caches for at least 30 minutes, while Claude defaults to 5 minutes with an optional 1 hour; cache reads on both are billed at 0.1x the input rate. For Claude usage, see Claude Prompt Caching.

​
Does caching affect output content?

No. Official position: prompt caching only affects input-side processing and billing; the model generates output exactly as it would without caching.

​
Official references

The mechanisms, rates, and parameter definitions on this page come from these official OpenAI sources: Actual AIHubMix prices for each model are listed in the model gallery.
Last updated: 2026-07-10