> ## Documentation Index
> Fetch the complete documentation index at: https://docs.aihubmix.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Prompt Caching
Prompt Caching is an important mechanism used to reduce model inference costs. By caching previously processed prompt content, it can be reused in subsequent requests, thereby reducing redundant computations, lowering costs, and improving response efficiency.
## Principle
When you send a request with prompt caching enabled, the system checks if the prompt prefix has been cached from recent queries. If found, it uses the cache, reducing processing time and costs; otherwise, it processes the full prompt and caches the prefix after the response begins. This is particularly useful in the following scenarios:
* Prompts containing numerous examples
* Extensive context or background information
* Repetitive tasks with consistent instructions
* Long multi-turn conversations
## Core Mechanism
Different model providers have varying support for caching:
### Automatic Caching
Automatic caching requires no additional configuration; the system automatically identifies and caches reusable content, applicable to models like OpenAI, DeepSeek, etc.
#### OpenAI
* Minimum prompt length: 1024 tokens; caching applies automatically when the prefix matches exactly
* Models before GPT-5.6: cache writes have no additional fee; cache reads are billed at the model's cached-input rate
* GPT-5.6 and later (official scope: "GPT-5.6 models and later model families", currently gpt-5.6-sol / terra / luna): cache writes are billed at 1.25x the input rate and cache reads at 0.1x; the family adds `prompt_cache_key` and explicit cache breakpoint parameters
* For usage, billing, and cache-hit troubleshooting, see [GPT Prompt Caching](/en/api/GPT-Cache)
#### Gemini
* Implicit context caching is enabled by default, and caching is automatically effective without manual configuration.
* Caching is only effective when the content, model, and parameters are identical; any differences will be treated as a new request and will not hit the cache.
* The cache validity period is set by the developer, and it can also be left unset. If unspecified, it defaults to 1 hour. There are no minimum or maximum duration limits, and costs depend on the number of cached tokens and cache duration.
#### DeepSeek / Grok / Moonshot / Groq
* Cost: Writing to cache is free or at the same price, reading from cache is below the original price
### Claude Model Explicit Caching
* Enabled via `cache_control`: a top-level field in the request body sets the breakpoint automatically (moving forward as the conversation grows), or content-block-level breakpoints give fine-grained control over cache placement
* Supported on all active Claude models, with uniform rates across the lineup: cache writes at 1.25x for the 5-minute tier and 2x for the 1-hour tier, cache reads at 0.1x
* Applicable to Anthropic Claude models
Claude sets a minimum cacheable token threshold per model (512 / 1,024 / 2,048 / 4,096, and not proportional to the model version number): for example, Claude Opus 4.8 is 1,024, Claude Opus 4.7 is 2,048, Claude Opus 4.6 / 4.5 and Claude Haiku 4.5 are 4,096, and Claude Fable 5 is 512. A prefix below the threshold will not be written to the cache even if marked with `cache_control`, and no error is returned — this is the case when both `cache_creation_input_tokens` and `cache_read_input_tokens` are 0 in the response. See the full breakdown and troubleshooting in [Claude Prompt Caching](/en/api/Claude-Cache).
#### OpenAI Compatible Interface
You can set caching breakpoints in `system`, `user` (including images), and `tools` using the `cache_control` field. The following examples only show the key structure:
**System Message Caching (default 5 minutes TTL):**
```json theme={null}
{
"model": "claude-opus-4-5",
"messages": [
{
"role": "system",
"content": [
{"type": "text", "text": "You are an AI assistant"},
{
"type": "text",
"text": "(long context)",
"cache_control": {"type": "ephemeral"}
}
]
},
{
"role": "user",
"content": [{"type": "text", "text": "Hello"}]
}
]
}
```
**User Message Caching (1 hour TTL):**
```json theme={null}
{
"model": "claude-opus-4-5",
"messages": [
{
"role": "system",
"content": [{"type": "text", "text": "You are an AI assistant"}]
},
{
"role": "user",
"content": [
{
"type": "text",
"text": "(long context)",
"cache_control": {"type": "ephemeral", "ttl": "1h"}
},
{"type": "text", "text": "Hello"}
]
}
]
}
```
**Image Message Caching:**
```json theme={null}
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {"detail": "auto", "url": "data:image/jpeg;base64,/9j/4AAQ..."},
"cache_control": {"type": "ephemeral"}
},
{"type": "text", "text": "What's this?"}
]
}
```
**Tool Definition Caching:**
Place the `cache_control` at the top level of the tool object (at the same level as `type` and `function`):
```json theme={null}
{
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a location",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
},
"cache_control": {"type": "ephemeral", "ttl": "1h"}
}]
}
```
#### Anthropic Compatible Interface
```shellscript cURL theme={null}
curl https://aihubmix.com/v1/messages \
-H "content-type: application/json" \
-H "x-api-key: $AIHUBMIX_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-d '{
"model": "claude-opus-4-6",
"max_tokens": 1024,
"system": [
{
"type": "text",
"text": "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n"
},
{
"type": "text",
"text": "",
"cache_control": {"type": "ephemeral"}
}
],
"messages": [
{
"role": "user",
"content": "Analyze the major themes in Pride and Prejudice."
}
]
}'
# Call the model again with the same input until the caching checkpoint
curl https://aihubmix.com/v1/messages # rest of input
```
```python Python theme={null}
import anthropic
client = Anthropic(
api_key="",
base_url="https://aihubmix.com"
)
params = {
"model": "claude-opus-4-6",
"max_tokens": 1024,
"system": [
{
"type": "text",
"text": "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n",
},
{
"type": "text",
"text": "",
"cache_control": {"type": "ephemeral"},
},
],
"messages": [
{
"role": "user",
"content": "Analyze the major themes in 'Pride and Prejudice'.",
}
],
}
response = client.messages.create(**params)
print(response.usage.model_dump_json())
# Call the model again with the same input until the caching checkpoint
response = client.messages.create(**params)
print(response.usage.model_dump_json())
```
#### Caching Duration
* Default: 5 minutes
* Optional: 1 hour ("ttl": "1h")
> For more information, please refer to: [Claude Prompt Caching](https://docs.aihubmix.com/en/blogs/aihubmix-openai-upgrade-claude#2-prompt-caching)
## Usage Recommendations
1. **Maintain Stable Prefixes**
Place fixed content at the beginning of the prompt, recommended structure:
```text theme={null}
[System Settings / Long Text / RAG Data]
[User Question (Variable Part)]
```
2. **Cache Large Texts**
Prioritize caching the following content:
* RAG data
* Long texts
* CSV / JSON data
* Role settings
3. **Control TTL**
* Short sessions → 5 minutes
* Long sessions → 1 hour (more cost-effective)
4. **Reduce Cache Writes**
## Avoid frequently changing content from entering the cache. Do not cache timestamps, user input variables, high-frequency changing data, etc.
Last updated: 2026-07-10