Mise en cache des prompts

​

Principe

• Prompts contenant de nombreux exemples
• Contexte ou informations de fond étendus
• Tâches répétitives avec des instructions cohérentes
• Longues conversations multi-tours

​

Mécanisme principal

​

Mise en cache automatique

​

OpenAI

• Longueur minimale de prompt : 1024 tokens ; le cache est touché automatiquement lorsque le préfixe est identique mot pour mot
• Modèles antérieurs à GPT-5.6 : l’écriture en cache est sans facturation supplémentaire, la lecture en cache est facturée au tarif de lecture de cache du modèle concerné
• GPT-5.6 et suivants (formulation officielle “GPT-5.6 models and later model families”, actuellement gpt-5.6-sol / terra / luna) : écriture en cache facturée à 1,25x le prix d’entrée, lecture à 0,1x ; nouveaux paramètres prompt_cache_key et points de rupture de cache explicites
• Usage, facturation et dépannage des hits : Mise en cache des prompts GPT

​

Gemini

• Implicit context caching is enabled by default, and caching is automatically effective without manual configuration.
• Caching is only effective when the content, model, and parameters are identical; any differences will be treated as a new request and will not hit the cache.
• The cache validity period is set by the developer, and it can also be left unset. If unspecified, it defaults to 1 hour. There are no minimum or maximum duration limits, and costs depend on the number of cached tokens and cache duration.

​

DeepSeek / Grok / Moonshot / Groq

• Cost: Writing to cache is free or at the same price, reading from cache is below the original price

​

Mise en cache explicite des modèles Claude

• Activation via cache_control : champ à la racine du corps de requête pour un point de rupture automatique (qui avance avec la conversation), ou point de rupture au niveau des blocs de contenu pour un contrôle précis de la position du cache
• Pris en charge par tous les modèles Claude actifs ; écriture en cache à 1,25x pour le palier 5 minutes, 2x pour le palier 1 heure, lecture à 0,1x, taux unifiés sur toute la gamme
• Applicable aux modèles Anthropic Claude

​

OpenAI Compatible Interface

{
  "model": "claude-opus-4-5",
  "messages": [
    {
      "role": "system",
      "content": [
        {"type": "text", "text": "You are an AI assistant"},
        {
          "type": "text",
          "text": "(long context)",
          "cache_control": {"type": "ephemeral"}
        }
      ]
    },
    {
      "role": "user",
      "content": [{"type": "text", "text": "Hello"}]
    }
  ]
}

{
  "model": "claude-opus-4-5",
  "messages": [
    {
      "role": "system",
      "content": [{"type": "text", "text": "You are an AI assistant"}]
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "(long context)",
          "cache_control": {"type": "ephemeral", "ttl": "1h"}
        },
        {"type": "text", "text": "Hello"}
      ]
    }
  ]
}

{
  "role": "user",
  "content": [
    {
      "type": "image_url",
      "image_url": {"detail": "auto", "url": "data:image/jpeg;base64,/9j/4AAQ..."},
      "cache_control": {"type": "ephemeral"}
    },
    {"type": "text", "text": "What's this?"}
  ]
}

{
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Get current weather for a location",
      "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"]
      }
    },
    "cache_control": {"type": "ephemeral", "ttl": "1h"}
  }]
}

​

Anthropic Compatible Interface

curl https://aihubmix.com/v1/messages \
  -H "content-type: application/json" \
  -H "x-api-key: $AIHUBMIX_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-opus-4-6",
    "max_tokens": 1024,
    "system": [
      {
        "type": "text",
        "text": "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n"
      },
      {
        "type": "text",
        "text": "<the entire contents of Pride and Prejudice>",
        "cache_control": {"type": "ephemeral"}
      }
    ],
    "messages": [
      {
        "role": "user",
        "content": "Analyze the major themes in Pride and Prejudice."
      }
    ]
  }'

# Call the model again with the same input until the caching checkpoint
curl https://aihubmix.com/v1/messages # rest of input

import anthropic

client = Anthropic(
  api_key="<AIHUBMIX_API_KEY>",  
  base_url="https://aihubmix.com"
)

params = {
    "model": "claude-opus-4-6",
    "max_tokens": 1024,
    "system": [
        {
            "type": "text",
            "text": "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n",
        },
        {
            "type": "text",
            "text": "<the entire contents of 'Pride and Prejudice'>",
            "cache_control": {"type": "ephemeral"},
        },
    ],
    "messages": [
        {
            "role": "user",
            "content": "Analyze the major themes in 'Pride and Prejudice'.",
        }
    ],
}
response = client.messages.create(**params)
print(response.usage.model_dump_json())

# Call the model again with the same input until the caching checkpoint
response = client.messages.create(**params)
print(response.usage.model_dump_json())

​

Caching Duration

• Default: 5 minutes
• Optional: 1 hour (“ttl”: “1h”)

For more information, please refer to: Claude Prompt Caching

​

Usage Recommendations

1. Maintain Stable Prefixes

[System Settings / Long Text / RAG Data] 
[User Question (Variable Part)]

2. Cache Large Texts

• RAG data
• Long texts
• CSV / JSON data
• Role settings

3. Control TTL

• Short sessions → 5 minutes
• Long sessions → 1 hour (more cost-effective)

4. Reduce Cache Writes

​

Avoid frequently changing content from entering the cache. Do not cache timestamps, user input variables, high-frequency changing data, etc.