Cache de prompts

​

Principle

• Prompts containing numerous examples
• Extensive context or background information
• Repetitive tasks with consistent instructions
• Long multi-turn conversations

​

Core Mechanism

​

Automatic Caching

​

OpenAI

• Comprimento mínimo do prompt: 1024 tokens; o acerto é automático quando o prefixo é idêntico, caractere por caractere
• Modelos anteriores ao GPT-5.6: a escrita de cache não tem cobrança adicional; a leitura de cache é cobrada pelo preço de leitura de cache do modelo correspondente
• GPT-5.6 e posteriores (critério oficial “GPT-5.6 models and later model families”; atualmente gpt-5.6-sol / terra / luna): escrita de cache cobrada a 1.25x o preço de entrada e leitura a 0.1x; novos parâmetros prompt_cache_key e pontos de quebra de cache explícitos
• Uso, cobrança e solução de problemas de acerto em Cache de Prompt do GPT

​

Gemini

• Implicit context caching is enabled by default, and caching is automatically effective without manual configuration.
• Caching is only effective when the content, model, and parameters are identical; any differences will be treated as a new request and will not hit the cache.
• The cache validity period is set by the developer, and it can also be left unset. If unspecified, it defaults to 1 hour. There are no minimum or maximum duration limits, and costs depend on the number of cached tokens and cache duration.

​

DeepSeek / Grok / Moonshot / Groq

• Cost: Writing to cache is free or at the same price, reading from cache is below the original price

​

Cache explícito dos modelos Claude

• Ativação via cache_control: campo de nível superior do corpo da requisição para ponto de quebra automático (move para frente com a conversa), ou em nível de bloco de conteúdo para controle preciso da posição do cache
• Suportado por todos os modelos Claude ativos; escrita de cache a 1.25x na faixa de 5 minutos e 2x na de 1 hora, leitura a 0.1x, com proporções de preço uniformes em toda a linha
• Aplicável aos modelos Claude da Anthropic

​

OpenAI Compatible Interface

{
  "model": "claude-opus-4-5",
  "messages": [
    {
      "role": "system",
      "content": [
        {"type": "text", "text": "You are an AI assistant"},
        {
          "type": "text",
          "text": "(long context)",
          "cache_control": {"type": "ephemeral"}
        }
      ]
    },
    {
      "role": "user",
      "content": [{"type": "text", "text": "Hello"}]
    }
  ]
}

{
  "model": "claude-opus-4-5",
  "messages": [
    {
      "role": "system",
      "content": [{"type": "text", "text": "You are an AI assistant"}]
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "(long context)",
          "cache_control": {"type": "ephemeral", "ttl": "1h"}
        },
        {"type": "text", "text": "Hello"}
      ]
    }
  ]
}

{
  "role": "user",
  "content": [
    {
      "type": "image_url",
      "image_url": {"detail": "auto", "url": "data:image/jpeg;base64,/9j/4AAQ..."},
      "cache_control": {"type": "ephemeral"}
    },
    {"type": "text", "text": "What's this?"}
  ]
}

{
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Get current weather for a location",
      "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"]
      }
    },
    "cache_control": {"type": "ephemeral", "ttl": "1h"}
  }]
}

​

Anthropic Compatible Interface

curl https://aihubmix.com/v1/messages \
  -H "content-type: application/json" \
  -H "x-api-key: $AIHUBMIX_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-opus-4-6",
    "max_tokens": 1024,
    "system": [
      {
        "type": "text",
        "text": "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n"
      },
      {
        "type": "text",
        "text": "<the entire contents of Pride and Prejudice>",
        "cache_control": {"type": "ephemeral"}
      }
    ],
    "messages": [
      {
        "role": "user",
        "content": "Analyze the major themes in Pride and Prejudice."
      }
    ]
  }'

# Call the model again with the same input until the caching checkpoint
curl https://aihubmix.com/v1/messages # rest of input

import anthropic

client = Anthropic(
  api_key="<AIHUBMIX_API_KEY>",  
  base_url="https://aihubmix.com"
)

params = {
    "model": "claude-opus-4-6",
    "max_tokens": 1024,
    "system": [
        {
            "type": "text",
            "text": "You are an AI assistant tasked with analyzing literary works. Your goal is to provide insightful commentary on themes, characters, and writing style.\n",
        },
        {
            "type": "text",
            "text": "<the entire contents of 'Pride and Prejudice'>",
            "cache_control": {"type": "ephemeral"},
        },
    ],
    "messages": [
        {
            "role": "user",
            "content": "Analyze the major themes in 'Pride and Prejudice'.",
        }
    ],
}
response = client.messages.create(**params)
print(response.usage.model_dump_json())

# Call the model again with the same input until the caching checkpoint
response = client.messages.create(**params)
print(response.usage.model_dump_json())

​

Caching Duration

• Default: 5 minutes
• Optional: 1 hour (“ttl”: “1h”)

Para mais informações, consulte: Claude Prompt Caching

​

Usage Recommendations

1. Maintain Stable Prefixes

[System Settings / Long Text / RAG Data] 
[User Question (Variable Part)]

2. Cache Large Texts

• RAG data
• Long texts
• CSV / JSON data
• Role settings

3. Control TTL

• Short sessions → 5 minutes
• Long sessions → 1 hour (more cost-effective)

4. Reduce Cache Writes

​

Avoid frequently changing content from entering the cache. Do not cache timestamps, user input variables, high-frequency changing data, etc.