> ## Documentation Index
> Fetch the complete documentation index at: https://docs.aihubmix.com/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Router

> AIHubMix LLM Router: set model to auto and the gateway picks the best model per request — cost / quality / latency policies, billed by the model used.

> One `model=auto`, and the gateway decides "which model to use."

The **LLM Router** analyzes each request and selects the most suitable model in real time from the **hundreds of models** on the platform. All you do is set `model` to `auto`—no picking models, no comparing prices, no tracking model releases.

<Note>
  Billed by the **resolved model**, with no surcharge and zero client code changes. Which model was hit is written into the response headers and body (see [How to confirm which model was used](#how-to-confirm-which-model-was-used)), so it is fully traceable.
</Note>

## Use cases

* **Auto-dispatch by context**: automatically assign the most suitable model based on the user's current context—especially useful for agents / apps that call models many times and can't hard-code the model choice for each step in advance.
* **Cost optimization**: let simple tasks land on cheaper, faster models automatically (`auto` is cost-first by default).
* **Quality optimization**: ensure complex requests are routed to more capable models (`auto:quality_first`).
* **Latency-critical scenarios**: multi-turn agent loops, real-time interactive chat, and other latency-sensitive scenarios prefer the fastest-responding model (`auto:latency_critical`).
* **Single entry point, no model selection**: different request types are dispatched to their own optimal models—no need to maintain a "task → model" mapping table, and no need to keep tracking model releases, comparing prices, and swapping names by hand.

***

## Quick start

Set `model` to `auto`; the rest of your request body stays exactly the same as a normal call. Use `https://aihubmix.com/v1` as the base\_url.

<CodeGroup>
  ```bash curl theme={null}
  curl https://aihubmix.com/v1/chat/completions \
    -H "Authorization: Bearer <AIHUBMIX_API_KEY>" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "auto",
      "messages": [
        { "role": "user", "content": "What is the meaning of life?" }
      ]
    }'
  ```

  ```python Python (openai SDK) theme={null}
  from openai import OpenAI

  client = OpenAI(
      api_key="<AIHUBMIX_API_KEY>",
      base_url="https://aihubmix.com/v1",
  )

  resp = client.chat.completions.create(
      model="auto",  # let the gateway pick the model
      messages=[
          {"role": "user", "content": "What is the meaning of life?"},
      ],
  )

  print(resp.choices[0].message.content)
  print("Resolved model:", resp.model)  # not "auto", but the real model name
  ```

  ```javascript Node.js (openai SDK) theme={null}
  import OpenAI from "openai";

  const client = new OpenAI({
    apiKey: "<AIHUBMIX_API_KEY>",
    baseURL: "https://aihubmix.com/v1",
  });

  const resp = await client.chat.completions.create({
    model: "auto", // let the gateway pick the model
    messages: [
      { role: "user", content: "What is the meaning of life?" },
    ],
  });

  console.log(resp.choices[0].message.content);
  console.log("Resolved model:", resp.model); // the real model name
  ```
</CodeGroup>

<Tip>
  The LLM Router runs prompt analysis **before** the request reaches the upstream, treating streaming (`stream: true`) and non-streaming requests identically, with no extra parameters; the whole decision **adds only about 1ms of overhead**, with virtually no impact on end-to-end latency.
</Tip>

***

## How to confirm which model was used

This is the trust anchor of the LLM Router: **you always know which model ended up handling this request.**

**Method 1 · AIHubMix console "Logs"**: at [console.aihubmix.com/logs](https://console.aihubmix.com/logs), every request shows the real resolved model that was actually used and billed—no code needed, verifiable by eye.

**Method 2 · Response fields** (handy for programmatic reads):

* **The `model` field in the response body** is backfilled with the real resolved model (e.g. `mimo-v2.5-pro`), not `auto`.
* **The response headers** give you the full decision detail:

| Header                             | Meaning                                                                     | Example value                                                      |
| ---------------------------------- | --------------------------------------------------------------------------- | ------------------------------------------------------------------ |
| `X-Aihubmix-Router-Resolved-Model` | The model actually hit and billed accordingly                               | `xiaomi-mimo-v2.5-pro`                                             |
| `X-Aihubmix-Router-Policy`         | The policy used for this request                                            | `cost_optimized`                                                   |
| `X-Aihubmix-Router-Dimension`      | The detected task dimension                                                 | `text.overall`                                                     |
| `X-Aihubmix-Router-Decision-Id`    | A unique ID for this decision, for troubleshooting                          | `05dbad09-33c5-42de-…`                                             |
| `X-Aihubmix-Router-Reason`         | A brief decision summary (policy / dimension / top score / candidate count) | `policy=cost_optimized dim=text.overall top=0.182 survivors=20/33` |
| `X-Aihubmix-Router-Fallback`       | Present **only** when the no-candidate fallback is triggered                | `true`                                                             |

> HTTP response headers are case-insensitive: the table above capitalizes them by convention, but the actual HTTP/2 response returns them lowercased as `x-aihubmix-router-*`—the two are equivalent.

Read the routing decision (curl to see the headers; SDKs use the raw response object to get headers):

<CodeGroup>
  ```bash curl theme={null}
  curl -i https://aihubmix.com/v1/chat/completions \
    -H "Authorization: Bearer <AIHUBMIX_API_KEY>" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "auto",
      "messages": [
        { "role": "user", "content": "What is the meaning of life?" }
      ]
    }' | grep -i "^x-aihubmix-router"
  ```

  ```python Python (openai SDK) theme={null}
  # Reuse the client created above; with_raw_response is needed to get the headers
  raw = client.chat.completions.with_raw_response.create(
      model="auto",
      messages=[
          {"role": "user", "content": "What is the meaning of life?"},
      ],
  )

  print("Resolved model:", raw.headers.get("x-aihubmix-router-resolved-model"))
  print("Policy:", raw.headers.get("x-aihubmix-router-policy"))
  print("Dimension:", raw.headers.get("x-aihubmix-router-dimension"))

  completion = raw.parse()  # parse into a normal completion object
  print("body.model:", completion.model)
  ```

  ```javascript Node.js (openai SDK) theme={null}
  // Reuse the client created above; .withResponse() is needed to get the raw headers
  const { data: completion, response } = await client.chat.completions
    .create({
      model: "auto",
      messages: [
        { role: "user", content: "What is the meaning of life?" },
      ],
    })
    .withResponse();

  console.log("Resolved model:", response.headers.get("x-aihubmix-router-resolved-model"));
  console.log("Policy:", response.headers.get("x-aihubmix-router-policy"));
  console.log("Dimension:", response.headers.get("x-aihubmix-router-dimension"));
  console.log("body.model:", completion.model);
  ```
</CodeGroup>

Actual curl output (production; the resolved model varies with the live catalog):

```text theme={null}
x-aihubmix-router-decision-id: 05dbad09-33c5-42de-85b5-559fdb73eb4c
x-aihubmix-router-dimension: text.overall
x-aihubmix-router-policy: cost_optimized
x-aihubmix-router-reason: policy=cost_optimized dim=text.overall top=0.182 survivors=20/33
x-aihubmix-router-resolved-model: xiaomi-mimo-v2.5-pro
```

How to read `reason`: `survivors=20/33` means that out of 33 candidates, 20 passed hard filtering and entered scoring; `top=0.182` is the winning model's normalized composite score within the candidate pool (capability / cost / latency weighted by policy).

<Note>
  The `Resolved-Model` in the example depends on the current candidates and prices in the live catalog, and will change as platform models come and go—which is exactly the value of the LLM Router: you don't have to track these changes. To keep your decisions auditable, rely on the real model name in the response headers / body rather than assuming it stays fixed.
</Note>

***

## Routing policies

`auto` without a suffix uses the default policy `cost_optimized`. You can use `auto:<policy>` to explicitly specify the emphasis:

| Policy syntax                    | Emphasis                                                              | Use case                                   |
| -------------------------------- | --------------------------------------------------------------------- | ------------------------------------------ |
| `auto` (= `auto:cost_optimized`) | **Cost first**: pick the cheapest model that meets the capability bar | Batch tasks, cost-sensitive workloads      |
| `auto:balanced`                  | **Balanced**: weighs capability / cost / latency                      | General purpose, a safe choice when unsure |
| `auto:quality_first`             | **Quality first**: prefer the most capable model                      | Complex reasoning, critical output         |
| `auto:latency_critical`          | **Latency first**: prefer the fastest-responding model                | Agent loops, real-time interactive chat    |

A policy is not a fixed "list of models" but a different weighting of **capability / cost / latency**. `auto` first scopes the task dimension by your request's content, then picks the best in real time from the candidate pool of **hundreds of models on the platform** under the chosen policy—so the same policy resolves to different models for different content. The "same request, different policies → different resolutions" table below is a direct illustration of this; which model wins each time is determined by the real model name in the response headers / console logs. The current model pool and per-dimension scores can be queried via the [LLM Router Model Scope](/en/api/RouterEndpoints/leaderboard) endpoint.

To specify a policy, just append the suffix to `model`:

<CodeGroup>
  ```bash curl theme={null}
  # Quality first + a coding task
  curl -i https://aihubmix.com/v1/chat/completions \
    -H "Authorization: Bearer <AIHUBMIX_API_KEY>" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "auto:quality_first",
      "messages": [
        { "role": "user", "content": "Write a Python function to reverse a linked list." }
      ]
    }' | grep -i "^x-aihubmix-router"
  ```

  ```python Python (openai SDK) theme={null}
  raw = client.chat.completions.with_raw_response.create(
      model="auto:quality_first",  # quality first
      messages=[
          {"role": "user", "content": "Write a Python function to reverse a linked list."},
      ],
  )

  print(raw.headers.get("x-aihubmix-router-resolved-model"))
  print(raw.headers.get("x-aihubmix-router-dimension"))
  ```

  ```javascript Node.js (openai SDK) theme={null}
  const { response } = await client.chat.completions
    .create({
      model: "auto:quality_first", // quality first
      messages: [
        { role: "user", content: "Write a Python function to reverse a linked list." },
      ],
    })
    .withResponse();

  console.log(response.headers.get("x-aihubmix-router-resolved-model"));
  console.log(response.headers.get("x-aihubmix-router-dimension"));
  ```
</CodeGroup>

**Same request, different policies → different resolved models** (measured in production, the same prompt `What is the meaning of life?`, all landing on the `text.overall` dimension):

| Policy                      | Resolved model          | top score |
| --------------------------- | ----------------------- | :-------: |
| `auto` (= `cost_optimized`) | `xiaomi-mimo-v2.5-pro`  |   0.182   |
| `auto:balanced`             | `claude-opus-4-6-think` |   0.488   |
| `auto:latency_critical`     | `claude-opus-4-6`       |   0.646   |
| `auto:quality_first`        | `claude-opus-4-6-think` |   0.758   |

> `latency_critical` picked the **non-`-think` version**—thinking variants have higher reasoning latency, so the latency policy actively avoids them. This shows that policy weights genuinely act on the "capability / cost / latency" tradeoff, not on capability alone.

> **Content changes the result too**: applying the same `auto:quality_first` to a **coding task** (the request in the example above), the dimension shifts from `text.overall` to `text.coding`, and the measured resolved model is `claude-opus-4-6-think`—the policy and the request content together determine the final model.

<Note>
  An unknown policy suffix (e.g. `auto:fast`) **falls back to the default policy `cost_optimized`** without raising an error.
</Note>

***

## How it works

Upon receiving `model=auto`, the gateway turns "intent" into "a concrete model" in three steps:

<Steps>
  <Step title="Extract request features">
    It analyzes this request's input / output modalities (text, image, file), content intent (code, math, OCR, diagrams, language, whether web search is on, etc.), and request scale (estimated input / output tokens), normalizing them into a single **task dimension**. For example: a question containing code → `text.coding`; an image plus an OCR request → `vision.ocr`; plain text → `text.overall`.
  </Step>

  <Step title="Hard-filter the candidates">
    Models that don't satisfy **hard constraints** are excluded outright: not supporting the required input / output modalities, a context window that can't fit the request, removed by the circuit breaker (see [Reliability and fault tolerance](#reliability-and-fault-tolerance)), or not within your Key's allowed model range.
  </Step>

  <Step title="Score by policy weighting">
    For the candidates that pass filtering, the router takes model capability scores from **authoritative industry benchmarks**, overlays real-time price and performance data, applies a three-dimensional weighted score over "capability / cost / latency" according to the chosen policy, and picks the highest-scoring one. The final model name is written back into the request and the response headers.
  </Step>
</Steps>

A real scoring example (under the `quality_first` policy, top 3 of the same candidate pool; example data based on historical production decision logs):

| Candidate model         | Capability score | Relative cost | Latency | Composite score |
| ----------------------- | :--------------: | :-----------: | :-----: | :-------------: |
| `claude-opus-4-6-think` |       1504       |      220      |  1963ms |    **0.758**    |
| `claude-opus-4-6`       |       1498       |      220      |  822ms  |      0.721      |
| `claude-fable-5`        |       1510       |      484      | 11130ms |      0.600      |

> Note that `claude-fable-5` has the **highest capability score** (1510), yet its higher cost and larger latency push its composite score down to third. This is exactly the point of weighted scoring: not "capability above all," but a policy-driven tradeoff among capability / cost / latency.

<Note>
  `claude-fable-5` was a staged preview baseline model and is now **deprecated** and no longer offered; its historical score is kept here only to demonstrate the weighted-scoring mechanism—real requests will no longer resolve to it.
</Note>

Dimension detection is automatic—the LLM Router has **30+ fine-grained task dimensions** built in (code / math / OCR / diagrams / long text / Chinese / web search…), far more precise than "coarse routing by model family." With the same `auto`, different content routes to different dimensions:

| Your request                                             | Detected dimension        |
| -------------------------------------------------------- | ------------------------- |
| Plain text question                                      | `text.overall`            |
| Contains code, asks to write / debug a program           | `text.coding`             |
| Math proof / solving                                     | `text.math`               |
| A very long question (around 500+ tokens)                | `text.longer_query`       |
| A question in Chinese                                    | `text.language.chinese`   |
| Image input + "What is in this image?"                   | `vision.overall`          |
| Image input + "OCR…" / "extract the text"                | `vision.ocr`              |
| Image input + chart / flowchart                          | `vision.diagram`          |
| Web search enabled                                       | `search.overall`          |
| Image generation (`/v1/images/generations`)              | `text_to_image.overall`   |
| Image editing (`/v1/images/edits`, image in → image out) | `image_edit.single_image` |

These dimension names come from authoritative industry leaderboards that break down model **fine-grained skills**; `auto` sends each kind of request to the model strongest at that skill. Common domains, for example:

* **Text**: `text.coding` = write / debug code, `text.math` = math solving, `text.longer_query` = long-form, `text.language.chinese` = Chinese, `text.occupational.legal` / `text.occupational.medicine` = legal / medical professional scenarios.
* **Vision**: `vision.ocr` = recognize text in images, `vision.diagram` = understand charts / flowcharts, `vision.overall` = general image understanding.

<Tip>
  Dimension detection uses conservative matching (high precision, low false positives): long-tail, ambiguous requests fall back to more general dimensions (e.g. `text.overall` / `vision.overall`) rather than being forced into a category, which avoids misrouting.
</Tip>

<Note>
  **Image input also goes through the LLM Router**: when you include an image in `/v1/chat/completions`, it is routed by the image task to a model with strong vision capabilities. Measured in production—"OCR this image" → `vision.ocr`, resolved to `qwen3.5-397b-a17b`; a general "What is in this image?" → `vision.overall`, resolved to `gpt-5.4-mini`. (This refers to image **understanding**; image **generation** at `/v1/images/*` also supports `auto`, see [FAQ](#faq).)
</Note>

***

## Scoring leaderboard and model pool

The per-dimension model scoring leaderboard and the current model pool can be browsed interactively on the [LLM Router page](https://aihubmix.com/llm-router/auto?lang=en), or fetched directly via login-free open endpoints: [LLM Router Model Scope](/en/api/RouterEndpoints/leaderboard) and [Model Vendor Icons](/en/api/RouterEndpoints/vendors). The leaderboard shows the same set as the routing candidates: only currently routable models appear, scores are normalized to 0–100 within each dimension, and the board updates continuously with the model pool.

***

## Reliability and fault tolerance

The LLM Router has **multiple layers of fault tolerance** built in, so the `auto` path **never fails for no reason**:

<AccordionGroup>
  <Accordion title="Circuit breaker: automatically remove failing models">
    The gateway maintains a sliding-window failure-rate statistic for each model. When a model fails enough times within the window and its failure rate exceeds the threshold, it is temporarily removed from the candidate pool and automatically recovers after a cooldown period—avoiding sending subsequent requests to a model that is currently misbehaving. The failure signal comes from **errors the upstream returns for that request**; the gateway's own "no available channel" does not count (that is not a problem with the model itself).
  </Accordion>

  <Accordion title="No-candidate fallback: never return 400 on auto">
    If hard filtering happens to exclude every candidate (for example, a certain modality combination temporarily has no available model), the gateway does not error out. Instead, it assigns a fallback model by output type to guarantee a response, and adds `X-Aihubmix-Router-Fallback: true` to the response headers to let you know.
  </Accordion>

  <Accordion title="Privilege boundary: a restricted Key is not bypassed by the fallback">
    If your Key limits the allowed model range, the model selected by the LLM Router (including the fallback) is **always** within that range. If there really is no model within the range that can serve this request, it explicitly returns 403 rather than silently using a model outside the range (which might be more expensive).
  </Accordion>
</AccordionGroup>

***

## Billing

**Billed at the list price of the resolved model; the LLM Router itself charges no surcharge.**

Whichever model ends up responding is what you're billed for, by that model's price, capability, and context limits—and that model is the value in the response header `X-Aihubmix-Router-Resolved-Model` and the response body `model` field. In other words, the LLM Router will not "secretly use an expensive model": every resolution is written into the response and can be reconciled line by line.

***

## Limitations

* The LLM Router currently targets the **chat completions** endpoint `/v1/chat/completions` and the **image generation / editing** endpoints `/v1/images/*` (see [FAQ: which endpoints are supported](#faq)).

* `?router=off` or the request header `X-Router-Off` makes `model=auto` return **400** directly—this is an explicit refusal of the ambiguous "want auto but also turn routing off" usage, rather than a silent ignore:

  ```bash theme={null}
  curl -i "https://aihubmix.com/v1/chat/completions?router=off" \
    -H "Authorization: Bearer <AIHUBMIX_API_KEY>" \
    -H "Content-Type: application/json" \
    -d '{"model":"auto","messages":[{"role":"user","content":"hi"}]}'
  # → HTTP/1.1 400 Bad Request
  # {"error":{"message":"auto requires router enabled; remove ?router=off / X-Router-Off", ...}}
  ```

* The candidate pool changes dynamically with the platform catalog: the same `auto` may resolve to different models at different times (this is by design, and reviewable via the response headers). The current candidate scope can be queried via the [LLM Router Model Scope](/en/api/RouterEndpoints/leaderboard) endpoint.

***

## Differences from OpenRouter / LiteLLM

"Automatic model selection" is not unique to AIHubMix; OpenRouter and LiteLLM both offer similar capabilities. The differences are mainly in **integration cost** and **hosting model**:

| Difference                                                                            | OpenRouter | LiteLLM | AIHubMix |
| ------------------------------------------------------------------------------------- | :--------: | :-----: | :------: |
| Automatic model selection by request content                                          |      ✅     |    ✅    |     ✅    |
| Zero config, works out of the box (no routing rules / utterances to write)            |      ✅     |    ❌    |     ✅    |
| Platform-hosted, no self-built / self-deployed proxy                                  |      ✅     |    ❌    |     ✅    |
| Cost / quality / latency multi-policy, switchable with one parameter                  |      ❌     |    ❌    |     ✅    |
| Traceable resolution decisions (response headers include dimension / policy / reason) |      ❌     |    ❌    |     ✅    |
| Billed by the final resolved model                                                    |      ✅     |    ❌    |     ✅    |

***

## FAQ

**Q: Which endpoints does the LLM Router support?**
A: Currently `model=auto` supports the **OpenAI-compatible chat completions endpoint** `/v1/chat/completions`, as well as the **image generation / editing endpoints** (`/v1/images/generations`, `/v1/images/edits`). Audio, `/v1/embeddings`, `/v1/rerank`, and other endpoints do not yet support `auto`—specify a concrete model directly.

**Q: Does the LLM Router support image input?**
A: Yes. Asking with an image (`image_url`) in `/v1/chat/completions` is image **understanding**, and it is routed by the image task to a model with strong vision capabilities (`vision.ocr` for recognizing text in images, `vision.diagram` for understanding charts / flowcharts, `vision.overall` for general image understanding, etc.). Image **generation** also supports `auto`: set `model` to `auto` on the `/v1/images/*` endpoints and the request is routed by image-generation dimensions (e.g. `text_to_image.overall`).

**Q: How do I know which model this request actually used?**
A: Check the response header `X-Aihubmix-Router-Resolved-Model`, or the `model` field in the response body—both are backfilled with the real model name. See [How to confirm which model was used](#how-to-confirm-which-model-was-used).

**Q: Will the LLM Router secretly use an expensive model?**
A: No. The default policy `cost_optimized` is cost-first; and every resolved model is written into the response and billed at its list price, so it can be reconciled line by line. See [Billing](#billing).

**Q: How do I control / estimate cost?**
A: Three measures stack together—① the default `auto` (`cost_optimized`) is already cost-first; ② use the **Key's allowed model range** to lock candidates within a price level you accept, which effectively sets an upper bound on cost; ③ each resolution is billed at the list price of the model in the response header `Resolved-Model`, reconcilable line by line. When you need stronger capability, explicitly use `auto:quality_first`.

**Q: What's the difference between `auto` and "model mapping / fallback"?**
A: [Model mapping / fallback](https://docs.aihubmix.com/en/api/Model-Mapping-Fallback) is a **Key-level fixed alias + an ordered fallback on failure** (the same target every time); the LLM Router selects a model dynamically **based on the content of each request**. The former solves "the client only knows a certain name / switch to a backup when the primary is down," while the latter solves "I don't care which one, just give me the most suitable."

**Q: Can I restrict the LLM Router to choose only among a few models?**
A: Yes—constrain it via the **Key's allowed model range**: the LLM Router will only choose among the models that Key permits, and out-of-range models will not be resolved.

**Q: Are streaming requests supported?**
A: Yes. Routing completes before the request reaches the upstream, treating streaming / non-streaming identically.

**Q: Why did the same sentence resolve to different models across two calls?**
A: The candidate pool and prices change dynamically with the platform catalog—this is by design. Use the `Decision-Id` and `Resolved-Model` in the response headers to review each decision; the current candidate scope can be queried via the [LLM Router Model Scope](/en/api/RouterEndpoints/leaderboard) endpoint.

**Q: How do I make requests consistently resolve to the same model (e.g. to reuse prompt caching)?**
A: `auto` selects a model dynamically against the current catalog and does not guarantee determinism. If you need to consistently resolve to the same model (for example, relying on the upstream's prompt cache, or requiring strict reproducibility), **specify a concrete model name directly**, or use the **Key to limit the allowed range to a single model**—under both approaches the resolution is deterministic.

***

## Related resources

* [Model Mapping and Fallback](https://docs.aihubmix.com/en/api/Model-Mapping-Fallback): Key-level fixed aliases + failure fallback, complementary to the LLM Router.
* [Unified Inference Parameters](https://docs.aihubmix.com/en/api/unified-inference): consistent request parameters across models.
* [AIHubMix models page](https://aihubmix.com/models): look up model names, prices, and `Input Modalities`.
* [LLM Router Model Scope](https://docs.aihubmix.com/en/api/RouterEndpoints/leaderboard): login-free access to the public 23-sub-dimension subset of the 30+ routing dimensions, plus the routable model pool.