> ## Documentation Index > Fetch the complete documentation index at: https://docs.aihubmix.com/llms.txt > Use this file to discover all available pages before exploring further. # Realtime Speech Transcription > Establish a persistent connection over WebSocket and get text as you speak, for low-latency streaming speech-to-text ## Introduction Realtime speech transcription establishes a persistent connection over WebSocket (a protocol that keeps a long-lived, bidirectional connection between client and server), receiving a continuous audio stream and **transcribing and returning results as the audio arrives**. It suits latency-sensitive voice scenarios. How it differs from file transcription [STT](/en/api/STT): | Dimension | File transcription (STT) | Realtime transcription (this page) | | --------- | --------------------------------------------- | -------------------------------------------------------------- | | Protocol | HTTP, one request returns the complete result | WebSocket, continuously pushes incremental results | | Input | Complete audio file (≤25MB) | Continuous audio stream (PCM chunks) | | Latency | Waits for the whole segment to finish | Returns text while you are still speaking | | Use cases | Recording transcription, subtitle generation | Live meeting captions, voice assistants, live-stream dictation | **Available model:** * **gpt-live-transcribe**: streaming transcription model, supports multiple languages, and outputs transcribed text in realtime as audio arrives. **This API is designed for server-side integration; browsers cannot connect directly.** For security reasons, the gateway validates and rejects connections that carry an `Origin` header, rejects the `openai-insecure-api-key` subprotocol, and accepts the key only through the standard `Authorization` header. A browser-initiated WebSocket automatically attaches an `Origin` header and is therefore rejected. To do realtime transcription in a frontend, establish the connection to the gateway from your own server and forward the results to the frontend. ## Quick Start ### Connection endpoint ```text theme={null} wss://aihubmix.com/v1/realtime?intent=transcription&model=gpt-live-transcribe ``` * `intent=transcription`: **required**, declares this is a transcription session. * `model=gpt-live-transcribe`: **required**. The model is fixed by URL parameters at connection time and cannot be changed during the session (see constraints below). ### Authentication Pass the key through a standard HTTP header during the handshake: ```text theme={null} Authorization: Bearer $AIHUBMIX_API_KEY ``` ### Audio format requirements Only one input format is currently supported. Convert your audio before sending: * **Encoding**: PCM16 (16-bit signed integer, little-endian) * **Sample rate**: 24000 Hz * **Channels**: mono That is, `audio/pcm@24000`. Sending any other format (such as G.711/µ-law) is rejected and the session is closed. **Transcription sessions do not support voice activity detection (turn\_detection / VAD)** and it must be explicitly set to `null`. If it is omitted or set to a non-`null` value, the model provider rejects transcription with `invalid_value`. The gateway forces `turn_detection` to `null` in the forwarded configuration, but we still recommend setting it to `null` explicitly on the client for clear behavior. ## Session Configuration (session.update) After the connection is established, the client first sends a `session.update` frame to configure transcription parameters. If you do not send one, the gateway injects a default configuration with the authorized model as a fallback, but explicit configuration is recommended. ```json theme={null} { "type": "session.update", "session": { "type": "transcription", "audio": { "input": { "format": { "type": "audio/pcm", "rate": 24000 }, "transcription": { "model": "gpt-live-transcribe", "languages": ["en", "zh"], "prompt": "会议录音，含产品名与英文缩写", "keywords": ["AiHubMix", "gpt-live-transcribe"], "delay": "low" }, "turn_detection": null, "noise_reduction": { "type": "near_field" } } } } } ``` ### Configuration parameters Session type, fixed to `transcription` for transcription scenarios. Input audio format, fixed to `{ "type": "audio/pcm", "rate": 24000 }`. Transcription model. Must match the `model` in the connection URL (`gpt-live-transcribe`). Passing any other model is treated as unauthorized and the session is closed with `1008`. Expected language list, in array form (such as `["en", "zh"]`). `gpt-live-transcribe` uses the **plural** `languages`, letting you declare multiple languages at once; specifying languages improves accuracy and lowers latency. See the value dictionary in [Language codes](#language-codes) below. The singular form, a single ISO-639-1 code (such as `"en"`). Use either this or `languages`, **not both** (passing both is rejected with `invalid_value`). The plural `languages` is recommended for `gpt-live-transcribe`; the gateway also accepts the singular `language` to ease migration from older code. A free-text prompt describing the recording scenario (such as "customer service call" or "medical consultation with clinical terms") to help the model match the register. In testing, the server echoes it verbatim in `session.updated`, confirming it has taken effect. An array of literal hint words for product names, abbreviations, proper nouns, and other error-prone terms (such as `["AiHubMix", "gpt-live-transcribe"]`). These are **hints**, not forced output; put each word as a separate item and avoid including `<`, `>`, or newlines. In testing, they are echoed and take effect. Latency / accuracy tier, with options `minimal`, `low`, `medium`, `high`, `xhigh`. A higher tier is more accurate but adds latency. **Note**: the gateway accepts this field (no error), but in testing it is not echoed in `session.updated`, so its effectiveness follows the official documentation and is not yet confirmed by echo. Voice activity detection. Must be `null` for transcription sessions. Optional noise reduction configuration, such as `{ "type": "near_field" }` (near-field, suited to a microphone close to the speaker) or `{ "type": "far_field" }` (far-field). ### Language codes Values for `languages` / `language` follow the formats below. They are **case-sensitive and must be one of the supported forms**; passing an unsupported or malformed code is rejected by the realtime API: | Category | Examples | Notes | | ---------------------------------- | ---------------------------- | ----------------------------------------------------------------------------------- | | ISO 639-1 (two letters) | `en`, `zh`, `es`, `fr`, `ja` | Most common, one two-letter code per language | | Selected ISO 639-3 (three letters) | `eng`, `spa`, `yue`, `cmn` | Used to distinguish dialects, e.g., `yue`=Cantonese, `cmn`=Mandarin | | Regionalized Chinese | `zh-cn`, `zh-tw`, `zh-hk` | Language + region, distinguishing simplified/traditional and Hong Kong/Taiwan usage | When using the plural `languages`, list the most likely languages first. For mixed-language scenarios (such as Chinese-English code-switching), you can write `["zh", "en"]`; for a single language, just write `["en"]`, which is more accurate and faster than not specifying one. ## Sending Audio Split PCM16 audio into small chunks (such as one chunk per 100ms), base64-encode them, and send them continuously via `input_audio_buffer.append` events: ```json theme={null} { "type": "input_audio_buffer.append", "audio": "" } ``` Because transcription sessions do not enable VAD (voice activity detection), the server does not automatically determine when a segment of speech ends. After sending a segment of audio, you must **manually** send an `input_audio_buffer.commit` frame to mark the end of that segment, which triggers transcription finalization and returns the `completed` result: ```json theme={null} { "type": "input_audio_buffer.commit" } ``` ## Receiving Transcription Results The server continuously pushes events. Key event types: Confirmation of session creation and configuration update. **Incremental** transcription result; the `delta` field is the newly added text fragment for this update. Returned as you speak, suitable for realtime display. A segment of speech transcription is **complete**; the `transcript` field is the full text of that segment. Error event, containing the error code and description. ## Full Examples Two approaches are shown below; choose either one: * **OpenAI official SDK (recommended)**: no need to hand-write WebSocket code, just point `websocket_base_url` (the SDK's WebSocket base-address parameter) at the gateway to reuse the official library. * **Native websockets**: no SDK installed, sending and receiving frames directly per the protocol, with minimal dependencies and easy troubleshooting. **Why does the official demo not pass the model name, but we do?** OpenAI's transcription intent puts the model in `transcription.model` of `session.update`, and the connection URL only carries `?intent=transcription`. The AiHubMix gateway is different: the model name **must** appear in the handshake URL (`?model=gpt-live-transcribe`), because the gateway needs it at the moment of the WebSocket handshake to **select the model provider, authenticate, and reserve quota**, while `session.update` only arrives after the handshake completes, too late. So when using the SDK, pass `model` explicitly to `connect()` (the SDK appends it to the URL query); without it, the gateway returns **`400 missing_model_parameter` during the handshake**, `connect()` throws an exception, the connection is never established, and you never reach the `session.update` step. Within the session, `transcription.model` still needs to match the URL. ```python Python (OpenAI SDK) theme={null} # Dependency: pip install "openai[realtime]" import asyncio import base64 from openai import AsyncOpenAI # Reusing the official SDK requires only three adaptations: # 1) point websocket_base_url at the AiHubMix gateway (instead of OpenAI's default address) # 2) pass model explicitly to connect() -- see the Note above, required for the gateway handshake # 3) include intent=transcription in extra_query client = AsyncOpenAI( api_key="sk-***", # Replace with your AiHubMix API key websocket_base_url="wss://aihubmix.com/v1", ) async def main(): async with client.realtime.connect( model="gpt-live-transcribe", # required: goes into the handshake URL, used by the gateway for routing + billing extra_query={"intent": "transcription"}, ) as conn: # 1) Configure the transcription session await conn.session.update(session={ "type": "transcription", "audio": {"input": { "format": {"type": "audio/pcm", "rate": 24000}, "transcription": { "model": "gpt-live-transcribe", "languages": ["en", "zh"], "prompt": "会议录音,含产品名与英文缩写", "keywords": ["AiHubMix", "gpt-live-transcribe"], }, "turn_detection": None, "noise_reduction": {"type": "near_field"}, }}, }) # 2) Read local PCM16 / 24kHz / mono raw audio and send it in chunks async def send_audio(): with open("audio_pcm16_24k.raw", "rb") as f: pcm = f.read() chunk = 24000 * 2 // 10 # 100ms = sample rate x 2 bytes / 10 for i in range(0, len(pcm), chunk): await conn.input_audio_buffer.append( audio=base64.b64encode(pcm[i:i + chunk]).decode() ) await asyncio.sleep(0.1) # simulate realtime pacing # no VAD, manually commit after all audio is sent to trigger transcription finalization await conn.input_audio_buffer.commit() asyncio.create_task(send_audio()) # 3) Receive transcription results async for evt in conn: etype = getattr(evt, "type", "") if etype.endswith("transcription.delta"): print(getattr(evt, "delta", ""), end="", flush=True) elif etype.endswith("transcription.completed"): print("\n[completed]", getattr(evt, "transcript", "")) break # exit once the full result is received elif etype == "error": print("\n[error]", evt.to_dict()) break asyncio.run(main()) ``` ```python Python (websockets) theme={null} import asyncio import base64 import json import websockets API_KEY = "sk-***" # Replace with your AiHubMix API key URL = ( "wss://aihubmix.com/v1/realtime" "?intent=transcription&model=gpt-live-transcribe" ) async def main(): # websockets >= 13 uses additional_headers; older versions use extra_headers async with websockets.connect( URL, additional_headers={"Authorization": f"Bearer {API_KEY}"} ) as ws: # 1) Configure the transcription session await ws.send(json.dumps({ "type": "session.update", "session": { "type": "transcription", "audio": {"input": { "format": {"type": "audio/pcm", "rate": 24000}, "transcription": {"model": "gpt-live-transcribe", "language": "en"}, "turn_detection": None, "noise_reduction": {"type": "near_field"}, }}, }, })) # 2) Read local PCM16 / 24kHz / mono raw audio and send it in chunks async def send_audio(): with open("audio_pcm16_24k.raw", "rb") as f: pcm = f.read() chunk = 24000 * 2 // 10 # 100ms = sample rate x 2 bytes / 10 for i in range(0, len(pcm), chunk): await ws.send(json.dumps({ "type": "input_audio_buffer.append", "audio": base64.b64encode(pcm[i:i + chunk]).decode(), })) await asyncio.sleep(0.1) # simulate realtime pacing # no VAD, manually commit after all audio is sent to trigger transcription finalization await ws.send(json.dumps({"type": "input_audio_buffer.commit"})) asyncio.create_task(send_audio()) # 3) Receive transcription results async for msg in ws: evt = json.loads(msg) etype = evt.get("type", "") if etype.endswith("transcription.delta"): print(evt.get("delta", ""), end="", flush=True) elif etype.endswith("transcription.completed"): print("\n[completed]", evt.get("transcript", "")) break # exit once the full result is received elif etype == "error": print("\n[error]", evt.get("error")) break asyncio.run(main()) ``` ```bash Connection test (wscat) theme={null} # Use wscat to quickly verify connectivity and authentication (first run npm i -g wscat) wscat -c "wss://aihubmix.com/v1/realtime?intent=transcription&model=gpt-live-transcribe" \ -H "Authorization: Bearer $AIHUBMIX_API_KEY" # After connecting, paste a session.update frame to begin (audio must be base64-encoded yourself and sent via input_audio_buffer.append) ``` To convert any audio to the raw PCM format this API requires, use ffmpeg: ```bash theme={null} ffmpeg -i input.mp3 -f s16le -acodec pcm_s16le -ac 1 -ar 24000 audio_pcm16_24k.raw ``` ## Live Run Results (production test) The following are the actual results of running the above example in the `aihubmix.com` production environment (model `gpt-live-transcribe`), configured with `languages: ["en", "zh"]` + `prompt` + `keywords` + `delay: "low"` + `noise_reduction: { "type": "near_field" }`: ```text theme={null} # 1) Server echo confirmation (session.updated): languages / prompt / keywords / noise_reduction are all echoed verbatim session.updated session.audio.input.transcription = { "model": "gpt-live-transcribe", "language": null, "languages": ["en", "zh"], "keywords": ["AiHubMix", "gpt-live-transcribe"], "prompt": "Product demo recording, contains the brand name AiHubMix." } noise_reduction = { "type": "near_field" } turn_detection = null # Note: the delay field sent in the request does not appear in the echo # 2) After the audio (PCM16 / 24kHz / mono) is sent in chunks and committed, the transcription returns incrementally word by word (delta), with the full text at the end (completed) Hello, this is a real-time transcription test for AIHubMix. The weather is really nice today [completed] Hello, this is a real-time transcription test for AIHubMix. The weather is really nice today ``` In testing, `languages`, `prompt`, `keywords`, and `noise_reduction` are all echoed verbatim by the server in `session.updated`, showing that the configuration has actually taken effect (it is accepted and processed, not merely accepted). The `delay` field is accepted by the gateway but not echoed, so its effectiveness follows the official documentation; you can pass only one of `language` (singular) and `languages` (plural). ## Billing * **Unit price**: `gpt-live-transcribe` is billed at **\$0.017 / minute** (the realtime listed price on the [model detail page](https://aihubmix.com/model/gpt-live-transcribe) is authoritative). * Billed by the **duration of audio transcribed**: based on the actual audio seconds forwarded to the transcription model, rounded up to the whole second. For example, transcribing 90 seconds of audio is billed as `90 / 60 x $0.017 = $0.0255`. * Billing is not affected by network round trips or idle waiting; only audio actually fed into transcription is timed. * **Pay-as-you-go settlement**: this is a long-lived connection, so the cost is settled incrementally in realtime during the session rather than all at once when the session ends. When the session is established, a **quota reservation** is first made for about 1 minute of usage (only an admission check, not an actual charge); the reservation is rolled over every 20 seconds during the session, the actual cost is deducted in segments based on the real forwarded seconds, and any remaining reservation is released when the session ends. **Therefore your available balance must cover at least about 1 minute of usage for the session to be established.** * In the consumption details under [Usage and Billing](https://aihubmix.com), the **note** on each realtime transcription record shows the "per-minute unit price" and the "actual billed seconds for this session", for easy line-by-line reconciliation. Billing record for a gpt-live-transcribe realtime transcription in the Usage and Billing Activity view, with a note showing 7s of audio billed at $0.017 per minute ## Limits and Constraints 1. **Single session duration**: a single WebSocket connection lasts at most **62 minutes**, after which the server closes it proactively; if you need longer, reconnect in segments. 2. **Insufficient balance**: there are two cases. **When establishing the session**, if the available balance is not enough to cover the roughly 1-minute reservation, the handshake is rejected directly (HTTP 403) and the session is not established. **During the session**, if the balance runs out (detected by the 20-second rollover check or by a recheck after segment deduction), the established connection is closed immediately. 3. **Server-side only**: direct browser connections are not supported (the `Origin` header is validated); integrate on the server side. 4. **Model lock**: the model is fixed in the connection URL; changing the model via `session.update` during the session is rejected and the session is closed. 5. **Format lock**: only `audio/pcm@24000` mono is supported; other formats are rejected. ## Common Errors | Scenario | Close code / status | Description | | ---------------------------------------------------- | ---------------------------------- | -------------------------------------------------------------------------- | | Service not enabled | HTTP 403 `realtime_disabled` | Realtime transcription is not open for this environment | | Carrying `Origin` header / direct browser connection | Handshake rejected | Switch to a server-side connection | | Changing the model during the session | `1008` `model_override_forbidden` | The model can only be specified in the connection URL | | Audio format is not PCM | `1008` `audio_format_unsupported` | Convert to `audio/pcm@24000` | | Insufficient balance when connecting | HTTP 403 `insufficient_user_quota` | Balance is not enough to reserve about 1 minute of usage; top up and retry | | Balance runs out during the session | Connection closed | Top up and reconnect | *** Last updated: 2026-09-16