Create realtime session

POST 
/v1/realtime/sessions

Start a voice session for the calling end user — or, with session_id, rejoin one of theirs that has not ended — and return its id with a short-lived client_secret.

Open a WebSocket to websocket_url before client_secret.expires_at, with the secret in the Authorization: Bearer header. A browser, which cannot set that header, offers two subprotocols instead: neweyes-realtime and neweyes-secret. followed by the secret. A secret opens one connection.

The voice session is a session like any other: read it, its transcript and its brief through /v1/sessions. A closed connection does not end it: to go on with the call, create a realtime session with its session_id for a fresh secret and connect again — the conversation picks up where it left off. One connection holds a session at a time; a newer one takes it over. End the session with POST /v1/realtime/sessions/\{session_id\}/end. One left without a connection ends by itself, though never sooner than 10 minutes after its last connection closed — a time that may grow but will not shrink. A rejoin after that is refused with session_ended: start a new session. Its brief is built once it ends.

The voice assistant may call two tools, delivered as tool_call frames for your client to run and answer with a tool_result:

• create_task — send its task text as a message to a text session (POST /v1/sessions, or POST /v1/sessions/\{session_id\}/messages on one you keep for the call) and answer with the returned task_id. The task's progress and reply arrive on GET /v1/sessions/{session_id}/events.
• get_task_status — answer with what that event stream has shown for the task: still running, or its end (completed with its reply, stopped, or failed).

A new voice session opens the capabilities you choose with capabilities or exclude_capabilities, as a text session does, and keeps them for its life; a rejoin keeps them too, so it sends neither field. The voice assistant is told what the session left out, so it does not offer or delegate it — but the text session a task runs in is yours to create, and the call's choice does not carry over to it: create it with the same capabilities or exclude_capabilities, or it opens every capability. A call whose capabilities cannot be read does not start: it closes with 1011.

A frame your client sends is dropped, with no reply, when it is over 4 MiB, when its audio or image data is not base64, when an input_text is over 32,000 characters or holds a NUL character, or when a tool_result's output is over 32,000 characters as JSON.

The socket has no request/response body, so its outcome is reported by the WebSocket close code:

• 1000 — normal closure, or the session was ended.
• 1001 — going away; one side is ending the session (may be relayed when the underlying session terminates).
• 1011 — a server-side error, or the voice service is temporarily unavailable; retry later. A close from the voice service with a code not listed here arrives as this one.
• 1012 — the server is restarting for a planned redeploy; reconnect shortly.
• 4001 — the secret is missing, invalid, expired or already used, its session was deleted or has ended, or the caller's access was revoked; get a fresh secret (with the same session_id, while the session has not ended).
• 4009 — a newer connection took the session over; do not reconnect from this one, or the two take it from each other in turn.
• 4401 — authentication failed, or the account is not active.
• 4429 — the billing account is out of credits, or too many voice sessions are already active; top up or close one, then retry.

Request

Responses

200

Successful Response

401

Authentication failed: the credential is missing, malformed, expired, or revoked.

• invalid_token — The access token is invalid.
• project_disabled — This project is not active.
• token_expired — The access token has expired.
• user_disabled — This user is not active.

404

The requested resource does not exist, or is not visible to the caller.

• session_not_found — Session not found

409

The request conflicts with the current state of the resource.

• session_not_realtime — This session is not a voice session.
• session_ended — This voice session has ended; start a new one.

413

The request body is larger than the allowed limit.

• payload_too_large — The request body is too large.

422

A request value failed validation; the details array names each offending field.

• validation_failed — The request payload failed validation.

429

Either too many requests (a Retry-After header says when to retry) or the billing account is out of credits (no Retry-After — top up to continue); the code field tells the two apart.

• insufficient_credits — The billing account has insufficient credits.
• rate_limited — Too many requests. Please slow down.

500

An unexpected server-side error occurred.

• internal — An unexpected error occurred.