Skip to main content

Streaming

Receive output as it is generated with server-sent events, for chat completions, Responses and Messages, and handle keepalives and in-band errors.

On this page

Set stream: true and the response becomes a text/event-stream. Events arrive as data: lines separated by blank lines.

Chat completions

Each event is a chat.completion.chunk. The stream ends with data: [DONE].

Chat completion stream
data: {"id":"chatcmpl-nebula","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"role":"assistant","content":"An IDOR"}}]}

data: {"id":"chatcmpl-nebula","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" lets a user reach"}}]}

data: {"id":"chatcmpl-nebula","object":"chat.completion.chunk","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Tool calls stream as delta.tool_calls fragments. Concatenate arguments per call index before parsing them as JSON.

Responses

Responses streams use named events: response.created, then response.output_item.added, response.output_text.delta and the matching .done events, one function_call item per tool call, and finally response.completed. If the request fails after the stream starts, the last event is response.failed.

Messages (Anthropic-compatible)

The Messages API streams Anthropic events: message_start, content_block_start, content_block_delta, content_block_stop, message_delta, message_stop, and periodic ping events. Anthropic SDKs handle them unchanged.

Keepalives

While the model is thinking or between chunks, the server sends SSE comment lines to keep connections open through proxies:

Keepalive comment
: keepalive

Comment lines start with a colon. Standard SSE clients ignore them; if you parse the stream yourself, skip any line that starts with :.

Errors after the stream starts

The HTTP status is already 200 once streaming begins, so a later failure arrives in-band as a final data: event with an error object, followed by data: [DONE]. See Errors.

Practical advice

  • Set a read timeout longer than the keepalive interval. Keepalives arrive about every 12 seconds while a stream is idle.
  • Treat a stream that closes without [DONE] (or message_stop, or response.completed) as incomplete and retry with an Idempotency-Key.

Search documentation

Type to search guides, the API reference, the CLI and the console guide.

    Enter open↑ ↓ moveEsc close