Parsing SSE Token Streams for Reliable Live Replies

§1 Live replies arrive in layers, not complete messages

A streaming AI response looks like text appearing a little at a time. Underneath, your application receives network bytes, reads Server-Sent Events (SSE), parses JSON, and extracts pieces of the assistant’s reply. Treating those layers as interchangeable causes many beginner bugs.

This primer covers the OpenAI-style Chat Completions streaming format. Other interfaces, including the Responses API, use different event structures. Check which interface your client calls before choosing a parser.

A network chunk is not an SSE event, and an SSE event is not necessarily one token. A single network read might contain half an event or several events together. A content update might contain a word fragment, punctuation, or multiple words.

§2 SSE boundaries need a persistent buffer

When reading the raw response body, start with an incremental UTF-8 decoder. Some characters occupy multiple bytes, and a network read can split those bytes. Decoding every read independently can corrupt text before JSON parsing even begins.

Keep decoded text in a persistent buffer. Read complete lines while retaining unfinished text for the next read. SSE events end at a blank line; support both LF and CRLF line endings rather than assuming every server uses the same separator.

• – Collect the contents of data: lines within each event, removing the optional single space after the colon.
• – Join multiple data lines with newline characters before interpreting the event payload.
• – Ignore SSE comment lines beginning with a colon, which can serve as keepalives.
• – Preserve incomplete lines and events until more bytes arrive.

Once an event is complete, check its data payload. In this streaming format, [DONE] is a completion sentinel, not JSON. Recognize it before attempting JSON parsing. Parse ordinary data payloads as JSON and handle failures explicitly.

§3 Chat Completions deltas build the assistant message

For ordinary chat text, inspect each item in choices and read delta.content when it contains a string. Append that string to the accumulated reply. Do not replace the whole reply with each update, and do not expect the non-streaming message.content structure.

The first delta may only announce the assistant role. Later deltas may contain text, tool-call updates, or no visible content. An empty delta is not automatically an error. When supporting multiple choices, maintain separate state for each choice index rather than combining their text.

For a responsive interface, display accumulated text as it arrives. You can batch screen updates briefly to reduce rendering work without changing the order of received content. Render model output safely as text unless you have a deliberate sanitization strategy.

§4 Tool-call arguments need their own assembly

Tool calls are structured output, not chat text. Their function arguments commonly arrive as fragments of a JSON string. A fragment alone may be invalid JSON, even when the completed argument string will be valid.

Track tool calls by their index within each choice, preserving identifiers and function names when supplied. Append argument fragments in order. Parse the completed arguments only after assembly, then validate them against your tool’s expected schema. Never execute a tool just because an early fragment looks usable.

§5 Finish reasons differ from stream completion

A choice’s finish_reason explains why that choice stopped, such as a normal stop, a length limit, or a tool-call handoff. It does not necessarily mean no more events will arrive. When supported and requested, a usage chunk can follow and may have an empty choices array.

Keep reading through the stream’s completion signal. If the connection disappears early, preserve the partial reply but mark it interrupted. Blindly retrying and appending the new response can duplicate text; recovery needs an explicit application policy.

§6 SDKs and Fluxio simplify the transport layer

An OpenAI SDK generally handles SSE framing and exposes parsed chunks through an iterator. You still need to assemble deltas and handle interruption. Fluxio fits applications that want a streaming-first, pay-per-token relay with one key across providers. OpenAI-style clients use https://api.fluxioapi.com/v1 as the base URL.

Sign up for Fluxio to build your next live reply.