An SSE Checklist for Parsing Live AI Replies

§1 Identify the streaming endpoint before parsing chunks

A live AI reply is not one JSON document delivered slowly. For OpenAI-style Chat Completions, it is typically a sequence of server-sent events, or SSE, whose data fields contain incremental JSON payloads. Reliable parsing starts by deciding which protocol and endpoint your client actually uses.

• – Confirm the response format. Check the HTTP status and content type before entering the SSE parser. An authentication failure may return ordinary JSON instead of a stream.
• – Distinguish Chat Completions from Responses. Chat Completions commonly uses choices and delta objects. Responses uses typed events with different payload structures. Do not send both through an identical chunk decoder.

§2 Separate network reads from complete SSE events

A network read is not an SSE event, a JSON object, or a model token. One read may contain several events, while another may stop halfway through a UTF-8 character or JSON string.

• – Decode bytes incrementally. Use a streaming UTF-8 decoder so characters split across reads remain intact.
• – Buffer incomplete lines and events. Recognize SSE line endings and dispatch an event only after its terminating blank line. Never parse each network read as JSON.
• – Apply SSE field rules. Collect data fields, join multiple data lines with newline characters, and ignore comment lines. Preserve event names when the endpoint uses them.
• – Recognize completion sentinels before JSON parsing. In Chat Completions streams that use it, the data value [DONE] is a protocol marker, not JSON.

§3 Assemble delta fields without assuming every chunk contains text

After decoding an SSE event, inspect its payload rather than assuming every event adds visible text. A chunk can announce a role, update a tool call, report completion, or provide usage information.

• – Route updates by choice index. Keep separate state for each choice instead of appending every delta to one shared response.
• – Append text only when content is a string. Missing content, null content, and empty deltas are valid cases. They should not produce visible placeholder text.
• – Keep metadata separate from rendered text. Roles, identifiers, finish reasons, and usage counters belong in structured state, not the chat transcript.

If an SDK already decodes SSE into chunk objects, start at this assembly layer. Adding another SSE parser on top usually creates unnecessary complexity.

§4 Buffer streamed tool arguments until the call is complete

Tool calls require their own accumulator. Their function arguments often arrive as fragments of a JSON string, so an individual chunk may contain incomplete syntax.

• – Track each tool call by its streamed index. Preserve identifiers and function names when supplied, and concatenate argument fragments into that call's buffer.
• – Validate only after argument assembly. Parse the completed arguments, check them against the expected schema, and apply authorization before executing the tool.
• – Keep parallel tool calls isolated. Interleaved fragments must never be combined into one argument document.

§5 Distinguish finish reasons from transport completion

A finish reason describes why a choice stopped generating. It does not necessarily mean the HTTP stream has ended. When supported and requested, a separate usage chunk can arrive afterward, potentially with an empty choices array.

• – Continue reading after a choice finishes. Capture trailing metadata until the endpoint's completion marker or documented terminal event arrives.
• – Treat unexplained disconnects as incomplete. Preserve partial output, but distinguish an interrupted connection from confirmed completion.
• – Make cancellation and retries explicit. Abort the upstream request when appropriate. A retry normally starts a new generation; appending it blindly can duplicate or contradict existing text.

§6 Test relay compatibility with fragmented token streams

Fluxio fits applications that need a streaming-first, pay-per-token AI relay behind OpenAI-style clients. Its API base is https://api.fluxioapi.com, with /v1 added for OpenAI-style clients. Compatibility simplifies integration, but your parser should still follow the selected endpoint's schema.

Test split UTF-8 characters, multiple events per read, role-only chunks, empty choices, fragmented tool arguments, trailing usage, and dropped connections. Compare the final assembled result across different network fragment sizes; correct parsing should produce the same result regardless of packet boundaries.

Sign up for Fluxio to build your next live AI experience.