Handling Reconnects and Partial Streaming Messages

A resilient streaming interface should preserve useful output without pretending an interrupted answer is complete. Build recovery around three separate concerns: decoding incoming bytes, assembling application messages, and restoring delivery after a disconnect. This walkthrough uses an AI chat interface as the working example.

§1 Step 1: Define Message Identity and Completion

Before connecting, define an application event contract. Give each generation a stable identifier and each replayable event an increasing sequence number. Track message status separately from connection status: connecting, streaming, interrupted, completed, and canceled are useful starting states.

Require an explicit application-level completion signal. Do not treat a closed connection as proof that the answer finished; native server-sent event connections can close and reconnect as part of their transport lifecycle. Map your upstream provider’s documented terminal signal into your own completion event. ([html.spec.whatwg.org](https://html.spec.whatwg.org/multipage/server-sent-events.html))

§2 Step 2: Decode Bytes Before Parsing Events

For a fetch-based client, pass response bytes through TextDecoderStream before processing text. This API converts a binary stream into strings, keeping decoding separate from event parsing. Maintain a text buffer across reads rather than treating each read as a complete message. ([developer.mozilla.org](https://developer.mozilla.org/en-US/docs/Web/API/TextDecoderStream))

Next, feed decoded text into a standards-compliant SSE parser. Dispatch events only after their terminating blank line, support the specified line endings, and preserve unfinished input between reads. Join multiple data fields with newlines before interpreting their combined payload. Parse JSON only after the complete event has been assembled. ([html.spec.whatwg.org](https://html.spec.whatwg.org/multipage/server-sent-events.html?utm_source=openai))

§3 Step 3: Apply Complete Deltas to Partial Answers

Keep incomplete transport data in the parser buffer, not in the visible answer. Once a complete application event arrives, validate its generation identifier and sequence number, then append its text delta to the matching message.

For this design, store the accumulated answer and last applied sequence together. For example, after applying event 42, commit both the updated text and cursor 42. Ignore replayed events through 42, accept 43 next, and request recovery if 44 arrives first. Never deduplicate by comparing text: repeated words may be intentional.

§4 Step 4: Resume Delivery Without Restarting Generation

Native EventSource reconnects automatically and sends Last-Event-ID when it has an event identifier. That identifier gives your server a recovery cursor, but your application must implement retained events and replay. The protocol does not create an event history for you. ([html.spec.whatwg.org](https://html.spec.whatwg.org/multipage/server-sent-events.html))

For authenticated POST-based generation, use a fetch stream or SDK transport rather than assuming native EventSource can carry the same request. Its constructor accepts a URL and credential option, not an arbitrary request body or authorization header. Implement reconnection explicitly for your chosen transport. ([html.spec.whatwg.org](https://html.spec.whatwg.org/multipage/server-sent-events.html))

Design your backend to retain generation events independently of the browser connection. Reconnect to that generation with the last applied cursor. If replay is unavailable or the upstream generation has stopped, preserve the partial answer and offer a clearly labeled restart instead of silently appending a new generation.

§5 Step 5: Bound Retries and Respect Cancellation

For application-managed recovery, choose a capped exponential backoff with jitter and a maximum retry budget. Keep this policy separate from native EventSource behavior, where the SSE retry field specifies a reconnection interval in milliseconds. ([developer.mozilla.org](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events))

• – Keep partial text visible while displaying a reconnecting status.
• – Stop automatic retries when the user cancels the generation.
• – Surface authentication failures instead of repeatedly reconnecting.
• – Show an explicit recovery action when the retry budget expires.

§6 Step 6: Keep the AI Relay Behind Your Recovery Layer

Fluxio fits the upstream role when you want a streaming-first, pay-per-token AI API relay with OpenAI SDK compatibility. Keep its API key on your backend, normalize incoming deltas into your application events, and let your own recovery layer handle browser reconnects.

Do not assume SDK compatibility guarantees resumable generations. Make replay support an explicit integration requirement, and distinguish reconnecting to retained output from issuing another potentially billable generation request.

§7 Step 7: Test Disconnects at Event Boundaries

Test split characters, fragmented JSON payloads, duplicate events, sequence gaps, expired cursors, and cancellation during backoff. Disconnect immediately before and after the completion event. Verify that recovery preserves exact text, never marks interrupted output complete, and never starts another generation without the intended policy.

Sign up for Fluxio.