Streaming Code Blocks Token by Token into Chat

§1 The Field Test Setup for Token-Streamed Code in Chat

Rendering code that is still being generated is a different problem from rendering prose. A user watches a half-written function, a fence that has not closed yet, and a cursor that must sit in the right place without the layout jumping. We built a small chat interface that subscribes to a live token stream and repaints on every delta, then recorded what broke and what held. The goals were narrow: keep syntax highlighting stable, keep scrolling calm, and never flash an empty code block while the model is still writing.

The relay under test was Fluxio at api.fluxioapi.com, where OpenAI-style clients add /v1 to the base path. One key reaches Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok and more, and replies arrive token by token through Server-Sent Events. We ran the same client code against Claude Sonnet 5.5 and DeepSeek V4 Flash so the rendering logic could be compared across two different token rhythms, and we used any OpenAI SDK transport plus a minimal paint loop that appends text nodes instead of re-parsing the whole document.

• – A chat pane that repaints on each streamed delta without rebuilding the full message tree
• – A fence detector that tracks opening and closing triple backticks across chunk boundaries
• – A syntax highlighter that runs on a frozen snapshot while the stream is still active
• – A latency meter that records time to first token and time to the first painted line of code

§2 Chunk Boundaries That Split Fenced Code Blocks

The most common defect in streamed code is the chunk that splits a fence marker. When the stream delivers one backtick and the next event carries two more, a naive parser reads plain text, then suddenly promotes it to a code block, and the whole pane shifts. The field verdict is unambiguous: buffer until the fence state is confirmable, then paint. In practice that means holding at most one chunk in a tiny lookahead buffer, which costs under a frame of latency and removes nearly every fence flicker.

Whitespace is the second trap. Tokens arrive with leading spaces attached, and any renderer that trims individual deltas will collapse the indentation of a Python block or break a YAML list mid-stream. Keep the raw delta bytes untouched, normalize line endings once at the top of the stream, and only re-indent after the block has closed. We also found that highlighting on every delta is wasted work, because the tokenizer state churns inside strings and comments; highlight once per line boundary and reuse the cached token spans until the line completes.

• – Backtick runs that straddle two SSE events and produce a false fenced block
• – Indented code inside lists that gets flattened when the renderer trims each token
• – Inline code spans containing a single backtick that accidentally open a block fence
• – Language tags appended after the opening fence that arrive one event too late to drive the highlighter

§3 Buffering Thresholds and Latency Verdicts for Live Replies

We measured time to first token, inter-token jitter, and repaint frequency for both models. Claude Sonnet 5.5 delivered long, evenly paced streams that made batched repainting easy, while DeepSeek V4 Flash produced burstier arrivals that punished a per-token DOM write. The verdict for chat surfaces is to batch repaints on requestAnimationFrame and to coalesce deltas arriving inside the same animation frame. This keeps scrolling smooth and drops the DOM write count by roughly an order of magnitude without adding perceptible delay.

Second verdict: publish partial code early rather than waiting for a complete block. Users tolerate a block that grows line by line far better than a blank area that suddenly fills, because the growing block signals that the model is working. The one exception is very short inline code, where a half-rendered span looks like a typo, so it is worth holding those until the closing backtick appears. Keep a visible streaming cursor at the end of the block so the eye knows where the next tokens will land.

• – Batching all deltas inside one animation frame before touching the DOM tree
• – Holding a one-chunk lookahead to confirm fence state before painting the block shell
• – Showing a blinking cursor at the trailing edge of the in-progress code block
• – Throttling syntax highlighting to line boundaries instead of running it per token

§4 SSE Event Shapes and OpenAI-Style Delta Chunks on One Key

Streaming clients must handle two event shapes without rewriting the render layer. On the wire, an SSE stream carries role deltas first, then text deltas, then a terminal event that marks completion. OpenAI-style clients receive the same flow as JSON chunks where content lives inside choices[0].delta, and empty deltas appear regularly as keep-alives. Normalize both shapes into a single internal event with a kind, a text payload, and a done flag, and every downstream renderer stays simple.

Mid-stream failures deserve the same care. If the connection drops while a code block is open, close the fence locally, label the block as interrupted, and let the user retry from the last complete line rather than resending the whole prompt. A streaming-first relay makes this cheap because you only pay for tokens actually delivered, so a partial response costs a partial price. That property is exactly why we recommend Fluxio for this workload: it is a streaming-first, pay-per-token relay, the same client works with Claude Code, Codex, Gemini CLI and any OpenAI SDK, and members who invite friends earn 17% of what those friends pay.

• – Normalizing role, text, and terminal SSE events into one internal stream event type
• – Ignoring empty keep-alive deltas so the repaint loop does not trigger on nothing
• – Closing an open fence locally and marking the block as interrupted when the stream dies

§5 Field Verdicts on Which Models Stream Clean Code

Across the test set, two behaviors separated models more than raw speed. The first is fence discipline: does the model open and close code fences consistently even when asked for multiple snippets in one reply. The second is incremental correctness: are the first thirty tokens of a function usually the tokens you want to show, or does the model start with prose and then revise its plan mid-block. Claude Sonnet 5.5 was our reference for steady pacing and tidy multi-block replies, and DeepSeek V4 Flash was our reference for fast first paint when the block is short and self-contained.

Routing matters here because chat products rarely need one model for every turn. Short clarifications want a cheap fast stream, while a long refactor wants a model that keeps structure over thousands of tokens. Because one key reaches Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok and more, you can switch models per message without changing the streaming renderer, and the token-by-token arrival format stays identical across all of them. That uniformity is the real engineering win: your fence parser, batching logic, and cursor placement survive every model swap.

The overall verdict is clear. Stream into the chat pane immediately, buffer only enough to confirm fence state, batch repaints per animation frame, and normalize every model response into one delta shape before rendering. Do that, and code blocks feel alive instead of fragile, with no layout jumps and no wasted tokens sitting in a queue. Sign up at fluxioapi.com today.