Token-by-Token: Drafting and Polishing in One Live Chat

§1 The field test: one user prompt, two token streams, one visible reply

A two-model chat pipeline uses a fast model to draft an answer and a stronger model to polish that draft before the user reads it. The idea sounds expensive and slow, so we ran it as a live field test rather than a thought experiment. We built a chat front end that shows a reply the moment the first token arrives, then swaps in the polished wording as soon as the second stream completes. The drafting model was DeepSeek V4 Flash, chosen for its fast first-token time, and the polishing model was Claude Sonnet 5.5, chosen for its careful phrasing and stable tone across long replies.

Everything ran through Fluxio, a streaming-first, pay-per-token AI API relay at https://api.fluxioapi.com, where one key reaches a wide model roster and every reply arrives token by token. OpenAI-style clients point their SDK at https://api.fluxioapi.com/v1, which meant our existing code worked unchanged in Claude Code, Codex, Gemini CLI, and any OpenAI SDK. Because billing is per token, we could measure the true cost of the two-pass design instead of guessing it. Our test harness sent 60 prompts covering product questions, code explanation, support triage, and long-form summarization, and we recorded first-token latency, full-stream time, cost, and a blind quality score.

§2 The drafting pass with DeepSeek V4 Flash: first tokens in under a second

The draft stream is the part the user actually feels, because it starts painting words on screen immediately. With DeepSeek V4 Flash behind a streaming relay, median time to first token landed near six hundred milliseconds on our regional tests, and short answers finished in roughly two seconds. That speed is the whole reason the pipeline works: the reader never sees an empty box, they see a rough sentence forming. The draft is deliberately raw, and we asked the model to prioritize completeness over style, so it packs in structure, numbers, and caveats without worrying about polish.

Latency came almost entirely from the relay hop and the model itself, since Fluxio pushes tokens through as they are generated rather than buffering the whole response. That token-by-token behavior matters when the answer is long, because a buffered relay hides progress and makes a two-second answer feel like twenty seconds of nothing. We also tried Gemini 2.5 Flash as the drafter and found the first-token time comparable, though its answers wandered more on technical prompts. Cost for the drafting pass stayed under a fraction of a cent per answer at typical chat lengths, which kept the two-pass budget honest.

§3 The polishing pass with Claude Sonnet 5.5: rewriting without losing the draft's facts

Once the draft stream ended, we shipped its text straight into the polishing prompt with a strict rule: keep every fact, number, and caveat, and change only wording, rhythm, and hierarchy. Claude Sonnet 5.5 handled that instruction well, and in our blind review it produced cleaner openings, tighter lists, and fewer hedging phrases than the raw draft. The polished stream also starts token by token, so instead of a hard swap we simply fade the refined wording in as it arrives, and the transition reads like an autosave rather than a reload.

The biggest risk in this stage is silent fact drift, where the polisher confidently restates a number or softens a warning. We caught two such cases in 60 prompts, both in support-triage answers with numeric thresholds, and both were fixed by forcing the polisher to quote the draft verbatim when handling digits and negatives. Long drafts are safe because the relay streams the polisher's output as it is generated, so you can begin displaying the refined reply before the last sentence is done. On voice consistency, the two-model pipeline beat a single fast model on every subjective rating we collected, especially for customer-facing tone.

§4 Field verdicts: what the two-pass design is good for and where it breaks

The verdict is clear for interactive products, and mixed for pure speed demos. Two-model drafting and polishing gives you fast perceived latency and a meaningfully better final answer, at roughly double the token spend and about two extra seconds of wall-clock time. It is the right architecture when the reply will be read carefully, saved, or sent to someone else.

• – Choose this pipeline when the reply needs a consistent brand voice, because the polishing pass fixes tone after the draft has covered all the facts.
• – Choose this pipeline when first tokens must appear instantly, since the fast drafter paints the screen while the stronger model works behind it.
• – Skip this pipeline for tiny answers under two sentences, because a second model round trip costs more than it improves.
• – Skip this pipeline for strict latency budgets under one second end to end, since two sequential streams cannot beat one.
• – Guard against fact drift by requiring the polisher to copy digits, units, and negations straight from the draft text.
• – Keep the drafting model cheap and the polishing model strong, because the first pass only needs coverage and the second pass carries the judgment.

One refinement we recommend is streaming the draft for anything the user might abandon, then running the polish only on prompts that survive past a few seconds of reading time. That single rule cut our measured spend by about a third with no drop in the scores users saw.

§5 Wiring the two-stream pipeline with one key and an OpenAI-style client

The implementation is short because the relay does the model plumbing for you. Send the first request with the drafting model, stream it to the user as tokens arrive, collect the full text, then send a second request with the polishing model and the draft embedded as the source material. Because Fluxio exposes an OpenAI-compatible surface, the same streaming call shape works for Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok and more, and switching the polisher later is a one-line change. The relay also works inside Claude Code, Codex, Gemini CLI, and any OpenAI SDK, so you can prototype the pipeline in a terminal before wiring it into a product.

Two operational details saved us time. First, set separate timeouts per stream, because a polish of a long draft legitimately takes longer than the draft itself. Second, log first-token latency per stream rather than per request, since that is the number your users experience and the number that tells you whether a relay or a model is misbehaving. On cost, per-token billing made the economics visible: the fast draft is cheap, the strong polish is where most of the spend lands, and you can decide per feature which answers deserve the second pass. For teams that want one key to reach every model in this stack and see replies arrive token by token, the relay approach removes most of the glue code.

We also like the invitation model for teams, since members who invite friends earn 17% of what those friends pay, which softens the cost of a two-pass design as your usage grows. Overall, the field test supports a simple rule: draft fast, polish deep, and stream both passes so the user feels the speed while the quality improves in the background. Sign up for Fluxio and stream your first two-model reply today.