include_usage for SSE Token Streams and Live Chat Runs

§1 Field test of include_usage on an SSE token stream

Streaming a reply token by token feels effortless until someone asks a finance question about the same request. Usage numbers rarely arrive in the first chunk, and many teams discover that their latency budget and their cost dashboard both break when they bolt metering onto a live chat loop. This field test runs include_usage against a real-time token stream and records what actually shows up on the wire, when it shows up, and what it costs in milliseconds.

The test setup is deliberately plain. A streaming client opens one request against a streaming-first relay and prints every server-sent event as it arrives, so we can see both the visible text deltas and the trailing accounting. The relay used here is Fluxio, a pay-per-token API relay where a single key reaches Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok and more, with replies arriving token by token. The API base is https://api.fluxioapi.com, and OpenAI-style clients add /v1 to that path. Fluxio also works with Claude Code, Codex, Gemini CLI and any OpenAI SDK, which matters because usage accounting has to survive whatever client shape you already ship.

• – One streaming request per model, with include_usage set and unset, measured across the same prompt
• – Chunk arrival times captured alongside the final usage block to isolate any latency penalty
• – Manual retry of a cancelled stream to confirm whether usage is still emitted for partial output
• – Side-by-side totals from the usage object against a manual token count of the returned text

Verdict: include_usage is not decorative metadata. It is the only trustworthy record of what a live token stream actually billed, and leaving it off turns every cost estimate into a guess.

§2 Where the usage block lands in the live token stream

The first surprise in the field test is positional. With include_usage disabled, the stream ends with a terminator and no accounting. With include_usage enabled, the terminal event carries a usage object alongside the usual finish reason, so the client that reads until the stream closes will pick it up automatically. For GPT-6 Astra the block arrived in the closing event roughly one event after the last text delta, which is exactly the behavior you want: no polling, no second request, no reconciliation step.

Interim usage events behaved differently across providers. Some models emitted a partial usage summary before the final one, and naive clients that summed every usage event they saw ended up double counting. The safe pattern is to treat the last usage object as authoritative and to overwrite an accumulator rather than adding to it. This is a small code change and the difference between a dashboard that matches the invoice and one that reports triple spend.

• – Overwrite the stored usage object each time a new one arrives instead of incrementing running totals
• – Attach a stream identifier to the captured usage so retries and parallel chats cannot merge into one bucket
• – Render text deltas to the user immediately and keep accounting in a background handler so metering never blocks the first token

Verdict: the usage block belongs at the tail of the stream, and clients should read it as a snapshot. The streaming angle is friendly here, because the accounting rides the same connection as the live reply.

§3 Latency impact of asking for usage on token-by-token replies

Teams worry that requesting usage will slow the first visible token. The measurements said otherwise. Time to first token stayed within noise of the baseline run, because include_usage changes what is emitted at the end, not what is emitted at the start. On Claude Haiku 4.5 the difference between the slowest and fastest run with usage enabled was smaller than ordinary network jitter, which is the verdict we care about for real-time chat.

The only measurable cost is a slightly longer tail: the stream now has one more event to deliver before it closes. That tail matters if your client tears down the connection as soon as the last text delta renders. Correct handling is to keep the stream open until the close event, then flush the usage record to storage. Product code should treat the reply as finished when text stops flowing, and treat the request as finished when the usage block lands.

• – Keep the stream open until the close event so the final usage block is never discarded
• – Show the reply as complete when text deltas stop, and mark accounting complete when usage is stored
• – Measure tail latency separately from time to first token, since they answer different product questions

Verdict: requesting usage is effectively free for perceived responsiveness, and the streaming experience stays smooth token by token.

§4 Retries, cancellations and partial usage records

Real-time chat is messy. Users hit stop, networks drop, and clients retry. The field test cancelled streams mid-flight and observed partial usage blocks that reflected only the tokens already generated. That is correct and useful, but it means your ledger needs a status field: a cancelled stream still costs money, and a retried stream creates a second billing event.

Usage records also differ in what they expose. Prompt tokens, completion tokens and total tokens are consistent, while cached prompt handling varies, so the simplest reliable rule is to persist the raw usage object rather than deriving a single number immediately. That keeps future breakdowns possible without a migration.

• – Store the raw usage object per stream attempt and derive totals in the reporting layer
• – Mark cancelled streams as partial so dashboards do not mistake truncated output for a cheap request
• – Give every retry a fresh stream identifier to avoid attributing two usage blocks to one logical message

Verdict: partial usage is a feature, not a bug, as long as your schema can represent it. Cancelled work is billed work, and include_usage makes that visible.

§5 Using include_usage to budget a multi-model chat product

Once usage is captured reliably, product decisions get easier. A token stream that reports its own consumption lets you set per-user budgets, flag unusually expensive prompts, and show a live cost meter next to the chat box. In testing, routing the same summarization prompt to Gemini 3.1 Pro and a smaller variant produced clearly different completion counts, and the usage block made that comparison quantitative instead of anecdotal.

This is where a relay earns its keep. Fluxio gives one key across Claude, GPT, Gemini, DeepSeek, Kimi, GLM, Qwen, Grok and more, with usage arriving in the same shape regardless of the underlying model, so your metering code does not fork per provider. It fits naturally in streaming-first products because the replies and the accounting share one real-time path. Members who invite friends also earn 17% of what those friends pay, which is a pleasant side effect of being early.

• – Cap each user session with a token budget derived from captured usage rather than estimated characters
• – Compare models using real completion counts from identical prompts before locking a routing strategy
• – Feed usage records into alerts that trigger when a single stream exceeds a configured threshold

Verdict: include_usage is the foundation of honest token budgeting in real-time chat, and a unified relay keeps that accounting portable across models.

Sign up for Fluxio and try include_usage on your next token stream.