Live Meeting Captions and Summaries with Token Streams

Live meeting captions help people follow the conversation, while a running summary captures decisions before they disappear into the next discussion. Building both requires two coordinated streams: speech recognition produces transcript segments, and a language model turns stable segments into readable updates. This walkthrough explains how to connect those streams without letting unfinished speech become an inaccurate meeting record.

§1 Step 1: Capture meeting audio with timestamps

Start with an audio source your application is authorized to access, such as a microphone or a meeting platform’s supported integration. Tell participants when transcription is active and provide a visible stop control. Send audio to a speech recognition service that supports incremental transcription.

Preserve the timing information returned by that service. Each transcript segment should include a meeting identifier, segment identifier, start time, end time, text, and any available speaker identifier. Keep speaker labels provisional until you can map them reliably to participants.

§2 Step 2: Separate provisional captions from final text

Speech recognizers often revise words as more audio arrives. Show these provisional results immediately in the caption area, but replace the existing segment when a revision arrives. Appending every revision creates duplicate sentences and confuses downstream summaries.

When the recognizer marks a segment as final, store it as committed transcript text. Your summary pipeline should normally consume these committed segments. This creates a clear boundary between responsive captions and the more stable evidence used to record decisions.

§3 Step 3: Connect a streaming model through Fluxio

Fluxio fits the text generation stage: it is a streaming-first, pay-per-token AI API relay that delivers replies token by token. Keep speech recognition as a separate component unless you have independently confirmed support for your audio workflow.

For an OpenAI-style SDK, configure the base URL as https://api.fluxioapi.com/v1, supply your Fluxio key on the server, and enable streaming on the generation request. Select an available model, such as Gemini 2.5 Flash, using the exact model identifier published by the service. Keep API credentials out of browser code.

One key can simplify model access while you evaluate summary quality and response time. Measure both with representative meeting transcripts before choosing your production configuration.

§4 Step 4: Summarize stable transcript batches

Trigger a summary update after a short interval or after enough final transcript text accumulates. Start by testing intervals around ten to twenty seconds, then adjust for meeting pace, cost, and acceptable delay. Avoid sending a separate request for every spoken word.

Give the model the previous summary and the new committed segments. Ask it to update a small, consistent set of fields:

• – Discussion points grounded in the supplied transcript.
• – Decisions that participants explicitly agreed to.
• – Action items with owners and deadlines only when stated.
• – Open questions that still require an answer.

Require the model to preserve uncertainty and avoid inventing missing details. Include segment identifiers in the input and request supporting references for decisions and action items, so readers can check the original wording.

§5 Step 5: Stream draft summaries into the meeting view

Read generated text chunks as they arrive and forward them to the browser through Server-Sent Events or a WebSocket. SSE is a practical choice when the browser only needs to receive updates. Buffer network data correctly: a transport chunk may contain only part of an event.

Label the incoming summary as a draft. When generation finishes successfully, commit the completed version and retain its transcript boundary. If a newer batch arrives during generation, queue it or supersede the older request using explicit version numbers. Prevent late responses from replacing newer summaries.

§6 Step 6: Recover interrupted streams and verify decisions

Persist committed transcript segments and completed summaries. After a disconnect, resume from the last processed segment identifier. Use segment identifiers to deduplicate retries, and keep an interrupted summary marked incomplete until a replacement finishes.

Test overlapping speech, corrections, silence, and dropped connections. Track caption delay separately from summary delay, and verify that each recorded decision has transcript support. At meeting end, generate a final summary from the committed transcript, then let participants review owners, deadlines, and disputed wording.

Sign up for Fluxio to try streaming meeting summaries.