Streaming AI Replies with Java, Kotlin, and SSE

§1 Step 1: Choose SSE for live AI replies

A streaming AI backend should deliver useful text before the model finishes its answer. Instead of collecting a complete response, your Java or Kotlin service forwards incremental events to the client. Server-sent events, or SSE, fit this one-way flow without requiring a WebSocket connection.

This walkthrough uses Spring Boot with Spring WebFlux. The request travels from your frontend to a backend endpoint, then to an OpenAI-compatible streaming API. Response events travel back along the same path. Individual events may contain text fragments, metadata, or completion signals; do not assume every event equals one token.

§2 Step 2: Configure Fluxio behind your backend

Fluxio fits applications that need streaming-first access through one API key and pay-per-token billing. Its API base is https://api.fluxioapi.com; OpenAI-style clients use https://api.fluxioapi.com/v1. You can evaluate models such as Gemini 2.5 Flash or Claude Haiku 4.5 without redesigning your frontend streaming protocol.

Add the Spring WebFlux starter to your project. Store your credentials in FLUXIO_API_KEY and your selected model identifier in FLUXIO_MODEL. Obtain the exact supported identifier from the provider rather than assuming a display name is an API identifier.

Java client configuration: WebClient client = WebClient.builder().baseUrl("https://api.fluxioapi.com/v1").defaultHeader(HttpHeaders.AUTHORIZATION, "Bearer " + System.getenv("FLUXIO_API_KEY")).build();

Create this client once as a Spring bean, then inject it into your controller. Never send the upstream API key to a browser or include it in request logs.

§3 Step 3: Open the upstream stream in Java

Create a POST endpoint named /api/chat/stream. Accept a validated request containing the user's message, and declare text/event-stream as the response media type. Build the upstream payload with model, messages, and stream set to true. Each message should contain a role and content.

Java streaming request: client.post().uri("/chat/completions").contentType(MediaType.APPLICATION_JSON).accept(MediaType.TEXT_EVENT_STREAM).bodyValue(payload).retrieve().bodyToFlux(new ParameterizedTypeReference<ServerSentEvent<String>>() {});

Return that publisher directly from a controller method whose return type is Flux<ServerSentEvent<String>>. Here, payload is the request map you constructed. Spring decodes upstream SSE frames and writes downstream SSE frames without collecting the entire answer.

Do not call block(), collectList(), or subscribe() inside the controller. Blocking defeats the reactive execution model, collecting delays delivery, and manual subscription disconnects request handling from the returned stream.

§4 Step 4: Use the same stream contract in Kotlin

A Kotlin Spring controller can return the same reactive type. Reuse the configured WebClient and construct the payload with mapOf. Set messages to listOf(mapOf("role" to "user", "content" to request.message)) and stream to true.

Kotlin streaming request: client.post().uri("/chat/completions").contentType(MediaType.APPLICATION_JSON).accept(MediaType.TEXT_EVENT_STREAM).bodyValue(payload).retrieve().bodyToFlux(object : ParameterizedTypeReference<ServerSentEvent<String>>() {})

Declare the method result as Flux<ServerSentEvent<String>>. Kotlin does not require a coroutine conversion for this approach. Keeping both implementations on the same SSE contract also makes integration tests and frontend behavior consistent.

§5 Step 5: Render incremental text in the frontend

Because this endpoint accepts POST requests, use fetch with a streaming response reader rather than the browser's native EventSource API. Decode bytes incrementally and parse complete SSE events, retaining partial frames between reads. Network chunks do not necessarily align with events or characters.

For this pass-through endpoint, parse each data payload as JSON and append choices[0].delta.content when present. Ignore absent content fields and handle the [DONE] sentinel before attempting JSON parsing. This example covers text replies; tool calls and structured output require additional handling.

§6 Step 6: Verify cancellation and production delivery

• – Keep the reactive subscription connected so browser disconnection can cancel the upstream request.
• – Disable response buffering in reverse proxies and verify that compression does not delay small events.
• – Measure time to first text, total duration, cancellation behavior, and upstream error rates.
• – Handle authentication failures before streaming begins, and define a safe application error event for failures after headers are committed.
• – Set appropriate idle timeouts and avoid automatic retries after partial output, which can duplicate text and increase charges.

Test through your deployed proxy, not just localhost, and confirm that text arrives progressively rather than all at once.

Sign up for Fluxio to build live AI replies.