Proxy SSE Through nginx Without Buffering

§1 Step 1: Verify SSE Before Adding nginx

Start with a controlled streaming endpoint rather than a full AI conversation. Have your application emit one numbered event every second for ten seconds. Return Content-Type: text/event-stream, encode the response as UTF-8, and terminate each event with a blank line. A message such as data: hello followed by two newline characters forms a complete event. Without that separator, the client waits instead of dispatching the message. ([html.spec.whatwg.org](https://html.spec.whatwg.org/multipage/server-sent-events.html))

Set an environment variable named UPSTREAM_SSE_URL to your application endpoint, then run curl -N -i "$UPSTREAM_SSE_URL". The -N option disables curl's output buffering, while -i displays response headers. Expect one event per second, not ten events together. ([curl.se](https://curl.se/docs/manpage.html))

Treat this direct test as your baseline. If it already arrives in bursts, fix application flushing before changing the proxy.

§2 Step 2: Disable Buffering on the SSE Location

Inside your existing nginx server block, add the following location. This example assumes your application listens on port 3000 and serves /events. The proxy_pass directive has no replacement URI, so the original request path is preserved. ([nginx.org](https://nginx.org/en/docs/http/ngx_http_proxy_module.html))

location /events { proxy_pass http://127.0.0.1:3000; proxy_http_version 1.1; proxy_set_header Connection ""; proxy_buffering off; proxy_cache off; gzip off; proxy_read_timeout 300s; }

proxy_buffering off is the essential response-streaming setting: nginx passes received data onward instead of collecting the response in its normal proxy buffers. Disabling proxy caching also overrides an inherited cache for this location. Do not confuse this with proxy_request_buffering, which controls the incoming request body rather than outgoing SSE events. ([nginx.org](https://nginx.org/en/docs/http/ngx_http_proxy_module.html))

Keep compression disabled during initial verification. The gzip off directive removes nginx's gzip filter from this route, giving your streaming test fewer moving parts. ([nginx.org](https://nginx.org/en/docs/http/ngx_http_gzip_module.html?utm_source=openai))

§3 Step 3: Send Streaming Headers From the Application

Have the application return X-Accel-Buffering: no. nginx recognizes this upstream response header as a buffering control unless its configuration explicitly ignores it. Adding that header to the downstream response with add_header is not equivalent to receiving it from upstream. nginx also normally hides X-Accel-* headers, so their absence in curl output does not prove failure. ([nginx.org](https://nginx.org/en/docs/http/ngx_http_proxy_module.html))

For this walkthrough, also return Cache-Control: no-cache and flush each completed event immediately. Avoid collecting the complete generated answer before beginning the response.

§4 Step 4: Align SSE Heartbeats With Idle Timeouts

The configured proxy_read_timeout measures inactivity between successive upstream reads, not the total lifetime of the response. A continuously active stream can therefore last longer than 300 seconds. ([nginx.org](https://nginx.org/en/docs/http/ngx_http_proxy_module.html))

For an application-owned SSE endpoint, send a comment heartbeat such as : heartbeat followed by a blank line every 15 seconds during otherwise quiet periods. SSE parsers ignore comments, and periodic comments help keep idle connections alive. ([html.spec.whatwg.org](https://html.spec.whatwg.org/multipage/server-sent-events.html))

Keep those heartbeats separate from any strict AI provider protocol. Start with the controlled endpoint, then test your real streaming payload unchanged.

§5 Step 5: Reload nginx and Compare Event Timing

Run nginx -t to validate the configuration. If validation succeeds, run nginx -s reload to apply it. nginx supports configuration reloads without immediately terminating existing worker connections. ([nginx.org](https://nginx.org/en/docs/beginners_guide.html))

$ Set PUBLIC_SSE_URL to the proxied endpoint and repeat curl -N -i "$PUBLIC_SSE_URL". Compare its cadence with the direct test. ([curl.se](https://curl.se/docs/manpage.html))

• – Verify that the first event arrives before the test completes.
• – Compare first-event latency and gaps between subsequent events.
• – If only the public route batches events, repeat the test at each intermediary.

§6 Step 6: Connect Fluxio Behind Your Streaming Backend

Fluxio fits this architecture as a streaming-first, pay-per-token AI API relay with one key across providers. Keep that key in your backend and relay its token-by-token responses through the verified nginx route. For OpenAI-style clients, configure Fluxio's API base with /v1 appended. Use the same timing checks when replacing the controlled test with a real generation.

Sign up for Fluxio.