Streaming AI Responses over SSE: Practical Guide

The beginner mistake is treating a network chunk as an event: one chunk can hold several SSE events or half a line. Accumulate a buffer, decode UTF-8 incrementally, split events on empty lines, keep the tail. Parse data events separately from control markers, completion is its own state.

On user cancel close the reader explicitly and record «cancelled by user», not «error»: different analytics and billing branches. Never treat a TCP break as success — wait for the final event and save the request ID.

Streaming is off for chat completions and messages endpoints — send stream:false there or use Responses API. Set separate connect and read timeouts; a single shared timeout either kills long answers or hangs dead connections.

Streaming does not change the price: same tokens, same formula, different delivery. Usage is finalized when the operation closes, so the total exists even if the user closed the tab mid-stream — reconcile against history, not displayed text.

Minimum set before prod: short answer, long answer, slow answer, user cancel, network break, error after the first event. For each record what the user sees and what total lands in usage.

Repeat after a partial answer only with an explicit duplication strategy. Long streaming tasks need heartbeats shorter than the operation, otherwise drops look like slow generation.