Stream LLM Responses for a Snappy UX

Why streaming matters

Ask a model for a long answer and, without streaming, the user stares at a blank screen for several seconds while the entire response is generated — then it appears all at once. Streaming sends the answer token-by-token as it's produced, so text starts appearing almost immediately. The total time is the same, but the perceived speed is dramatically better. It's the difference between a frozen app and a lively one, and one of the cheapest UX upgrades you can ship.

Stream in Python

Set stream=True and iterate over the chunks as they arrive:

stream = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Write a haiku about RAG."}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content or ""
    print(delta, end="", flush=True)

Instead of one big message.content, each chunk carries a small delta — the next snippet of text. You print each delta as it lands, building the answer in real time. The or "" guards against empty deltas, which the first and last chunks often carry.

Wiring it into a web app

The same idea extends to a real interface:

• Backend: as you receive each delta from the model, forward it to the browser over Server-Sent Events (SSE) or a WebSocket, rather than waiting to send the whole response.
• Frontend: append each incoming delta to the message bubble as it arrives, so the user watches the answer type itself out.

Things to watch

• Accumulate the full text as you stream if you need to store or post-process the complete reply.
• Handle disconnects — if the user navigates away, cancel the stream so you're not billed for tokens nobody sees.
• Errors mid-stream can happen; wrap the loop so a failure halfway through degrades gracefully.

The takeaway

Streaming doesn't make your model faster — it makes your app feel faster, which is what users actually notice. For any response longer than a sentence, it's one of the highest-impact changes you can make.

Try it now: Take a non-streaming call you already have, add stream=True, and print the deltas. The instant the first words appear, you'll understand why every polished AI product streams.