Mental model
Streaming is an ordered event log, not text arriving character by character. The client reduces typed events into visible state while retaining a request identity, sequence position, cancellation path, and terminal result.
Theory
Define an event envelope containing request ID, event type, sequence number, timestamp, and payload. Separate text deltas from tool states, citations, usage, warnings, errors, and completion. Decode bytes incrementally and buffer partial records before parsing. The UI must never infer completion only because a connection closed.
Cancellation propagates from interface to transport, server orchestration, provider request, and tools where possible. The server records whether work stopped or continued after client disconnect. Reconnection needs an idempotent request identity and either replay from a cursor or a clean restart policy. Measure queue time, time to first meaningful event, generation duration, and finalization time separately.
Alternatives and trade-offs
Server-Sent Events are simple for server-to-client updates. Fetch streams support request bodies and custom framing. WebSockets help with bidirectional event traffic but add lifecycle complexity. Polling can be more reliable for long-running jobs. Choose based on interaction semantics, not fashion.
Failure modes and misconceptions
Rendering every token causes excessive layout work. Arbitrary network chunks are not message boundaries. Client cancellation without server propagation wastes resources. Retrying a side-effecting stream without idempotency can duplicate work. Showing partial structured output before validation may create unsafe UI state.
Knowledge check
Which terminal event fields let the client distinguish success, partial success, cancellation, and transport loss?
Decision scenario
A support workspace streams answer deltas and tool events under one request ID. Sequence gaps trigger replay, cancellation stops downstream retrieval, and the final event includes citations, usage, completion status, and an immutable trace ID.
Learning outcomes
- Explain Streaming AI Interfaces as a system mechanism rather than a slogan.
- Compare its alternatives, trade-offs, and production failure modes.
- Apply the concept to a decision and identify evidence that would validate it.
Trade-offs
Using Streaming AI Interfaces can improve capability or control, but it also introduces cost, latency, complexity, and failure modes that must be measured against an explicit objective.