Real-Time Streaming Architecture

Learning outcomes

• Select optimal real-time streaming protocols based on client requirements
• Build fallback mechanisms for web browser environments

Mental model

Choosing a real-time streaming protocol requires matching directionality (unidirectional vs full-duplex), payload format (JSON vs Protobuf), and browser network proxy constraints.

Theory

• Server-Sent Events (SSE): Best for unidirectional server-to-client streaming (e.g. AI token generation, live dashboards). Works over standard HTTP/1.1 and HTTP/2.
• WebSockets: Best for full-duplex bi-directional communication (e.g. chat applications, collaborative whiteboards). Operates over upgraded TCP sockets.
• gRPC-Web: Enables web browsers to invoke gRPC services through a translation proxy (Envoy). Supports server streaming and unary RPCs, but web browsers cannot support true bi-directional streaming over standard Fetch/XHR APIs without WebSockets.

| Metric | Server-Sent Events (SSE) | WebSockets | gRPC-Web | |---|---|---|---| | Direction | Unidirectional (Server -> Client) | Full-Duplex (Bi-directional) | Unary & Server Streaming | | Protocol | Standard HTTP / HTTP/2 | Upgraded TCP Protocol | Standard HTTP / HTTP/2 | | Payload | UTF-8 Text | Text / Binary | Binary Protobuf / Base64 | | Auto Reconnect | Native (EventSource) | Manual JS Implementation | Manual / Library |

// Client Protocol Selector Decision Function
export function selectStreamingProtocol(needsBiDirectional: boolean, needsBinaryProtobuf: boolean): string {
  if (needsBiDirectional && !needsBinaryProtobuf) {
    return 'WebSocket';
  } else if (needsBinaryProtobuf) {
    return 'gRPC-Web';
  } else {
    return 'Server-Sent Events (SSE)';
  }
}

Alternatives and trade-offs

• HTTP Long-Polling: Works everywhere; wasteful HTTP connection overhead and latency.
• WebSockets: Bi-directional framing; bypasses standard HTTP caching and proxy infrastructure.
• SSE: Simple, native browser auto-retry; unidirectional text-only.

Failure modes and misconceptions

1. Assuming gRPC-Web Supports True Bi-directional Streaming in Browsers: Modern browser W3C Fetch/XHR APIs do not expose writable request streams to JS. Full bi-directional gRPC-Web requires a WebSocket transport framing layer.
2. Ignoring Connection Pool Exhaustion: Long-lived SSE/WebSocket connections consume server sockets. Sockets must be multiplexed or scaled horizontally using stateless message brokers (Redis Pub/Sub).

Decision scenario

Select Server-Sent Events (SSE) for unidirectional text token streams and WebSockets for full-duplex interactive multi-user applications.

Learning outcomes

• Compare architectural trade-offs across SSE, WebSockets, and gRPC-Web.
• Address browser limitations regarding HTTP request stream writing.
• Scale long-lived real-time connections using Redis Pub/Sub backplane queues.

Trade-offs

Real-time streaming protocols deliver instant user feedback, but require managing socket connection lifecycles and horizontal scaling backplanes.

Primary sources

• FastAPI Framework Architecture & Dependency Injection Specification — Tiangolo, verified 2026-07-22