Reverse Proxies & Load Balancing

Learning outcomes

• Configure Nginx and Traefik reverse proxies for microservices
• Manage TLS termination and HTTP keep-alive connection pools

Mental model

A Reverse Proxy acts as an edge gateway positioned in front of backend application servers. It terminates TLS/SSL certificates, inspects incoming HTTP request paths, compresses responses, and balances traffic across upstream server clusters.

Theory

• TLS Termination: Decrypts HTTPS traffic at the edge proxy, allowing internal microservice communication over fast unencrypted HTTP within isolated private networks.
• Upstream Connection Pooling: Reuses persistent TCP connections between the reverse proxy and backend application servers (proxy_http_version 1.1; proxy_set_header Connection "";).

# Nginx production reverse proxy configuration
upstream fastapi_backend {
    least_conn; # Route to server with fewest active connections
    server 10.0.1.10:8000 max_fails=3 fail_timeout=10s;
    server 10.0.1.11:8000 max_fails=3 fail_timeout=10s;
    keepalive 32; # Maintain 32 persistent backend sockets
}

server {
    listen 443 ssl http2;
    server_name fullstackaihub.com;

    ssl_certificate /etc/letsencrypt/live/fullstackaihub.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/fullstackaihub.com/privkey.pem;

    location / {
        proxy_pass http://fastapi_backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Alternatives and trade-offs

• Direct Application Exposure: Simple; forces web application workers to manage TLS certificates, static file serving, and DDoS filtering.
• Reverse Proxy Gateway (Nginx / Traefik): High performance edge security, static file caching, and seamless zero-downtime rolling deployments.

Failure modes and misconceptions

1. Missing X-Forwarded-For Headers: Failing to set X-Forwarded-For causes application logs and rate limiters to see the reverse proxy internal IP address for all client requests.
2. HTTP/1.0 Default Proxy Pass: Nginx defaults to proxy_http_version 1.0, which closes backend TCP sockets after every request. Always explicitly configure HTTP/1.1 with persistent keep-alive connections.

Decision scenario

Deploy Nginx or Traefik as the reverse proxy edge router in front of containerized FastAPI pods to handle TLS termination, Gzip/Brotli compression, and upstream load balancing.

Learning outcomes

• Configure TLS termination and modern SSL cipher suites at the edge.
• Implement upstream load-balancing strategies (Round Robin, Least Connections, IP Hash).
• Enable persistent HTTP/1.1 keep-alive connection pools between proxy and backend apps.

Trade-offs

Reverse proxies add an extra network hop to request latency, but provide essential edge security, TLS termination, and traffic management capabilities.

Primary sources

• FastAPI Framework Architecture & Dependency Injection Specification — Tiangolo, verified 2026-07-22