API Gateway Pattern & Circuit Breaking

Learning outcomes

• Centralize authentication and routing at the API Gateway layer
• Implement circuit breakers to isolate failing microservices

Mental model

An API Gateway acts as a single entry point for all client requests in a microservices system. It encapsulates internal service boundaries, routes traffic dynamically, offloads cross-cutting concerns (authentication, rate limiting, logging), and protects downstream services using Circuit Breakers.

Theory

• Request Routing: Maps public external routes (/api/v1/orders) to internal microservice IP endpoints (http://order-service.internal:8080).
• Circuit Breaker States:
  • CLOSED: Normal operation; requests flow through to upstream service.
  • OPEN: Upstream error threshold breached; gateway immediately fails requests locally (503 Service Unavailable) without calling upstream service.
  • HALF-OPEN: Trial state after timeout; sends limited trial requests to check if upstream has recovered.

# Python Circuit Breaker State Machine Pattern
import time

class CircuitBreaker:
    def __init__(self, failure_threshold=5, recovery_timeout=30):
        self.failure_threshold = failure_threshold
        self.recovery_timeout = recovery_timeout
        self.state = "CLOSED"
        self.failure_count = 0
        self.last_state_change = time.time()

    def can_execute(self) -> bool:
        now = time.time()
        if self.state == "OPEN":
            if now - self.last_state_change > self.recovery_timeout:
                self.state = "HALF-OPEN"
                self.last_state_change = now
                return True
            return False
        return True

    def record_result(self, success: bool):
        if success:
            if self.state == "HALF-OPEN":
                self.state = "CLOSED"
                self.failure_count = 0
        else:
            self.failure_count += 1
            if self.failure_count >= self.failure_threshold:
                self.state = "OPEN"
                self.last_state_change = time.time()

Alternatives and trade-offs

• Direct Microservice Exposure: Simple initial setup; forces every microservice to re-implement authentication, SSL, rate limiting, and CORS.
• API Gateway Pattern: Centralized policy enforcement and fault isolation; creates a potential single point of failure requiring high-availability deployment.

Failure modes and misconceptions

1. Cascading Service Failures: A failing downstream microservice causing threads in caller services to block while waiting for timeouts. Remedy: Wrap external calls in Circuit Breakers to fail fast.
2. Monolithic API Gateway: Putting heavy business logic into the gateway code turns the gateway into a bloated distributed monolith. Keep the gateway focused strictly on routing and security.

Decision scenario

Implement the API Gateway pattern with Circuit Breakers to centralize authentication and prevent cascading microservice outages during downstream service degradation.

Learning outcomes

• Structure central API Gateway routing and policy enforcement layers.
• Implement Circuit Breaker state transitions (CLOSED, OPEN, HALF-OPEN).
• Prevent cascading failures across microservice call chains.

Trade-offs

API Gateways streamline client communication and isolate microservices, but introduce an additional network hop and operational control point.

Primary sources

• FastAPI Framework Architecture & Dependency Injection Specification — Tiangolo, verified 2026-07-22