Distributed Transactions & Saga Pattern

Learning outcomes

• Architect Saga workflows with compensating actions
• Implement the Transactional Outbox pattern for reliable messaging

Mental model

In a distributed microservice architecture, a single user action spans multiple independent database services (e.g. Order Service, Payment Service, Inventory Service). ACID transactions cannot span remote network boundaries without sacrificing availability.

Theory

• Two-Phase Commit (2PC): Synchronous protocol locking resources across databases during Prepare and Commit phases. High coupling and availability bottleneck.
• Saga Pattern: A sequence of local ACID transactions. Each local transaction updates its own database and triggers the next step via event messages. If a step fails, the Saga orchestrator executes Compensating Transactions in reverse order to undo prior changes.
• Transactional Outbox: Writes domain events to a local outbox database table within the local ACID transaction before asynchronous relay workers publish messages to Redis/RabbitMQ.

# FastAPI Saga Orchestrator Example
async def execute_checkout_saga(order_id: str, user_id: str, amount: float):
    # Step 1: Create Order
    await order_service.create_pending_order(order_id, amount)
    
    # Step 2: Authorize Payment
    payment_success = await payment_service.charge(user_id, amount)
    if not payment_success:
        await order_service.compensate_cancel_order(order_id)
        return {"status": "failed", "reason": "Payment declined"}
    
    # Step 3: Reserve Inventory
    inventory_success = await inventory_service.reserve(order_id)
    if not inventory_success:
        # Compensate preceding steps in reverse order
        await payment_service.compensate_refund(user_id, amount)
        await order_service.compensate_cancel_order(order_id)
        return {"status": "failed", "reason": "Inventory unavailable"}
    
    await order_service.confirm_order(order_id)
    return {"status": "success", "order_id": order_id}

Alternatives and trade-offs

• Two-Phase Commit (2PC): Immediate consistency; low availability, susceptible to blocking network deadlocks.
• Saga Pattern (Compensating Actions): Eventual consistency, high availability, fault-tolerant; requires implementing explicit reverse rollback logic for every action.

Failure modes and misconceptions

1. Missing Idempotency Keys: Network retries during Saga execution can trigger double charges if payment or inventory services lack idempotency keys (X-Idempotency-Key).
2. Dual Writes without Outbox: Writing to PostgreSQL and directly publishing to RabbitMQ without an outbox table risks state drift if the broker connection fails mid-transaction.

Decision scenario

Implement the Saga pattern with compensating transactions and Transactional Outbox event tables for multi-microservice checkout workflows to achieve high availability and eventual consistency.

Learning outcomes

• Compare Two-Phase Commit (2PC) with the Saga pattern for distributed systems.
• Design compensating transaction flows for multi-step distributed workflows.
• Implement the Transactional Outbox pattern to prevent database/message broker state drift.

Trade-offs

The Saga pattern maintains high system availability across microservices, but replaces strict immediate consistency with eventual consistency and requires writing explicit rollback handlers.

Primary sources

• FastAPI Framework Architecture & Dependency Injection Specification — Tiangolo, verified 2026-07-22