lesson depth
Mastery
not started · 0%

Redis Cache-Aside & Thundering Herd Defense

Cache-Aside pattern, TTL eviction, Redis distributed locking, and thundering herd stampede protection.

Freshness: current15 min readData Engineering and Databases

Key Learning Outcomes

  • Implement Cache-Aside pattern with explicit TTL invalidation
  • Protect database backends from thundering herd spikes

Mental model

The Cache-Aside (Lazy Loading) pattern positions an in-memory data store (Redis) beside the primary database. The application queries Redis first; on a cache miss, it reads from PostgreSQL, writes the result to Redis with a TTL, and returns the response.

HTTP Request Route
Check Redis Cache Key
Cache HIT -> Return In-Memory Payload
Cache MISS -> Lock & Query PostgreSQL
Write Result to Redis with TTL
Conceptual teaching model synthesized from:FastAPI Framework Architecture & Dependency Injection Specification

Theory

When a popular cache key expires under high concurrency (e.g. 1,000 concurrent requests/sec), all workers encounter a cache miss simultaneously and hammer PostgreSQL. This is the Thundering Herd (Cache Stampede) problem.

Defense requires acquiring a distributed Redis lock (redis.set(lock_key, "1", nx=True, ex=5)) so that only ONE worker queries PostgreSQL while other workers wait briefly or receive stale cached data.

python(29 lines)
1import json
2import redis.asyncio as aioredis
3from fastapi import FastAPI, Depends
4
5redis_client = aioredis.from_url("redis://localhost:6379/0", decode_responses=True)
6
7async def get_cached_user_profile(user_id: int, db_session) -> dict:
8 cache_key = f"user:profile:{user_id}"
9 cached_data = await redis_client.get(cache_key)
10
11 if cached_data:
12 return json.loads(cached_data)
13
14 # Thundering Herd Defense via mutex lock
15 lock_key = f"lock:{cache_key}"
16 acquired = await redis_client.set(lock_key, "1", nx=True, ex=5)
17
18 if acquired:
19 try:
20 profile = await fetch_user_from_db(db_session, user_id)
21 await redis_client.setex(cache_key, 300, json.dumps(profile))
22 return profile
23 finally:
24 await redis_client.delete(lock_key)
25 else:
26 # Wait briefly for lock holder to populate cache
27 await asyncio.sleep(0.05)
28 return await get_cached_user_profile(user_id, db_session)
9 lines hidden

Alternatives and trade-offs

  • Read-Through Cache: Cache layer intercepts database reads transparently; complex middleware setup.
  • Cache-Aside: Simple application-level caching control; risks serving stale data if invalidation logic fails on state updates.

Failure modes and misconceptions

  1. Missing Cache Invalidation on Writes: Updating a user record in PostgreSQL without executing redis_client.delete(cache_key) causes users to see stale data until TTL expires.
  2. Infinite Cache Lock Deadlock: Acquiring a distributed lock without setting an explicit ex (expiration TTL) locks out all concurrent workers permanently if the process crashes mid-query.
Reflect before revealing the guide

Decision scenario

Implement Cache-Aside with Redis set(nx=True, ex=5) distributed locking on high-traffic product catalog endpoints to prevent database crash loops caused by cache stampedes.

Learning outcomes

  • Implement the Cache-Aside pattern with explicit Redis TTL expiration.
  • Defend against Thundering Herd / Cache Stampede spikes using Redis distributed locks.
  • Invalidate cached data deterministically during database mutation operations.

Trade-offs

Redis caching dramatically reduces database query load and response latency, but increases system state complexity and requires strict cache invalidation rules.

Prerequisites & Related Concepts (2)

Private notes

0 words
Next