Mental model
CPUs do not read RAM byte-by-byte; CPUs fetch memory in fixed 64-byte Cache Lines into L1, L2, and L3 caches. In multi-socket servers, NUMA (Non-Uniform Memory Access) means accessing local RAM connected to the CPU socket is significantly faster than accessing remote RAM across inter-socket interconnects.
CPU Core requests memory byte
Check L1 Cache (1ns) -> L2 (4ns) -> L3 (12ns)
L3 Cache MISS -> Read 64-byte Cache Line from RAM (60ns)
NUMA Local RAM Access (Fast) vs Remote Socket RAM (Slow)
Cache Coherency MESI Protocol invalidates modified lines
Theory
- False Sharing: Occurs when two independent threads running on different CPU cores update distinct variables that happen to reside within the same 64-byte cache line. The MESI cache coherency protocol continuously invalidates and bounces the cache line back and forth between cores, degrading performance.
- Cache Padding: Padding struct members to 64-byte boundaries ensures distinct variables reside on separate cache lines.
- NUMA Binding (
numactl): Pinning application process memory and execution threads to the same NUMA node to guarantee local RAM access.
c(20 lines)
1// C example: Eliminating False Sharing via Cache Line Alignment
2#include <stdio.h>
3#include <pthread.h>
4
5// Struct with 64-byte alignment to prevent False Sharing
6struct alignas(64) ThreadCounter {
7 long count;
8 char padding[56]; // Pad struct to fill 64-byte cache line
9};
10
11struct ThreadCounter counters[2];
12
13void* worker(void* arg) {
14 long id = (long)arg;
15 for (long i = 0; i < 100000000; i++) {
16 counters[id].count++;
17 }
18 return NULL;
19}
Alternatives and trade-offs
- Unaligned Data Structures: Compact memory usage; vulnerable to false sharing performance penalties under concurrent multi-threaded writes.
- Cache-Line Aligned Structs (
alignas(64)): Eliminates false sharing bounce; slightly increases struct memory footprint.
Failure modes and misconceptions
- False Sharing Cache Bouncing: Multi-threaded counters placed side-by-side in an unpadded array cause throughput drops up to 20x due to continuous MESI cache line invalidation loops.
- Cross-NUMA Memory Access Penalties: Running latency-sensitive memory stores across arbitrary CPU sockets without NUMA binding causes remote memory access latency spikes.
Reflect before revealing the guide
Decision scenario
Align thread counters to 64-byte cache line boundaries and pin database process memory using numactl --membind to eliminate false sharing and remote NUMA latency.
Learning outcomes
- Structure C/C++ data structures to align with CPU 64-byte cache lines.
- Detect and eliminate multi-threaded False Sharing using cache line padding.
- Bind memory allocations and execution threads to local NUMA nodes.
Trade-offs
Hardware cache line alignment and NUMA pinning deliver maximum multi-threaded throughput, but require explicit memory layout design and CPU topology awareness.