Mental model
When network packets arrive at a Network Interface Card (NIC), hardware DMA places frames into driver Ring Buffers, firing NAPI interrupts that allocate sk_buff kernel socket buffer structs processed through the Linux TCP/IP stack.
NIC Ring Buffer DMA receive
NAPI Interrupt fires
Allocate Kernel sk_buff struct
TCP Stack processing (TCP Window & Congestion Control)
Socket Receive Buffer -> Application recv()
Theory
sk_buff: The fundamental kernel data structure representing a network packet across all OSI layers.- TCP SYN Cookies: Defends against TCP SYN Flood DoS attacks by encoding connection state into the initial TCP sequence number, eliminating unestablished connection state storage in the kernel SYN backlog queue.
- Congestion Control:
- CUBIC: Loss-based congestion control; drops window size when packet loss occurs. Inefficient on lossy high-bandwidth long-distance networks.
- BBR (Bottleneck Bandwidth and RTT): Model-based congestion control developed by Google; measures actual bandwidth and round-trip time, maximizing throughput without filling packet queues.
bash(14 lines)
1# Sysctl tuning for high-throughput Linux network performance
2# Enable TCP BBR Congestion Control
3sysctl -w net.core.default_qdisc=fq
4sysctl -w net.ipv4.tcp_congestion_control=bbr
5
6# Expand Max Socket Memory Buffers (Read/Write)
7sysctl -w net.core.rmem_max=16777216
8sysctl -w net.core.wmem_max=16777216
9sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216"
10sysctl -w net.ipv4.tcp_wmem="4096 65536 16777216"
11
12# Enable TCP SYN Cookies
13sysctl -w net.ipv4.tcp_syncookies=1
Alternatives and trade-offs
- CUBIC Congestion Control: Default standard; safe for low-latency LAN networks, severely degrades on lossy wireless connections.
- BBR Congestion Control: Maximum throughput on high-latency lossy networks; reduces bufferbloat latency spikes.
Failure modes and misconceptions
- TCP Bufferbloat: Over-sizing socket memory buffers (
rmem_max/wmem_max) without active queue management (fq/codel) causes network packets to sit in massive kernel queues, creating multi-second latency spikes. - SYN Backlog Overflow: Small
net.ipv4.tcp_max_syn_backloglimits cause legitimate client connection attempts to be dropped during traffic bursts.
Reflect before revealing the guide
Decision scenario
Enable TCP BBR congestion control with fair queuing (fq) and tune socket buffer sizes on edge gateways to maximize network throughput while eliminating bufferbloat.
Learning outcomes
- Trace network packet processing through NIC Ring Buffers and kernel
sk_buffstructs. - Defend against TCP SYN flood attacks using SYN Cookies.
- Configure BBR congestion control algorithms for high-performance networks.
Trade-offs
TCP BBR congestion control maximizes throughput over lossy networks, but requires modern kernel sysctl tuning and fair-queue scheduling.