Linux TCP/IP Stack & Socket Buffers

Learning outcomes

• Tune Linux kernel socket buffer memory parameters
• Configure BBR congestion control for high-throughput networks

Mental model

When network packets arrive at a Network Interface Card (NIC), hardware DMA places frames into driver Ring Buffers, firing NAPI interrupts that allocate sk_buff kernel socket buffer structs processed through the Linux TCP/IP stack.

Theory

• sk_buff: The fundamental kernel data structure representing a network packet across all OSI layers.
• TCP SYN Cookies: Defends against TCP SYN Flood DoS attacks by encoding connection state into the initial TCP sequence number, eliminating unestablished connection state storage in the kernel SYN backlog queue.
• Congestion Control:
  • CUBIC: Loss-based congestion control; drops window size when packet loss occurs. Inefficient on lossy high-bandwidth long-distance networks.
  • BBR (Bottleneck Bandwidth and RTT): Model-based congestion control developed by Google; measures actual bandwidth and round-trip time, maximizing throughput without filling packet queues.

# Sysctl tuning for high-throughput Linux network performance
# Enable TCP BBR Congestion Control
sysctl -w net.core.default_qdisc=fq
sysctl -w net.ipv4.tcp_congestion_control=bbr

# Expand Max Socket Memory Buffers (Read/Write)
sysctl -w net.core.rmem_max=16777216
sysctl -w net.core.wmem_max=16777216
sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216"
sysctl -w net.ipv4.tcp_wmem="4096 65536 16777216"

# Enable TCP SYN Cookies
sysctl -w net.ipv4.tcp_syncookies=1

Alternatives and trade-offs

• CUBIC Congestion Control: Default standard; safe for low-latency LAN networks, severely degrades on lossy wireless connections.
• BBR Congestion Control: Maximum throughput on high-latency lossy networks; reduces bufferbloat latency spikes.

Failure modes and misconceptions

1. TCP Bufferbloat: Over-sizing socket memory buffers (rmem_max / wmem_max) without active queue management (fq / codel) causes network packets to sit in massive kernel queues, creating multi-second latency spikes.
2. SYN Backlog Overflow: Small net.ipv4.tcp_max_syn_backlog limits cause legitimate client connection attempts to be dropped during traffic bursts.

Decision scenario

Enable TCP BBR congestion control with fair queuing (fq) and tune socket buffer sizes on edge gateways to maximize network throughput while eliminating bufferbloat.

Learning outcomes

• Trace network packet processing through NIC Ring Buffers and kernel sk_buff structs.
• Defend against TCP SYN flood attacks using SYN Cookies.
• Configure BBR congestion control algorithms for high-performance networks.

Trade-offs

TCP BBR congestion control maximizes throughput over lossy networks, but requires modern kernel sysctl tuning and fair-queue scheduling.

Primary sources

• Linux Kernel Kernel.org Official Architecture & Systems Documentation — Linux Kernel Organization, verified 2026-07-23