Mental model
To handle tens of thousands of concurrent TCP sockets without spawning a thread per socket, Linux uses I/O multiplexing. epoll registers interest in socket file descriptors in a kernel Red-Black tree and returns ready sockets via a ready list in $O(1)$ time.
Create epoll Instance (epoll_create1)
Register Socket FDs in Kernel Red-Black Tree
Kernel interrupt pushes active sockets to Ready List
epoll_wait() fetches active events in O(1) time
Event Loop processes non-blocking socket I/O
Theory
select()/poll(): Copies file descriptor sets between user and kernel space on every call; performs linear $O(N)$ scanning over all descriptors. Fails at scale.epoll(epoll_create,epoll_ctl,epoll_wait): Maintains in-kernel Red-Black tree of watched descriptors. Operates in $O(1)$ time relative to total watched sockets.- Level-Triggered (LT): Default mode;
epoll_wait()continues notifying as long as unread data remains in the socket buffer. - Edge-Triggered (ET):
epoll_wait()notifies ONCE when new data arrives. Requires reading the socket untilEAGAINorEWOULDBLOCK.
c(24 lines)
1// C snippet: Creating an epoll event loop
2#include <sys/epoll.h>
3#include <unistd.h>
4#include <stdio.h>
5
6#define MAX_EVENTS 64
7
8int main() {
9 int epoll_fd = epoll_create1(0);
10 struct epoll_event ev, events[MAX_EVENTS];
11
12 // Watch socket_fd for read events (Level Triggered)
13 ev.events = EPOLLIN;
14 ev.data.fd = 0; // STDIN
15 epoll_ctl(epoll_fd, EPOLL_CTL_ADD, 0, &ev);
16
17 // Wait for events
18 int nfds = epoll_wait(epoll_fd, events, MAX_EVENTS, 5000);
19 printf("Active file descriptors: %d\n", nfds);
20
21 close(epoll_fd);
22 return 0;
23}
Alternatives and trade-offs
- Thread-Per-Socket: Simple synchronous code; severe RAM overhead (1MB stack per thread) and CPU context switching degrade performance past 1,000 threads.
epollEvent Loop (Uvicorn / Node.js / Nginx): Handles 100,000+ concurrent connections on a single thread with minimal RAM footprint.
Failure modes and misconceptions
- Edge-Triggered Starvation & Deadlock: In Edge-Triggered (
EPOLLET) mode, failing to loopread()until receivingEAGAINleaves remaining buffer data unread, causing the socket to hang indefinitely. - Blocking Sockets in epoll: Registering blocking file descriptors in an
epollevent loop blocks the entire event loop thread if a read operation blocks. Sockets MUST be configured withO_NONBLOCK.
Reflect before revealing the guide
Decision scenario
Build non-blocking high-concurrency network servers using Linux epoll in Level-Triggered mode to achieve $O(1)$ event multiplexing without socket starvation bugs.
Learning outcomes
- Compare $O(N)$ file descriptor polling (
select/poll) with $O(1)$epoll. - Distinguish Level-Triggered (LT) from Edge-Triggered (ET) event notification modes.
- Configure non-blocking file descriptors (
O_NONBLOCK) for event loop processing.
Trade-offs
epoll enables scalable $O(1)$ socket event loops for high-concurrency servers, but requires asynchronous non-blocking event handling state machines.