One core, two ways to wait: threads and the event loop

Here is what "suspends the thread" means in the kernel. On Linux every thread is a task_struct. To wait, a task puts itself on a wait queue, sets its state to sleeping, and calls the scheduler, which picks some other task to run. When the awaited thing happens, whoever caused it walks the wait queue and wakes the sleepers.

The thread-per-connection server blocks in read(), which for TCP reaches sk_wait_data and waits on that one socket's queue. The event-loop server blocks in epoll_wait(), which reaches ep_poll and waits on one queue for every socket it watches. The mechanism is identical. The difference is the number of sleepers: N threads each waiting on one socket, versus one thread waiting on N.

These are minimal C programs, not from any project, written to make the structural difference concrete. The left one creates a thread per connection, and each thread spends its life blocked in read(). The right one has a single thread that blocks in epoll_wait() and handles whichever sockets are ready. Error handling is omitted.

On Linux there is no separate "thread" object in the kernel. A thread is a task created by clone with flags saying which resources to share with its creator. glibc's create_thread spells out exactly what our report describes: share memory (CLONE_VM), cwd and umask (CLONE_FS), file descriptors (CLONE_FILES) and signal handlers (CLONE_SIGHAND), join the same thread group so the process has one PID (CLONE_THREAD), and give the new task its own stack and thread-local storage.

This is that function. When the scheduler picks a different task, it calls context_switch. If the next task is a user task with a different address space, switch_mm_irqs_off loads its page tables (CR3 on x86). Two threads of the same process share mm, so that step costs almost nothing between them. Then switch_to, a short piece of architecture-specific assembly, swaps the stack pointer and callee-saved registers. When switch_to "returns", it returns on the next task's stack, into wherever that task last called schedule().

counter++ compiles to three instructions: load into a register, add, store. Each thread has its own registers, so if the scheduler interleaves those instructions, both threads can load the same old value. Step through it below: pick an interleaving, then turn the mutex on.

The mutex is glibc's futex lock. Its word holds 0 (unlocked), 1 (locked, no waiters) or 2 (locked, maybe waiters). The fast path is a single atomic compare-and-swap in user space with no system call. Only when it fails does __lll_lock_wait set the word to 2 and call futex_wait, which puts the thread to sleep in the kernel. Unlock swaps the word back to 0 and, only if it was 2, calls futex_wake.

When we epoll_ctl(ADD) a socket, the kernel hooks a callback, ep_poll_callback, onto that socket's own wait queue: the same queue a blocked read() would sleep on. When a packet arrives and the TCP stack wakes the socket's queue, this callback runs instead of waking a thread. It appends the socket to the epoll instance's ready list and wakes the one task sleeping in ep_poll.

libuv's uv__io_poll is the user-space side: call epoll_pwait, then loop over the returned events and dispatch each to its watcher. In Node, that dispatch eventually runs our JavaScript callback.

This is a small discrete-event simulation with one CPU core. Each request needs some CPU time, may wait on I/O (a database or network call), then needs more CPU. In the threads model, each request gets a thread, the scheduler runs ready threads round-robin with a time slice, and every switch between threads costs CPU time. In the event loop model, one thread runs each callback to completion and starts I/O without blocking.

Things to look for. With the I/O-bound workload the two models finish at about the same time, but the threads row pays for every switch, and on a real server that overhead grows with the number of runnable threads. With CPU-bound jobs on one core neither model can go faster; threads just interleave the work and add switches, so the mean latency gets worse. More cores are what help, and only threads can use them without extra processes. With one expensive request, the event loop's head-of-line blocking appears: every cheap request that becomes ready while the expensive callback runs waits behind it, while the time slice lets threads keep cheap requests moving.

Two corrections make this more precise than the usual "8 MB per thread" line. First, 8 MB is reserved virtual address space, not RAM. Linux backs stack pages lazily, so an idle thread typically uses only the pages it has touched, often tens of KB of resident memory. Second, the cost that is always resident is the kernel side: a task_struct plus a 16 KB kernel stack per thread, and a scheduler that has more runnable tasks to consider. The model shows reserved stack and kernel stack separately for that reason. Socket buffers are the same in both models, so they are left out.

Tracing the source turned up a few places where the report is right in spirit but can be stated more precisely.

Five questions in the style of a CSE321 theory exam on concurrency. Write our answer on paper first, then open the model answer and mark ourselves honestly. The marks follow the usual pattern of university theory exams; they are not copied from any real paper.