The Longest Path in Computing

Every file read, network packet, and disk write travels the same long road: a user-space library call → a syscall → the virtual file system → the block layer → a device driver → a controller register → the physical device. From the Computer Architecture topic you know why: memory is the bottleneck. From this topic you’ll learn how the OS bridges that gap — with interrupts, DMA, and buffering — and why an operation that the CPU starts in nanoseconds can still end milliseconds later.

How a Program Reaches a Device

Two distinct levels of interface sit between software and hardware:

• Software interface — the syscall/VFS layer from earlier topics (read, write, open). The program never names a device directly; it names a file or a socket.
• Hardware interface — how the CPU actually talks to the device’s controller: register reads/writes plus either polling or interrupts.

The OS’s job is to translate the generic software interface into the concrete hardware one, via a device driver that knows the controller’s register layout and protocol.

Port-Mapped vs Memory-Mapped I/O

CPUs address devices two ways:

• Port-mapped I/O — dedicated I/O instructions (in/out on x86) address a separate I/O space. Simple isolation, but the special instructions are clunky and the space is tiny.
• Memory-mapped I/O (MMIO) — device registers appear at physical addresses in the same address space as RAM. The CPU just does loads/stores; the address decoder routes them to the device. This is the dominant modern approach (x86 PCIe, ARM, RISC-V all use it).

MMIO is why a driver looks like pointer arithmetic into a magic region of physical memory — and why I/O ordering matters: the CPU may reorder or buffer device register writes unless the driver uses explicit barriers or non-posted accesses.

Polling vs Interrupts

The CPU needs to know when a device has finished an operation. Two strategies:

• Polling — the CPU continuously checks the device’s status register until a “ready” bit appears. Predictable, no hardware support needed, but wastes CPU whenever the device is slower than the check interval.
• Interrupts — the device raises a hardware line; the CPU stops what it’s doing, saves state, runs an interrupt handler, and resumes. The CPU is free while the device works — but each interrupt carries context-switch overhead, so interrupt storms can thrash the system.

The engineering answer is a blend: poll when the device is always ready (network interfaces under heavy load poll; a fast NVMe device is often polled because interrupts would dominate), interrupt when the event is rare (keyboard, disk completion). Linux names this adaptive approach interrupt coalescing and NAPI polling.

The Interrupt Handling Flow

1. A device finishes its work and raises its interrupt line.
2. The interrupt controller (APIC on x86, GIC on ARM) maps the line to a vector and pokes the CPU.
3. The CPU saves the interrupted context and jumps to the interrupt service routine (ISR) registered by the driver.
4. The ISR — kept fast — acknowledges the device, stashes a few bytes, and defers the heavy work to a bottom half / softirq / tasklet / workqueue that runs later in a less-sensitive context.
5. The CPU restores the interrupted program and continues.

The split into a hard (top-half) and soft (bottom-half) is deliberate: interrupt handlers run with interrupts disabled (or masked), so they must be as short as possible. Doing the slow work in the deferred context keeps the system responsive.

DMA vs Programmed I/O

For a large transfer (say, a 64 KB disk block), having the CPU copy each byte to the device is wasteful. Programmed I/O (PIO) does exactly that — CPU-tight and fine for tiny transfers. Direct Memory Access (DMA) lets a dedicated controller move data straight between RAM and the device, then interrupt the CPU once when it’s done.

Modern systems go further with I/O memory management:

• Scatter-gather lists — the DMA controller handles physically scattered buffers.
• IOMMU (VT-d / AMD-Vi, ARM SMMU) — translates device-visible addresses, so DMA can’t read arbitrary kernel memory (a key security boundary — see the OS Security topic) and so user buffers can be handed to devices safely.

DMA is why a modern SSD read burns microseconds of CPU: the CPU issues the command, the controller moves the data, and the CPU gets a single completion interrupt.

Blocking, Non-Blocking, and Async I/O

From the process’s point of view, I/O comes in three flavors:

• Blocking — the syscall doesn’t return until data is available; the thread sleeps. Simplest to write; each I/O ties up a thread.
• Non-blocking — the syscall returns immediately with EAGAIN/WOULD_BLOCK if data isn’t ready; the caller must retry (usually in a loop or with a readiness mechanism like epoll/kqueue/IOCP).
• Asynchronous — the syscall queues the operation and returns; the OS calls back (signal, completion port, io_uring completion queue) when it’s done. Most scalable; most complex.

The transition from blocking to non-blocking/async is the same scaling story you saw in Threads & Concurrency: thread-per-I/O doesn’t scale to tens of thousands of concurrent connections, so high-concurrency servers use readiness- or completion-based I/O. Linux epoll + non-blocking sockets, and the newer io_uring for batched async, are the concrete tools; Windows uses IOCP.

The Block I/O Stack

Storage I/O gets an entire kernel machinery, because disk latency is so high:

• Page cache (from File Systems & Storage) absorbs repeated reads and defers writes.
• The block layer represents I/O as requests (or on Linux, the modern bio/blk-mq). It may merge adjacent requests and reorder them.
• I/O schedulers / elevators decide the order: FCFS (arrival), SSTF / shortest-seek (closest request first), elevator / SCAN (sweep in one direction to minimize head movement), and deadline / mq-deadline (give each request a deadline so reads aren’t starved). NVMe SSDs have no seek, so they usually skip the elevator entirely and use multi-queue scheduling.

The payoff: what looks like “I wrote 1000 small files” becomes a few large, contiguous device transfers — the buffering/batching rule from Computer Architecture applied at the storage layer.

The Visualizer

Use the disk I/O scheduling visualizer above to watch the head move across tracks 0–199 for the classic request queue [98, 183, 37, 122, 14, 124, 65, 67] (head starts at track 53). The disk bar shows each pending request, the amber head line, and the direction of travel; the metrics track total vs current seek; the service-order strip shows the exact order each algorithm picks.

Compare the totals for this same queue:

• FCFS — arrival order, total head movement 640 tracks. Simple and fair, but the head zig-zags across the whole disk.
• SSTF — always the closest request next, total 236. Far better average seek, but distant requests can starve.
• SCAN (elevator) — sweeps down, reaches the end of the disk, reverses, total 236. No starvation, because every track eventually gets swept.
• C-SCAN — sweeps in one direction only and wraps back, total 386. Even wait times: the arm always approaches each track the same way.
• LOOK — SCAN without the wasted end-of-disk travel, total 208.
• C-LOOK — C-SCAN without the wrap travel, total 326.

These six schedulers are the “elevator” family named in the block I/O stack above — and the exact trade-offs (fairness vs average seek vs starve-freedom) are the ones Linux’s mq-deadline and NVMe schedulers make in production.

Worked Example: A 4 KB File Read

Trace the full path for read(fd, buf, 4096):

1. Library call → syscall → VFS read on the file.
2. Page-cache lookup: hit? Copy to buf, return — no device involved (this is most reads).
3. Miss: the VFS asks the file system to find the blocks; the file system issues a block-layer request.
4. The block layer merges/schedules the request; the driver programs a DMA transfer from the device into a cache page.
5. The device does the I/O; the DMA completes; an interrupt wakes the waiter.
6. The kernel copies the page to buf; read returns 4096.

Steps 3–5 are where interrupts, DMA, and the block layer do their work — and why a page-cache hit is microseconds while a cold read is milliseconds.

Practice Trajectory

1. On Linux, compare a buffered vs O_DIRECT read with strace and a timing loop; on Windows, compare normal vs unbuffered (FILE_FLAG_NO_BUFFERING) file reads. Explain where the page cache hides the latency.
2. Read an I/O stats output (iostat -x on Linux, Performance Monitor Disk counters on Windows) and identify queue depth, utilization, and await time.
3. Write an epoll-based (or IOCP-based) echo server that handles a few thousand idle connections without a thread per connection; compare thread-per-connection.
4. Explain, in one paragraph, why a network driver under heavy load switches from interrupts to polling.
5. Sketch the interrupt path for a mouse click, naming where the ISR ends and the deferred (bottom-half) work begins.

When It’s the Right Tool