Computer Architecture Fundamentals

Why Study Hardware Before Software

Every OS feature — scheduling, virtual memory, paging, file caching — exists because of a hardware constraint. You can’t reason about why the OS pages memory, why O(1) algorithms still get slow, or why one data layout beats another without knowing roughly how fast each level of the machine is. This topic gives you the mental model: a CPU that executes instructions far faster than memory can feed it, patched by a hierarchy of increasingly large, increasingly slow storage.

The Fetch-Execute Loop

A CPU runs a program as an endless loop over instructions:

1. Fetch the next instruction from memory (address in the program counter / PC).
2. Decode what it does (opcode, operands).
3. Execute it — arithmetic, load/store, branch, etc.
4. Advance the PC and repeat.

Everything a program does — function calls, loops, if statements, reading a file — reduces to this loop over an instruction set. A modern CPU doesn’t literally execute one instruction at a time; it pipelines them (fetch while executing the previous), fetches several at once (superscalar), and speculates ahead of branches. But the observable model you need is the fetch-execute loop plus one hard fact: memory is the bottleneck.

The Memory Hierarchy

The core trick of every computer is a tiered store — each level bigger, slower, and cheaper than the last:

Notice the jumps: RAM is ~100× slower than L1, and an SSD is ~1000× slower than RAM. Latency, not bandwidth, is the fundamental currency — a random-access read from an SSD costs about as much as hundreds of thousands of L1 hits.

Caches and the Principle of Locality

Caches work only because programs are local:

• Temporal locality — if you used a memory location, you’ll likely use it again soon (loops, hot variables).
• Spatial locality — if you used one location, you’ll likely use nearby ones (arrays, sequential scans).

The cache copies a block (cache line) of nearby memory into a fast store; later accesses hit the cache. A cache miss is the expensive event — the CPU stalls waiting on a lower tier. This is why data layout matters more than algorithm constant factors: iterating an array row-major vs column-major across a 2D structure can be 10–50× apart purely due to locality, even for the same operation count.

Working set is the set of pages your program actively touches; when it exceeds a cache or RAM tier, performance collapses — the same collapse the OS’s page replacement policies (FIFO/LRU/Optimal, which you’ll animate in the memory-management studio) try to prevent.

Storage Tiers

Persistent storage is another hierarchy, and each tier has a different cost model:

• SSDs — random access ~µs, no mechanical seek, wear out with writes, price per GB moderate.
• HDDs — sequential reads are fast but random seeks cost ~ms; cheap per GB.
• Object / network storage (S3, NFS) — effectively unlimited, but latency measured in milliseconds, and the bottleneck moves to the network.

The consequence for systems design: sequential beats random, in-memory beats on-disk, and buffering/batching (write-ahead logs, journaling, page caches) exists to amortize the slow tiers. When you study file systems, databases, and distributed storage, you’re seeing the same hierarchy at ever larger scale.

Virtual vs Physical Memory (Preview)

The OS gives every process its own virtual address space and maps it to physical RAM through page tables, with an on-CPU TLB (translation lookaside buffer) caching those mappings — because otherwise every memory access would double in cost. A miss in the TLB or page table drives a page fault, and a fault with no physical frame free triggers page replacement. This is the hardware mechanism the Memory Management topic and its interactive address-translation studio build on. For now, just keep the chain: virtual address → TLB → page table → physical frame → maybe disk.

Endianness and Representation

Hardware also dictates byte order. Little-endian (x86, most ARM) stores the least-significant byte at the lowest address; big-endian stores the most significant first. It only matters when you read raw bytes (protocols, files, binary formats) — but it explains real-world bugs, so keep the concept on hand.

Practice Trajectory

1. Use lscpu to find your CPU’s cache sizes and free -h for RAM — confirm the hierarchy’s size ratios.
2. Write two loops that sum the same 2D array row-major vs column-major and time them; expect a big gap. Explain it with locality.
3. Measure a 1 GB read from an SSD vs RAM with dd/hdparm (careful!) and compare to the latency table above.
4. Explain, in one paragraph, why the OS would ever “waste” RAM caching disk blocks even when memory is “full.”
5. Strace a small program and identify which syscalls are purely about moving data between user memory and the kernel.

When It’s the Right Tool