Why the Build Matters

Every line of code in production went through a build: the transformation from source files into an artifact (binary, container image, deployable bundle) that can be tested and shipped. CI/CD systems are, at their core, automated build-and-test pipelines.

A systems engineer who understands the build understands why “it works on my machine” happens, how to make deployments reproducible, and how to shave minutes off every release.

Compile, Interpret, JIT

The boundary is blurry (Python compiles to bytecode; V8 compiles to native). The mental model that matters: AOT gives fast startup and direct control; JIT gives portability plus adaptive optimization; interpreted gives simplicity.

Container runtimes and OS processes care about AOT because the binary is what the kernel executes.

The C Toolchain

A C program is not compiled in one step — it passes through four stages:

1. Preprocessor (cpp) — expands #include and #define macros into one giant translation unit.
2. Compiler (cc) — translates C to assembly for the target CPU.
3. Assembler (as) — turns assembly into an object file (.o) containing machine code plus a symbol table of functions/variables it defines and needs.
4. Linker (ld) — combines object files and libraries, resolves symbols, and produces the final executable.

gcc -E main.c -o main.i        # preprocess only
gcc -S main.c -o main.s        # compile to assembly
gcc -c main.c -o main.o        # assemble to object file
gcc main.o -o app              # link
gcc -O2 -Wall -g main.c -o app # typical one-shot

The -O2/-O3 flags enable the optimizer — the same source can be 5–50× faster or slower based on optimization and target. This is why release builds differ from debug builds.

“Works in debug, breaks in release” is a real phenomenon: optimizer bugs, uninitialized reads, and strict aliasing surface only under optimization.

Static vs Dynamic Linking

The linker has two ways to pull in libraries:

• Static (.a) — library code is copied into your executable. Self-contained, larger, version-immutable.
• Dynamic (.so / .dll) — the executable records a dependency; the OS loads the library at runtime. Smaller, upgradeable, but version-sensitive (ldd shows dependencies; a missing .so is a classic “it won’t start” failure).

Containers essentially freeze a set of dynamically linked libraries into an image so the binary finds its dependencies — the modern answer to dependency drift.

Build Systems

A build system decides what to rebuild when files change and in what order. The minimum viable build tool is a script; the industry tools are:

The key engineering properties of a good build are incrementality (rebuild only what changed), determinism (same input → same output), and hermeticity (no dependence on the state of the builder’s machine). Bazel enforces all three; Make requires discipline.

Package Managers

Builds are rarely self-contained — they depend on libraries that come from package managers, which resolve, download, and lock dependencies:

• Language-level: npm, pip, cargo, go modules, Maven.
• OS-level: apt/yum/apk (also used to assemble container base images).

The critical practice is locking: a lockfile (package-lock.json, Cargo.lock, go.sum) records exact versions and hashes so every environment builds the identical dependency graph. “It works on my machine” is usually a missing or stale lockfile.

Reproducible Builds

A build is reproducible when the same source at the same revision produces byte-identical artifacts. Threats to reproducibility:

• timestamps embedded at compile time
• non-deterministic map iteration
• machine-specific paths
• un-pinned dependencies

Reproducibility is what makes supply-chain verification possible — you can verify that a published binary matches the source. Container images and CI both push toward this: pin everything, build from clean checkouts, and make the build the single source of truth.

CI Wiring

In practice the build lives inside a CI pipeline. A commit triggers a runner that:

1. checks out the source at a known revision
2. installs the locked toolchain
3. builds
4. runs tests
5. produces an artifact that CD then deploys

The build stage failing is the cheapest failure in the entire system — which is why CI culture prizes fast, deterministic builds: the shorter the feedback loop, the faster teams can ship.

Practice Trajectory

1. Build a small C program with gcc step by step (-E, -S, -c, link) and inspect the assembly and symbols with nm/objdump.
2. Compare a static and a dynamic build (-static vs default), then run ldd on the dynamic one and deliberately break a dependency.
3. Write a Makefile with phony targets and incremental compilation; verify make skips unchanged files.
4. Generate a lockfile for a project you know (e.g., npm install → package-lock.json) and explain what a full rebuild reads from it.
5. Time a clean CI build vs an incremental one and identify which stage dominates — then propose one change to speed it up.

When It’s the Right Tool