Threads & Concurrency

Threads vs Processes

A process is the OS’s unit of resource ownership: an address space, file descriptors, and a PCB. A thread is the unit of execution: a program counter, a stack, and register state. Threads inside one process share the process’s address space and files — which makes them cheap to create and switch (no address-space swap, no TLB flush) but dangerous to use (shared mutable state).

This trade-off drives most systems design: processes give isolation (containers, microservices), threads give throughput on shared data (servers, runtimes). A modern server is typically one process, many threads.

User-Level vs Kernel-Level Threads

There are two ways threads can exist:

• Kernel threads — the OS knows each thread and schedules it; a blocking syscall blocks only that thread. Slightly higher overhead per thread.
• User-level threads (green threads) — the runtime multiplexes many lightweight threads onto a few kernel threads; switching is very cheap but a blocking call can stall the whole group unless the runtime is careful.

Most production systems are hybrids. Go’s goroutines and Rust’s Tokio are user-level runtimes scheduled on a pool of kernel threads; Java and C++ use kernel threads (with JIT/runtime pools). The practical rule: green threads scale to hundreds of thousands, kernel threads to thousands — with kernel threads giving simpler semantics for blocking I/O.

Race Conditions and Atomicity

The core problem of concurrency: two threads reading and writing the same variable interleave arbitrarily. counter++ is not atomic — it’s a read-modify-write, and two threads can both read the old value and both write the same result, losing an increment.

A race condition is any bug caused by such unsynchronized access. The fix is atomicity: make a critical section indivisible. The OS provides primitives:

• Mutex (mutual exclusion) — a lock; only one thread holds it at a time.
• Semaphore — a counter with wait/signal; generalizes locking (mutex ≈ binary semaphore) and enables producer/consumer.
• Condition variable — lets a thread wait until a condition is true (paired with a mutex); the standard way to block on “queue non-empty.”

Using these correctly is subtle — the discipline is: lock around shared state, unlock on every exit path (RAII/locks in the language), and never hold locks while blocking on user I/O.

Deadlock

Deadlock = every thread waits on a resource held by another, so nothing progresses. The four necessary conditions (Coffman):

1. Mutual exclusion — resources are non-sharable.
2. Hold and wait — a thread holds a resource while waiting for another.
3. No preemption — resources can’t be forcibly taken.
4. Circular wait — threads form a cycle of waiting.

Break any one and deadlock disappears. Practically, teams avoid circular wait: impose a global lock ordering (always acquire locks A→B→C), or use a single lock + timeouts (pthread_mutex_timedlock), or in Go prefer channels to mutexes. The kernel detects the pathological cases with wait-for graphs (that’s what check_deadlock/sysrq traces show).

Condition Variables and Producer/Consumer

The classic pattern: a producer fills a buffer, a consumer drains it. A naive busy loop (while (empty) ;) burns CPU; a condition variable sleeps properly:

pthread_mutex_lock(&m);
while (queue_empty)           // re-check: spurious wakeups
  pthread_cond_wait(&cv, &m); // releases m while waiting
item = dequeue();
pthread_mutex_unlock(&m);

The while (not if) matters: wait can wake spuriously, and another consumer may have drained the queue. This pattern is the foundation of thread pools, message queues, and the scheduler’s own blocking/waking machinery.

Thread Pools

Creating a thread per request doesn’t scale (allocation + context-switch overhead, unbounded resource use). A thread pool pre-creates N workers that pull work from a shared queue; the queue uses the producer/consumer pattern above. N is typically tuned to the number of cores (CPU-bound) or I/O concurrency (I/O-bound). Almost every server — Java executors, Go’s scheduler, Node’s worker pool, and the runtime behind your web server — is a thread pool.

Thread-Safe Data Structures

Synchronization is a scaling bottleneck, so libraries ship lock-free structures (atomics, CAS, std::atomic, Go channels, Java ConcurrentHashMap) that coordinate with hardware atomics instead of blocking. The trade: lock-free code is fast and deadlock-free but notoriously hard to write correctly — prefer battle-tested libraries over hand-rolled lock-free logic.

Practice Trajectory

1. Write a program where 10 threads each increment a shared counter 100,000 times; observe the lost-update race, then fix it with a mutex.
2. Reproduce deadlock with two mutexes acquired in opposite order across two threads; fix with a global lock ordering.
3. Build a producer/consumer with a bounded buffer using a condition variable (or Go channels) and verify it blocks instead of busy-waiting.
4. Use top/htop (-H for threads) on a busy server and identify how many threads a process has and why.
5. Compare a thread-per-request server vs a fixed thread pool under wrk load; note the throughput difference.

When It’s the Right Tool