Caching: Trading Space for Latency
A cache stores the result of an expensive computation or read so that repeated requests skip the expensive path. The cache’s power comes from the pareto shape of access: a small set of hot keys usually accounts for most reads. Its danger is staleness: the cache and the source of truth can disagree. Every caching decision is a trade between how fresh and how fast.
Cache Placement
- Client caches — the browser’s HTTP cache (
Cache-Control,ETag). Zero server cost, but you don’t control it. - Edge caches / CDNs — shared caches at the network edge serving identical content (static assets, product pages) close to users.
- In-memory caches — Redis/Memcached in front of your database or computation. The classic “cache as a speed layer in front of a DB.”
- CPU caches / OS page cache — the same principles at a smaller scale (LRU policies, working-set behavior). Caching is a general law of systems, not a database feature.
The Strategies
| Strategy | Write path | Read path | Notes |
|---|---|---|---|
| Cache-aside (lazy) | Update DB; invalidate or update cache on write | Miss → read DB → populate cache | Simplest; the application orchestrates |
| Write-through | Write DB + cache together | Cache always has the value | Cache can’t go stale; every write pays both |
| Write-back | Write cache only, flush to DB later | Very fast writes | Risk of losing un-flushed writes on crash |
Cache-aside is the default choice: reads miss → fill the cache; writes go to the DB and invalidate the cached key. It’s robust because the cache is always re-fillable from the source of truth. Write-back is the risky high-performance variant (used for hot counters, session aggregation) where durability is intentionally deferred.
Eviction Policies
A cache is finite — when full, something must go:
- LRU (Least Recently Used) — evict the item not touched for the longest. The universal default; matches the hot-set shape well.
- LFU (Least Frequently Used) — evict the least-frequently-touched. Better for a stable hot set, worse at reacting to new trends.
- TTL (Time To Live) — expire keys on a timer regardless of access. The invalidation mechanism; a TTL is what saves you from serving a five-year-old price forever.
A TTL is not an eviction policy but it is the simplest correct invalidation: every key gets a lifetime, and expiry means the cache re-fetches from the source. The art is choosing TTLs that bound staleness to what your product tolerates.
Invalidation Hazards
- Cache stampede (thundering herd) — a hot key expires, and ten thousand concurrent requests all miss and all hit the database at once. Fixes: locking / single-flight (one request refills, others wait), or probabilistic early recomputation (refresh before the TTL actually expires).
- Cache-aside with update-then-invalidate — if you update the DB and the cache is invalidated in the wrong order, a racing reader can refill the cache with the old value right after your invalidation. The robust patterns are write-through (cache updated atomically with the DB) or versioned keys (
user:42:v7). - Negative caching — caching “not found” responses too (with a short TTL) so a hot missing key doesn’t hammer the source.
Redis: The Operational Cache
Redis is the industry’s in-memory cache/coordination layer: key-value ops at microsecond latency, rich structures (hashes, sorted sets, streams), TTLs, and cluster sharding. It is also the workhorse behind rate limiting, distributed locks, leaderboards (sorted sets), and session stores. Its power is the same as its hazard: data lives in RAM — you must design what is evictable and what belongs back in durable storage.
CDNs and HTTP Caching
A CDN caches at the edge of the network so users hit a nearby node instead of your origin. The contract that makes CDNs safe is HTTP’s Cache-Control:
max-age=3600— CDN and browsers may reuse for an hour.no-cache— revalidate with the origin before reuse (usesETag/Last-Modified).no-store— never cache (auth pages, PII).
Static assets get long max-ages + versioned URLs (bundle.abc123.js) so invalidation is “deploy a new URL,” not “purge everywhere.” This is the same cache law at a different scale: cache hard, and design the cache key so you never need a global purge.
The Visualizer
Use the cache visualizer above to step through requests against a small cache. Compare cache-aside (fill on miss, invalidate on write) with write-through (keeps the cache always current) and write-back (delays durability for speed), and watch LRU vs FIFO eviction decide which hot key gets dropped when the cache overflows.
Practice Trajectory
- Trace a cache-aside read: miss → source read → cache write; then a write: source write → cache invalidate.
- Simulate a stampede: expire a hot key with 10 concurrent readers and design the single-flight fix.
- Decide TTLs for a product catalog vs a user session vs an auth token, and defend each number.
- Draw the consistency hazard where update-then-invalidate races a reader; then apply a versioned key.
- Pick eviction policies for a news-feed cache (trending) vs a stable dictionary cache — and justify.
When It’s the Right Tool
| Situation | Takeaway |
|---|---|
| Repeated expensive reads | Cache-aside with LRU + TTL |
| Hot key expires and thunders | Single-flight or early refresh |
| Static/global content | CDN with versioned URLs + max-age |
| Freshness matters more than speed | Write-through or no cache |
| Durable-but-slow writes | Write-back only with a flush safety net |