Kubernetes: The Distributed Orchestrator

Kubernetes (k8s) is a platform that runs and manages containerized workloads across a cluster of machines. Where Docker gives you one host’s containers, Kubernetes gives you declarative infrastructure: you describe the desired state (I want 3 replicas of this image), and the control plane continuously drives the cluster toward it. If a pod dies, a node fails, or traffic spikes, Kubernetes reconciles reality to the declared state.

Architecture

The cluster is split into the control plane (the brain) and worker nodes (the muscles):

• Control plane — kube-apiserver (the single API endpoint — everything talks to it), etcd (the durable store of cluster state, a Raft-based key-value database — remember consensus), kube-scheduler (decides which node runs a new pod), kube-controller-manager (runs the reconciliation loops: Deployments, ReplicaSets, Node lifecycle).
• Worker nodes — kubelet (the agent that runs pods on the node and reports status), kube-proxy (implements Services/networking rules), plus the container runtime (containerd/cri-o) and kubeadm/k3s tooling.

Every kubectl apply is a write to etcd through the apiserver; every controller watches that state and acts. The whole system is the replicated-state-machine pattern from the consensus topic, running at infrastructure scale.

Pods: The Atomic Unit

A pod is the smallest deployable unit: one or more containers that share a network namespace, IP, and storage volumes — always co-located and co-scheduled. The common case is one container per pod, but sidecars (a logging agent, a proxy alongside the app) are why pods exist.

Pods are ephemeral by design: they can be killed and rescheduled at any time, and their IP is not stable. You almost never manage pods directly — you manage a controller that manages pods.

Deployments and ReplicaSets

A ReplicaSet keeps a desired number of identical pod replicas running. A Deployment wraps a ReplicaSet and adds rollouts: update the pod template, and it rolls new pods out incrementally with configurable strategy:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
spec:
  replicas: 3
  selector:
    matchLabels: { app: api }
  template:
    metadata:
      labels: { app: api }
    spec:
      containers:
        - name: api
          image: registry.example.com/api:v2
          ports: [{ containerPort: 8080 }]

• Rolling update — replaces pods one by one; maxSurge/maxUnavailable control the pace.
• Rollback — kubectl rollout undo reverts to a previous ReplicaSet. Every change is versioned.
• Self-healing — a killed pod is recreated elsewhere; the deployment drives 3 replicas forever.

Services: The Stable Doorway

Pods come and go; Services give them a stable identity and load-balance traffic:

• ClusterIP (default) — a virtual IP reachable only inside the cluster; the standard way services call each other.
• NodePort — exposes the service on a port on every node (node-ip:30000) — good for dev and for ingress to reach.
• LoadBalancer — provisions a cloud load balancer (AWS ELB, GCP LB) that fronts the service. The classic “expose to the internet” path.

A Service selects pods by label selector — this is how k8s keeps the mapping alive as pods churn: selector: { app: api } matches the Deployment’s pod labels. Combined with DNS (service-name.namespace.svc.cluster.local), Service discovery is built in.

ConfigMaps and Secrets

Containers shouldn’t hardcode configuration. ConfigMaps hold non-sensitive config (environment variables, config files) as key: value data; Secrets hold sensitive data (tokens, TLS certs, DB passwords) — base64-encoded, mountable as env vars or files. Both are injected at pod start. Secrets are not encryption at rest by default (that requires encryption-at-rest config or an external provider like Vault) — treat them as “obfuscated config + RBAC guardrails,” and put the crown jewels in a real secrets manager.

Ingress: Routing at the Edge

For a fleet of HTTP services you don’t want a LoadBalancer per service. An Ingress resource declares hostname/path routing rules, and an Ingress controller (nginx, Traefik, the cloud LB controller) implements them: api.example.com/* → the api service, app.example.com/* → the web service, TLS via certificates. One external entry point, rules in front of all Services.

Autoscaling

• Horizontal Pod Autoscaler (HPA) — scales replicas on CPU/memory/custom metrics: targetAverageUtilization: 70 means “keep replicas so average CPU is ~70%.”
• Cluster Autoscaler — adds/removes nodes when pending pods can’t be scheduled.
• Vertical Pod Autoscaler — adjusts resource requests per pod (less common; rolling changes).

HPA is the production workhorse — but it only works if pods have requests/limits set, so the scheduler and HPA have a basis to measure. A pod without resource requests is both a scheduling guess and an HPA black box.

Probes: How kubelet Knows a Pod Is Healthy

Without health checks, a pod that has “started” but serves 500s forever is indistinguishable from a healthy one — and rolling updates would happily ship broken replicas. Three probes, answered by kubelet on each node:

• Liveness probe — “is the process alive (or deadlocked)?” If it fails, kubelet restarts the container (restartPolicy: Always). Catches hangs, deadlocks, and memory-locked loops. Default backoff escalates the restart delay.
• Readiness probe — “is the container ready to receive traffic?” If it fails, the pod is removed from Service endpoints but not restarted. Catches the classic “started but still warming up / loading a model / connecting to the DB” window, so the Service never routes to a pod that will error.
• Startup probe — for slow-booting containers (heavy JVMs, big caches): it gates the other probes until the app is truly up, preventing liveness from killing a container that just needs 90 seconds to boot.

All three support the same three check styles:

containers:
  - name: api
    image: registry.example.com/api:v2
    ports: [{ containerPort: 8080 }]
    readinessProbe:
      httpGet: { path: /healthz, port: 8080 }
      initialDelaySeconds: 3
      periodSeconds: 5
    livenessProbe:
      httpGet: { path: /healthz, port: 8080 }
      periodSeconds: 10
    startupProbe:
      httpGet: { path: /healthz, port: 8080 }
      failureThreshold: 30
      periodSeconds: 10

The golden rule: readiness checks the app’s dependencies (DB, cache, warmup); liveness checks only the process itself. A common production bug is a liveness probe pointing at an endpoint that fails when dependencies are down — the pod gets killed in a restart loop instead of being drained, and every node ends up hammering a degraded database. Liveness should stay dumb (the process is responsive); readiness carries the load-shedding logic.

Practice Trajectory

1. Run a Deployment with 3 replicas locally (kind/minikube); kubectl scale it and watch pods reconcile.
2. Create a Service and verify DNS + load balancing from inside a test pod.
3. Do a rolling update with a bad image, then kubectl rollout undo — observe the versioned rollback.
4. Set CPU requests/limits and attach an HPA; generate load and watch replica count react.
5. Add liveness/readiness probes to a Deployment; break the readiness path and observe the pod being drained but not restarted.
6. Split config into a ConfigMap and a Secret; mount both into a pod and verify injection.

When It’s the Right Tool