DevOps
The Canary That Received All the Traffic
The weight was set on the route and not on the service, so the one-pod canary took the full load, and the errors were the capacity rather than the code, and…
DevOps
The Volume That Would Not Grow Until the Pod Restarted
The claim was expanded and the filesystem stayed the same size, so the application kept hitting the old limit while the dashboard showed the new capacity…
DevOps
The ConfigMap Change That Nobody Restarted
The config was updated in the cluster and the pods kept running with the values they read at boot, so the change was live in the manifest and absent in the…
DevOps
The Node That Evicted Everything Because Logs Filled the Disk
Container logs grew without rotation until the node hit disk pressure, and the kubelet evicted every pod on it, including the healthy ones, and the eviction…
Error Autopsy
The Sidecar That Bound the App's Port Before the App Could
The pod crash looped with EADDRINUSE on its own port at boot, because the sidecar started faster and bound the port first, and the app, losing the race it…
DevOps
The Autoscaler That Scaled on a Metric That Lagged the Traffic
The HPA added pods after the spike had passed and removed them after the next one began, because its metric was smoothed, delayed and averaged.
DevOps
The Liveness Probe That Killed the App When the Database Was Slow
The probe queried the database, so the slow database made the app look dead, and the restarts made the database slower, and the loop ran until someone read…
Deep Dive
Readiness, Liveness and the Probe That Restarted a Healthy Pod
The pod was serving fine until Kubernetes killed it, repeatedly, because a liveness probe shared the event loop with the work.
Error Autopsy
The Health Check the CDN Answered Instead of the App
The probe hit the public hostname and the edge returned a cached 200, so the platform believed the service was healthy while every pod behind it was…
DevOps
The Ingress Annotation That Silently Changed Your Timeout
The app's timeouts were tested and correct, and the 504s kept coming, because the ingress controller has its own defaults for body size, buffering and read…
DevOps
Your Node Drain Hung Because of a Pod Disruption Budget
The node upgrade stopped at one node and the drain sat for hours. A Pod Disruption Budget with minAvailable equal to the replica count means nothing can…
DevOps
Requests, Limits and the QoS Class That Decided Who Died
When a node runs short of memory, Kubernetes does not choose eviction victims at random. It uses the QoS class, which your requests and limits silently…