Security
The Debug Endpoint That Was Still Enabled in Production
The route was added for one investigation and guarded by a flag nobody set back, so it served the environment and the stack traces to anyone who guessed the…
Tooling
The dig Query That Showed Which Resolver Was Lying
The domain resolved differently depending on who was asked, and querying each resolver in turn separated the stale cache from the broken record, and the…
Tooling
The jq Filter That Found the One Different Field
Two payloads looked identical to the eye and differed in one nested field, and the filter diffed them in one line where the visual comparison had taken an…
Tooling
The tcpdump Capture That Showed Who Sent the Reset
The connection dropped at the same byte count every time, and the capture showed the reset arriving from the load balancer rather than from the application…
Tooling
The git bisect Run That Found the Commit in Nine Steps
Three hundred commits sat between the working build and the broken one, and the binary search narrowed it to a single change in nine tests instead of three…
Tooling
The curl Command That Reproduced the Bug in Four Seconds
The report was a screenshot and a guess, and the reproduction was one command with the right headers, and the command turned an hour of speculation into a…
Shopify
The Shopify Webhook Retry That Double Booked the Order
The delivery timed out after the handler had already run, so the platform retried, and the handler, which assumed delivery meant not processed, booked the…
Engineering Culture
One Hundred Posts About Things Breaking
What I learned writing a hundred debugging writeups, including which ones people actually read and what that says about technical content.
Code Review
Reviewing Python for the Mutable Default Argument and Its Cousins
The shared list default is the famous one, and reviewers catch it on sight. The module level cache, the captured dict and the class attribute list are its…
Tooling
The strace Output That Showed the File Was Never Opened
The config was ignored and the logs said nothing, and the syscall trace showed the open call failing on a path that differed by one character from the path…
Tooling
Reading /proc Like a Dashboard
The kernel already exports every counter you would install an agent to collect. The files under /proc are the metrics endpoint, and reading them is the…
Debugging
Debugging an Agent That Will Not Reproduce Its Own Mistake
The agent broke the build once, and no rerun would do it again, because the failure was a draw from a distribution, not a function of the input.