Bug Hunt
The Race Condition That Vanished When I Added Logging
The classic heisenbug. Here is why observation changed the outcome, and the three techniques that found it anyway.
Long form accounts of single debugging investigations, false leads included. The wrong turns are the part you learn from.
Bug Hunt
The classic heisenbug. Here is why observation changed the outcome, and the three techniques that found it anyway.
Bug Hunt
Every replica restarted at once during a brief database blip. The health check was working exactly as designed, and the design was wrong.
Bug Hunt
p99 jumped on one service. The database was fine, every dependency reported healthy, and the flame graph pointed at nothing. That turned out to be the clue.
Bug Hunt
The dashboard looked right. The numbers were wrong by a few percent every day, and the reason was one implicit conversion in a reporting query.
Shopify
Items vanish from the cart, quantities are wrong, and it never reproduces on your machine. Throttle your network and it appears every time.
Bug Hunt
A weekly export failed every Tuesday and worked every other day. Three wrong hypotheses later, the cause was a date format string nobody had looked at in…
PostgreSQL
Adding a column should be instant. Sometimes it is not, and the difference is worth knowing before you run it in production.