Every Timeout Should Be Shorter Than the One Above It

The client gave up after five seconds. The service it called would have answered in nine, and spent all nine doing it, because its downstream timeout was thirty. For four of those seconds the service was computing a result for a client that no longer existed, burning CPU and holding a connection in tribute to a deadline that had already passed upstream.

A timeout is not a number. It is a relationship between the layers of a request chain, and the relationship is the thing that is almost never reviewed. Individual timeouts get chosen with care. The ladder they form gets no attention at all, and the ladder is where the waste and the confusion live.

This is the checklist I run on any diff that touches timeouts, retries or cancellation, written from too many incidents where the chain disagreed with itself.

The invariant: deadlines must shrink downward

The rule is one sentence. Every layer's deadline must be shorter than the deadline of the layer that called it, by a margin that covers the network between them. If the client waits five seconds, the service must give up on its downstream before five, so that it can still answer the client in time, or at least stop working the moment the client stops caring.

When the invariant is violated, the lower layer outlives the upper one, and the work it completes is delivered to nobody. That is the CPU burn from the opening paragraph, and it scales with traffic, so a violated ladder is a capacity leak.

The review question is therefore not "is this timeout reasonable" but "what is the timeout above this, and is this shorter". A thirty second downstream timeout inside a service whose own SLA is five seconds is wrong regardless of what the downstream does.

Cancellation must propagate, and usually does not

A timeout that fires should cancel the work below it, and cancellation is the half that is missing in most codebases. A context or deadline that is created but never checked is a decoration. The review wants to see the deadline threaded through every hop, database query, HTTP call and queue publish, and checked at the loops that could run long.

The specific smells: an HTTP client constructed without the request's context, a database call using a background context inside a request handler, a retry loop that does not consult the deadline between attempts. Each one is a point where the chain forgets its own deadline, and each one converts a cancelled request into continued work.

In languages with explicit contexts, the lint that bans background contexts inside request scope catches most of it. In others, the review is the only net.

The retry multiplier on the ladder

Retries multiply the ladder's errors. If the client allows five seconds and the service retries a two second downstream call three times, the worst case is six seconds, already over the client's deadline, so the third retry is guaranteed waste. The review must compute the worst case of attempts times per attempt timeout plus backoff, and compare it to the deadline above.

This is the arithmetic from reviewing retry and backoff logic, and it belongs in the timeout review because the two are one budget. A retry policy and a timeout policy that were each reviewed separately will very often sum to more than the layer above allows.

Deadlines that never fire are also a defect

The opposite failure is a timeout so large it is a fiction. A thirty second connect timeout on a datacentre network will never fire before something else dies, so it provides no protection, only a long hang when the impossible happens. Timeouts should be set from observed behaviour plus margin, so that firing is rare but possible, and so that when it fires it fires before the layer above gives up.

The review question for a large number is "what observed behaviour justifies this", and if the answer is "we wanted to be safe", the number is unmoored and should be replaced with a measurement.

Timeouts on the work that outlives the request

Not all work should die with the request. A job that must complete, a payment that must not half apply, belongs off the request ladder entirely, on a queue with its own deadline and its own retries. The review should flag any long or consequential operation inside a request scoped context, because tying it to the client's patience makes the client's browser tab a load bearing part of your payment path.

The split is the same one as moving reports out of the request path in 504 gateway timeout while your application was still working.

The shape of a good review comment

Not "timeout too high". Instead: "this downstream call allows thirty seconds, but the enclosing handler's client timeout is five, so the last twenty five seconds of any slow response are computed for nobody. Set the downstream deadline to four, thread the request context into the client, and the retry budget to two attempts so worst case stays under five."

That comment names the ladder, the waste and the fix, and it is actionable because it treats the timeouts as one system.

The rule I keep

Timeouts are a budget that must shrink at every lower layer, cancellation is the mechanism that refunds the budget when a layer gives up, and retries multiply the budget's worst case. Review the chain as one number line from the client down, and reject any rung that is longer than the one above it, because the difference is work performed for an audience that has already left.

The orphaned work this prevents is the disconnect family in EPIPE broken pipe, which is what the lower layer experiences when the upper one finally enforces its deadline.

Published on debugly.dev by Rohit Bhadani | Founder of Krova Cloud