Tokio Timed Out. Why Is the Blocking Task Still Running?
A Tokio timeout can end your wait while a blocking worker keeps running. Trace task ownership, cancellation limits, and the permit that must stay with the work.
The request times out. The CPU doesn't. If your Rust service starts blocking work and then wraps its task handle in tokio::time::timeout, those two observations can agree perfectly. The timer has stopped waiting. Nobody has told the worker to leave.
A Tokio timeout around a JoinHandle limits the wait for that task; dropping the handle detaches the task. For ordinary async tasks, retain the handle, request cancellation with abort(), and observe completion. For a spawn_blocking closure that has already started, abort() cannot stop it. Give blocking work a cooperative stop mechanism, use the underlying operation's own cancellation/deadline API, or isolate work behind a process boundary when you need an enforceable termination boundary.
This is an AI-prepared, documentation-based explainer, not a personally reproduced incident. The service scenario, code and diagnostic walkthrough are illustrative and have not been executed. See Debugly's editorial policy. The API contracts below were checked against Tokio's current documentation on 5 November 2026.
The clue is the thing you're timing
Imagine an endpoint that generates a report using a synchronous library. You correctly move that work off the async executor with spawn_blocking. You also give the request a deadline. Sensible decisions, both of them. The trouble is the assumption connecting them.
// Illustrative fragment: generate_report is application-specific.
let job = tokio::task::spawn_blocking(move || generate_report(input));
let outcome = tokio::time::timeout(budget, job).await;
// Err(Elapsed) does not establish that generate_report stopped.
Read the second argument of timeout carefully. It isn't the report generator. It's a handle that waits for the generator's result. Tokio's JoinHandle documentation defines dropping that handle as detaching its task: the task can continue, and you lose its return value.
That distinction gives us a concrete investigation. Record a job identifier at submission, at the first instruction inside the blocking closure, when the caller's wait expires, and when the closure exits. A late completion with the same identifier is stronger evidence than a busy CPU graph. CPU could belong to unrelated requests; a correlated lifecycle tells you which work outlived which waiter.
Keep the actual output separate from the HTTP response too. A caller receiving a timeout doesn't prove that a report wasn't written, an external request wasn't accepted, or a database operation didn't finish. Before retrying work with side effects, establish its state and use the operation's deduplication mechanism. The timeout is a statement about what the caller observed.
Three boundaries that look identical in a dashboard
| What you wrap | What expiring the wait establishes | What to inspect next |
|---|---|---|
| An operation future owned directly by the timeout | The wrapper returns a timeout error and drops the inner future. | The operation's cancellation safety and any effects already sent outside the process. |
A handle from tokio::spawn | Your wait for the task ended; dropping its handle detaches it. | Handle ownership, explicit cancellation, and observed task completion. |
A handle from spawn_blocking | Your wait ended; an already-running closure can continue. | The blocking library's deadline/stop API or cooperative checkpoints. |
The first row follows Tokio's timeout contract. It has another important qualification: a future that doesn't yield can exceed the duration and still complete without a timeout error. An async timer isn't a thread preemption mechanism. If the timeout itself appears late, inspect blocking or long computation inside async polling before blaming the clock.
These are two different bugs to distinguish: the timer didn't get a chance to run, and the timer ran but the work continued independently. Increasing the timeout makes both harder to notice. It is a very effective way to improve a dashboard's mood.
For an async task, keep something you can cancel
For a normally yielding async task, pass a mutable reference to its handle into the timeout. That leaves ownership with you when the wait expires:
// Illustrative Tokio fragment; async_work is application-specific.
let mut job = tokio::spawn(async_work());
match tokio::time::timeout(budget, &mut job).await {
Ok(join_result) => {
// Handle JoinError separately from the operation's own Result.
consume_join_result(join_result);
}
Err(_) => {
job.abort();
let completion = job.await;
record_completion_after_deadline(completion);
}
}
The task cancellation documentation explains why that last await matters: abort() requests cancellation; it doesn't wait for it. The task may also finish normally before cancellation takes effect. Record the result rather than assuming every deadline produces a cancelled task.
Don't copy this into a request path and promise that cleanup is bounded by the original budget. Awaiting termination takes additional time, and non-yielding work defeats prompt cooperative cancellation. If the response must return immediately, give a separate supervisor ownership of cleanup and its outcome. Also audit what happens if the request future itself is dropped before this match completes; this local timeout branch doesn't handle every outer cancellation path.
For blocking work, abort is the wrong lever
Tokio's spawn_blocking contract is explicit: abort cannot stop a closure once it has started. Before it starts, abort may prevent execution. Don't turn that possibility into a guarantee about a queued job racing to start.
If you control the algorithm, split it into bounded chunks and check a stop signal between them. For example, this intentionally incomplete sketch shows where the signal belongs:
// Illustrative sketch, not a complete cancellation framework.
use std::sync::{Arc, atomic::{AtomicBool, Ordering}};
let stop = Arc::new(AtomicBool::new(false));
let worker_stop = Arc::clone(&stop);
let mut job = tokio::task::spawn_blocking(move || {
for chunk in chunks {
if worker_stop.load(Ordering::Relaxed) {
return None;
}
process_bounded_chunk(chunk);
}
Some(finish_report())
});
if tokio::time::timeout(budget, &mut job).await.is_err() {
stop.store(true, Ordering::Relaxed);
// Retain job in an application supervisor and observe its exit.
// Do not report that this store has already stopped the worker.
}
Here the atomic carries only a stop flag, not associated data. More importantly, the flag is useless while a chunk is stuck inside an uninterruptible library call. Your stop latency depends on reaching a checkpoint and exiting safely. If the library offers a real deadline, configure that too. If it offers neither cancellation nor bounded calls, no arrangement of handles around it creates those properties.
For work requiring stronger isolation, a managed subprocess can provide a termination boundary, but you still need an explicit kill-and-wait policy, platform-aware child cleanup, and a plan for partial output. Killing local work cannot roll back an effect already committed by another service.
Keep capacity attached to the worker
A subtler consequence appears when the request owns a concurrency permit. The request times out, drops its permit, and admits another request. Meanwhile the first blocking worker is still active. The limiter is now counting waiters rather than the resource-consuming work.
Acquire a shared semaphore permit before submitting work and move the owned permit into the closure, keeping it alive until that closure exits. Tokio's owned semaphore permits support transfer across task boundaries. This is a resource-ownership choice: cancellation of a waiter must not falsely declare capacity free.
Bound admission as well. Waiting forever to acquire a permit before starting the timeout leaves an unbounded queue outside your budget. Use an overall deadline that includes admission, define overload rejection, and keep the queue bounded. Don't release capacity merely because a client stopped listening.
Verify the exit, not just the error
In a controlled environment, distinguish queued, started, stop-requested and finished states. Exercise a deadline before start, a deadline during a bounded chunk, a library call that doesn't consult the stop flag, and normal completion racing the deadline. These are proposed checks, not reported test results.
For each case, inspect both the response and the worker lifecycle. Confirm when permits are released and whether partial output can be retried safely. A test asserting only Elapsed can pass while the expensive mistake continues behind it.
If you're tracing the same ownership problem across languages, compare Python gather and TaskGroup failure behavior. For the wider service boundary, reviewing timeouts and cancellation provides useful context. This article's specific contract is Tokio task ownership; cancellation rules don't transfer automatically between runtimes.
On the next timeout review, ask for two pieces of evidence: when the caller stopped waiting, and when the work actually ended. Put both in the trace. Otherwise your deadline may be doing exactly what you asked, while the worker does exactly what you forgot to ask.