Debugging asyncio Deadlocks and the Blocking Call Checklist

The short answer

An async application that is slow, unresponsive, or hung almost always has synchronous code blocking the event loop. One blocking call stops every coroutine on that loop, not just the one that made it.

Find it:

import asyncio
loop = asyncio.get_running_loop()
loop.set_debug(True)
loop.slow_callback_duration = 0.1   # warn on anything over 100ms

Executing <Task ... coro=<handler() at app.py:42>> took 2.417 seconds

That log line names the coroutine and the file. It is usually the whole diagnosis.

Tested on Python 3.13.2.

Why one blocking call stops everything

An event loop is a single thread running a loop: take the next ready callback, run it to completion, repeat.

"To completion" is the important part. There is no preemption. A coroutine yields control only at an await that actually suspends. If your code calls something synchronous that takes two seconds, the loop runs that for two seconds and every other task waits, including the ones that were ready.

So the symptom is not a slow endpoint. It is every endpoint being slow whenever any request hits the blocking path, which makes it look like a general capacity problem rather than a specific bug.

The tell in metrics: CPU pinned at roughly one core regardless of load, throughput that does not improve with concurrency, and latency that rises for requests that should be fast.

The blocking call checklist

Things that look fine in an async function and are not.

requests and urllib. Synchronous HTTP. Use httpx or aiohttp.

# blocks
r = requests.get(url)

# does not
async with httpx.AsyncClient() as client:
    r = await client.get(url)

time.sleep. Use await asyncio.sleep. This one is usually obvious and occasionally hides in a retry helper somebody imported.

File I/O. open(), read(), write() are all blocking. For small config files at startup nobody cares. For per request file access it matters. Use aiofiles, or asyncio.to_thread.

Database drivers. psycopg2 is synchronous. psycopg version 3 has async support, asyncpg is async native. SQLAlchemy needs the async engine and an async driver, and using the sync engine inside async code is a common mistake because it works right up until load.

subprocess.run. Use asyncio.create_subprocess_exec.

CPU bound work. JSON parsing a 50MB document, image resizing, cryptography, compression, regex over a large string. None of these have an async version because the problem is not I/O. They need a process pool.

Logging to a slow handler. A logging handler writing to a network destination synchronously blocks the loop on every log line. Use a QueueHandler.

DNS resolution in some paths. socket.getaddrinfo is blocking. asyncio's default resolver runs it in a thread pool, which is fine, but a library calling it directly is not.

Finding the blocker

Debug mode first

asyncio.run(main(), debug=True)

Or PYTHONASYNCIODEBUG=1. It logs any callback taking longer than slow_callback_duration, defaulting to 100ms, and also warns about coroutines that were never awaited.

Do not run this in production, since it adds meaningful overhead. It is the fastest path to an answer in development.

Dump the tasks when it hangs

For a hang rather than slowness, you want to know what every task is doing.

import asyncio, signal, sys, traceback

def dump_tasks(*_):
    for task in asyncio.all_tasks():
        print(f"--- {task.get_name()} done={task.done()}", file=sys.stderr)
        task.print_stack(file=sys.stderr)

# in an async context
loop = asyncio.get_running_loop()
loop.add_signal_handler(signal.SIGUSR1, dump_tasks)

Then kill -USR1 <pid> and read the stacks. Same technique as triggering a heap snapshot on a signal in Node, and equally useful: no restart, no port exposed, works in production.

A task stuck in _run_once or showing a stack inside synchronous library code is your blocker.

py-spy for the running process

py-spy dump --pid 1234
py-spy top --pid 1234

py-spy attaches without modifying the process and shows you where time is being spent. For an async application, py-spy dump shows the current stack of every thread, and if the main thread is inside psycopg2 or requests, you have found it in about five seconds.

In a container you need SYS_PTRACE, the same as strace.

Fixing it

Offload to a thread

For blocking I/O you cannot replace:

result = await asyncio.to_thread(blocking_io_function, arg)

asyncio.to_thread runs it in the default executor and awaits the result. The loop stays free.

This works because the GIL is released during I/O. It does not help CPU bound work, where the thread holds the GIL and blocks the loop anyway.

Use a process pool for CPU work

from concurrent.futures import ProcessPoolExecutor

pool = ProcessPoolExecutor(max_workers=4)

async def handle():
    loop = asyncio.get_running_loop()
    return await loop.run_in_executor(pool, cpu_heavy_function, data)

Separate processes, separate GILs. The cost is pickling arguments and results, so this is only worth it when the work is substantially larger than the serialisation.

Create the pool once at startup, not per request. Creating a process pool is expensive and doing it in a handler is a bug I have seen more than once.

Yield inside long loops

If you have a genuinely long running pure Python loop that you cannot move:

for i, item in enumerate(large_list):
    process(item)
    if i % 1000 == 0:
        await asyncio.sleep(0)   # yield to the loop

await asyncio.sleep(0) yields without delay. It is a pressure valve rather than a fix, and it keeps the service responsive while the work proceeds.

Actual deadlocks

Distinct from blocking, and less common.

Awaiting your own task.

async def process():
    task = asyncio.current_task()
    await task          # waits for itself, forever

Lock ordering. Two coroutines acquiring two asyncio.Lock objects in opposite orders deadlock exactly like threads do. The fix is the same as ordering database locks: acquire in a consistent order everywhere.

An unbounded queue with no consumer, or a bounded queue where the consumer is waiting on something the producer holds.

Calling asyncio.run inside a running loop. Raises rather than deadlocking in modern Python, and in older code you find loop.run_until_complete nested inside a coroutine, which does hang.

For all of these, the task dump is the tool. A task blocked on Lock.acquire with another task holding it tells you the story immediately.

Prevention

Ban the sync libraries with a lint rule. A flake8 or ruff rule forbidding import requests in your async package catches the most common cause at review time. Same for time.sleep.

Set slow_callback_duration in development and treat the warnings as failures. Most blocking calls are introduced by someone who did not realise the library was synchronous, and a warning at the moment they add it is far cheaper than finding it in production.

Load test with concurrency. A blocking call is invisible at one request at a time. Ten concurrent requests reveals it immediately, because throughput does not scale.

Watch event loop lag as a metric. Schedule a callback every 100ms and measure how late it actually runs. Lag above a few tens of milliseconds means something is blocking, and this is the single best production signal for this class of problem:

async def monitor_lag():
    while True:
        start = time.monotonic()
        await asyncio.sleep(0.1)
        lag = (time.monotonic() - start - 0.1) * 1000
        if lag > 50:
            logger.warning("event loop lag %.0fms", lag)

That is ten lines and it is the difference between knowing and guessing. It belongs in every async service, in the same category as the other metrics worth having by default.