Python Deadlock: Causes, Detection, and Prevention

A Minimal Python Deadlock Example

A deadlock in Python occurs when two or more threads (or coroutines) block forever, each waiting for a resource the other holds. The simplest reproduction uses two locks acquired in opposite order:

import threading
import time

lock_a = threading.Lock()
lock_b = threading.Lock()

def worker_one():
    with lock_a:
        time.sleep(0.1)
        with lock_b:
            print("worker_one got both locks")

def worker_two():
    with lock_b:
        time.sleep(0.1)
        with lock_a:
            print("worker_two got both locks")

thread1 = threading.Thread(target=worker_one)
thread2 = threading.Thread(target=worker_two)
thread1.start()
thread2.start()
thread1.join()
thread2.join()

When this runs, worker_one acquires lock_a and then waits for lock_b, while worker_two acquires lock_b and waits for lock_a. Neither can proceed, and the program hangs indefinitely. This is the classic deadlock pattern: circular wait with mutual exclusion, hold-and-wait, and no preemption.

How Deadlocks Occur in Python Threading

Deadlocks in Python threading usually arise from a few common mistakes:

• Inconsistent lock ordering – When multiple locks are acquired in different orders across code paths, a circular wait becomes possible. The example above is the simplest case.
• Nested locks without a clear hierarchy – If a function acquires a lock and then calls another function that acquires a second lock, the order must be consistent everywhere.
• Forgetting to release a lock – An exception between acquire() and release() can leave a lock held forever. Using with statements mitigates this, but if the lock is acquired manually, a missing finally block is a direct cause.
• Reentrant lock misuse – threading.Lock is not reentrant. If a thread tries to acquire the same lock again without releasing it, it deadlocks against itself. threading.RLock allows reentrancy, but using RLock when a plain Lock is expected can hide design issues.

These issues are not always obvious because they depend on timing. A program may run correctly for hours until a particular interleaving triggers the deadlock.

Detecting a Deadlock in Python

When a Python program hangs, you need to identify whether it is a deadlock and where. The standard library provides faulthandler, which can dump tracebacks of all threads after a timeout:

import faulthandler
import threading

faulthandler.dump_traceback_later(10, exit=True)
# rest of the program

If the program is still running after 10 seconds, faulthandler prints the stack traces for every thread to stderr, showing exactly which locks each thread is waiting on. This is often enough to spot the circular wait.

Another approach is to use a timeout on acquire():

if not lock_a.acquire(timeout=5):
    print("Could not acquire lock_a, aborting")
    return

This does not prevent deadlocks, but it turns a permanent hang into a recoverable error. The timeout value must be chosen carefully: too short and you may abort a legitimate wait; too long and the program remains unresponsive.

For production systems, a watchdog thread that periodically checks for thread progress can be effective. If a thread has not advanced its last-seen timestamp within a threshold, dump the stacks and alert.

Deadlocks in asyncio: A Different But Related Problem

Asyncio uses cooperative multitasking, so a deadlock occurs when a coroutine awaits an event that never arrives. The most common cause is mixing blocking calls with the event loop. For example, calling time.sleep() inside a coroutine blocks the entire loop, but that alone does not deadlock. A true asyncio deadlock often involves asyncio.Lock:

import asyncio

async def worker(lock, other_lock):
    async with lock:
        await asyncio.sleep(0.1)
        async with other_lock:
            print("got both")

async def main():
    lock_a = asyncio.Lock()
    lock_b = asyncio.Lock()
    await asyncio.gather(
        worker(lock_a, lock_b),
        worker(lock_b, lock_a)
    )

asyncio.run(main())

Here, the two coroutines acquire the locks in opposite order, and because await yields control, they can interleave exactly like threads. The deadlock is the same, but the solution differs: you must never hold an asyncio.Lock while awaiting another lock without a consistent ordering.

Another asyncio-specific trap is awaiting a task that never completes because it is waiting on a lock you still hold. This can happen when you await task inside a async with lock block, and the task tries to acquire the same lock.

Preventing Deadlocks: Lock Ordering and Timeouts

The most reliable prevention is to enforce a global ordering for all locks. If every code path acquires locks in the same order, a circular wait cannot occur. In the example, if both workers acquire lock_a before lock_b, the deadlock disappears:

def worker_one():
    with lock_a:
        with lock_b:
            pass

def worker_two():
    with lock_a:
        with lock_b:
            pass

Document the lock hierarchy and enforce it with code review. For complex systems, a lock ordering graph can help identify violations.

Timeouts provide a safety net. Even with correct ordering, a lock may be held for an unexpectedly long time due to I/O or a bug. Using acquire(timeout=...) allows the thread to back off and retry, or to fail gracefully. In asyncio, asyncio.wait_for can wrap a lock acquisition:

try:
    async with asyncio.timeout(5):
        async with lock:
            pass
except asyncio.TimeoutError:
    print("Lock not acquired in time")

Using Context Managers to Reduce Risk

The with statement guarantees that a lock is released even if an exception occurs. This eliminates the