python semaphore: Practical Usage and Code Examples

A Python semaphore is a synchronization primitive that limits how many threads or coroutines can enter a critical section at the same time. The threading.Semaphore class implements this for threads, and asyncio.Semaphore provides the equivalent for async code.

The core idea is a counter. When a thread calls acquire(), the counter is decremented. If the counter reaches zero, subsequent acquire() calls block until another thread calls release(), which increments the counter. This makes semaphores the right tool when you need to cap concurrent access to a resource such as a database connection pool, an external API, or a limited number of worker slots.

Creating and Using threading.Semaphore

The constructor takes an optional initial value. The default is 1, which behaves like a lock, but the common use case is passing a larger number.

import threading
import time

semaphore = threading.Semaphore(3)

def worker(worker_id):
    semaphore.acquire()
    try:
        print(f"Worker {worker_id} entered")
        time.sleep(1)
    finally:
        semaphore.release()

threads = [threading.Thread(target=worker, args=(i,)) for i in range(5)]
for t in threads:
    t.start()
for t in threads:
    t.join()

With an initial value of 3, at most three threads run the protected section at once. The other two block until one of the first three calls release().

The try/finally pattern is essential. If the code inside the critical section raises an exception, release() still runs, so the semaphore counter is not permanently reduced. Forgetting this is the most common way a semaphore-based program deadlocks.

How acquire() and release() Behave

acquire() blocks indefinitely by default. You can pass a timeout argument to limit how long the thread waits:

if semaphore.acquire(timeout=2):
    try:
        # protected work
        pass
    finally:
        semaphore.release()
else:
    print("Timed out waiting for a slot")

When acquire(timeout=...) returns False, the semaphore was not acquired, so you must not call release(). This is a common source of bugs: checking the return value and then releasing unconditionally.

release() increments the counter. If the counter was already at its initial value, threading.Semaphore allows it to grow beyond that value. That is where BoundedSemaphore differs.

BoundedSemaphore vs Semaphore

threading.BoundedSemaphore raises ValueError if release() is called more times than acquire(), which means the counter can never exceed its initial value. This catches a double-release bug immediately.

from threading import BoundedSemaphore

sem = BoundedSemaphore(2)
sem.acquire()
sem.release()
sem.release()  # ValueError: Semaphore released too many times

For most production code, BoundedSemaphore is the safer choice. A plain Semaphore silently allows the counter to drift upward, which weakens the limit you were trying to enforce. The extra safety costs nothing at runtime.

asyncio.Semaphore for Async Code

Async code has the same need for limiting concurrency, but threading.Semaphore cannot be used directly inside a coroutine because it blocks the event loop. asyncio.Semaphore provides an awaitable acquire() and a non-blocking release().

import asyncio

async def fetch(semaphore, url):
    async with semaphore:
        # limited concurrent HTTP requests
        await asyncio.sleep(1)

async def main():
    semaphore = asyncio.Semaphore(5)
    await asyncio.gather(*(fetch(semaphore, f"url-{i}") for i in range(20)))

asyncio.run(main())

The async with statement calls acquire() on entry and release() on exit, including when the body raises an exception. This is the cleanest way to use an async semaphore and avoids the manual try/finally boilerplate.

One important difference: asyncio.Semaphore does not have a timeout parameter on acquire(). To add a timeout, wrap the acquisition in asyncio.wait_for:

try:
    await asyncio.wait_for(semaphore.acquire(), timeout=2)
except asyncio.TimeoutError:
    print("Could not acquire semaphore in time")
else:
    try:
        await work()
    finally:
        semaphore.release()

The else clause runs only when acquisition succeeded, so release() is never called without a matching acquire().

Common Pitfalls

Several mistakes recur in semaphore usage:

Calling release() without a prior successful acquire(), which inflates the counter and lets more threads in than intended.
Using acquire() without try/finally, so an exception leaves the counter decremented permanently.
Using a threading.Semaphore inside an async coroutine, which blocks the event loop.
Assuming the semaphore guarantees fairness. Python's threading.Semaphore does not guarantee FIFO ordering; a thread that has been waiting longer is not necessarily the next one to acquire.

Performance and Operational Considerations

A semaphore is a cheap primitive. The acquire/release cycle involves a lock and a counter update, so the overhead is in the microsecond range. The real cost appears when threads block: a blocked thread consumes a kernel-level lock wait, and with many threads contending, the scheduler spends time waking and re-suspending them.

For I/O-bound work, asyncio.Semaphore is usually the better fit because it avoids creating a thread per task. For CPU-bound work, a semaphore does not help with the GIL; you would need multiprocessing instead.

When the resource being limited is a connection pool, consider whether the pool itself already provides a semaphore-like limit. Many database drivers expose their own pool size configuration, and adding a separate semaphore on top can create confusing double-limiting behavior.

Choosing Between Semaphore, Lock, and Queue

A Lock allows only one thread at a time. A semaphore with value 1 is equivalent, but using a semaphore for mutual exclusion is misleading. Use Lock when the goal is exclusive access.

A Queue with a maximum size is an alternative when the work items themselves are the limiting factor. The queue blocks producers when full and blocks consumers when empty, which is a different pattern from a semaphore that guards a shared resource.

Use a semaphore when you have a fixed number of identical resource slots and multiple threads or coroutines need to claim one temporarily. Use a queue when the work distribution itself needs buffering or ordering guarantees.