RDRYUSLOG DEV
Back to Blog
Python

Python GIL: How It Works and Workarounds

python gil: Understand how the CPython GIL serializes threaded execution, when it matters for I/O-bound and CPU-bound code, and how to design around it.

Global Interpreter LockCPythonconcurrencymultithreadingmultiprocessing
A diagram showing multiple colored threads converging on a single central lock, illustrating the Python Global Interpreter Lock serializing thread execution.

python gil requires a clear understanding of the core syntax, runtime behavior, and practical implementation patterns demonstrated in the examples below.

Python's Global Interpreter Lock, commonly called the GIL, is a mutex that guards the CPython interpreter's internal state. It ensures that only one thread executes Python bytecode at any given moment, even on multi-core hardware. Two threads running Python code cannot execute simultaneously, regardless of how many CPU cores are available.

The GIL is not part of the Python language specification. It is an implementation detail of CPython, the reference implementation. Other implementations such as Jython and IronPython do not use a GIL.

What the Python GIL Actually Does

The GIL serializes execution of Python bytecode. When a thread wants to run Python code, it must first acquire the GIL. Once acquired, the thread runs for a short interval, then releases the lock so another thread can acquire it. The default switch interval in CPython 3.x is 5 milliseconds, controlled by sys.setswitchinterval().

This means that a multithreaded Python program never achieves true parallel execution of Python-level code. The operating system may schedule threads onto different cores, but only one thread holds the GIL at a time. The other threads are blocked waiting for the lock.

The GIL also protects internal interpreter state beyond bytecode execution. The garbage collector's data structures, the interned string cache, and type caches all rely on the GIL for thread safety.

Why CPython Needs the GIL

CPython manages object lifetimes through reference counting. Every Python object carries a reference count, and when that count reaches zero, the object's memory is reclaimed immediately. Incrementing and decrementing reference counts is not atomic. If two threads modify the same object's reference count concurrently, the count can become incorrect, leading to a memory leak or premature deallocation.

One alternative is to protect every object with its own fine-grained lock. That approach adds overhead to every attribute access, function call, and container operation. The GIL is a simpler design: one lock for the entire interpreter. It trades parallelism for implementation simplicity and predictable memory safety.

The GIL also simplifies the C API. Many C extensions rely on the GIL to protect their own global state, because they assume only one thread executes Python code at a time. Removing the GIL requires those extensions to adopt their own locking.

How the GIL Affects Threaded Code

Consider a CPU-bound task such as computing SHA-256 hashes in a loop. If you split the work across threads, the GIL serializes execution. Each thread acquires the GIL, runs for a switch interval, then releases it so another thread can run. The total CPU time is the same as running the loop sequentially, plus the overhead of thread switching.

import threading
import hashlib

def hash_loop():
    for _ in range(500_000):
        hashlib.sha256(b"data").hexdigest()

threads = [threading.Thread(target=hash_loop) for _ in range(4)]
for t in threads:
    t.start()
for t in threads:
    t.join()

This code does not run faster than a single-threaded version on a multi-core machine. In practice it can run slightly slower, because the interpreter periodically releases and reacquires the GIL, and the scheduler may migrate threads between cores, incurring cache misses.

The GIL is released during certain blocking operations. When a thread performs I/O, such as reading from a socket or waiting on a network response, it releases the GIL so other threads can run Python code. This is why I/O-bound applications can benefit from threads despite the GIL.

import threading
import urllib.request

def fetch(url):
    with urllib.request.urlopen(url) as response:
        return response.read()

urls = ["https://example.com"] * 10
threads = [threading.Thread(target=fetch, args=(u,)) for u in urls]
for t in threads:
    t.start()
for t in threads:
    t.join()

While one thread waits for the network response, the GIL is free, and another thread can make progress. For I/O-bound workloads, threading in CPython is a legitimate concurrency strategy.

The GIL is also released by many C extension functions that perform long-running native computations. Libraries such as NumPy and compression modules explicitly release the GIL while they work, allowing other threads to execute Python code during that window.

When the GIL Is Not the Bottleneck

For I/O-bound programs, the GIL rarely limits throughput. The dominant cost is waiting on external resources, not executing Python bytecode. Threads spend most of their time blocked on sockets, files, or database connections, and during those waits the GIL is released.

For CPU-bound programs, the GIL is the primary constraint on parallelism. A single-threaded loop that performs heavy arithmetic or hashing will not speed up when split across threads. The only way to use multiple cores for Python-level computation is to bypass the GIL entirely.

There is a middle ground: C extensions that release the GIL around native computation. If the heavy work happens inside a C library that releases the GIL, threads can run that native code in parallel. This is how some scientific and data-processing libraries achieve multicore scaling.

Working Around the GIL

When CPU-bound work must run in parallel, the standard approach is the multiprocessing module. Each process gets its own interpreter and its own GIL, so processes can run on separate cores without contending for a single lock.

from multiprocessing import Pool

def hash_loop(_):
    import hashlib
    for _ in range(500_000):
        hashlib.sha256(b"data").hexdigest()

if __name__ == "__main__":
    with Pool(4) as pool:
        pool.map(hash_loop, range(4))

The tradeoff is that processes have higher startup cost and communicate through inter-process channels rather than shared memory. For CPU-bound tasks where the data can be split into independent chunks, multiprocessing is the standard solution.

For I/O-bound code, asyncio is often a better fit than threads. It uses cooperative multitasking on a single thread, so the GIL is irrelevant. A single-threaded event loop can handle thousands of concurrent connections as long as the work is I/O-bound and does not block the loop.

C extensions can also release the GIL while performing long-running native computations. If you write a C extension using the Python C API, you can wrap the computation with Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS. This is how libraries like NumPy achieve parallelism for certain operations.

The GIL in Other Python Implementations

The GIL is specific to CPython. Other implementations handle concurrency differently:

  • PyPy: Uses a GIL, though it has explored software transactional memory approaches.
  • Jython: Runs on the JVM and does not have a GIL. Threads can run in true parallel.
  • IronPython: Runs on the .NET runtime and does not have a GIL.

For most developers, CPython is the implementation in use, so the GIL is a practical constraint. The distinction matters when evaluating deployment environments or porting code to another runtime.

Free-Threaded Python and the Future of the GIL

Python 3.13 introduced an experimental free-threaded build, sometimes called "nogil," which removes the GIL entirely. In this build, the interpreter uses per-object locking and other techniques to make the memory manager thread-safe without a single global lock.

The free-threaded build is experimental and not the default. It requires a separate installation and may not be compatible with all C extensions, since many extensions rely on the GIL for thread safety. Extensions that do not release the GIL or that assume single-threaded access may need changes.

For most production code, the GIL remains a reality. The practical guidance is unchanged: use threads for I/O-bound work, use processes for CPU-bound work, and use asyncio when a single-threaded event loop fits the workload. When evaluating a dependency for a threaded application, check whether it releases the GIL during heavy native operations; if it does not, threading will not provide the expected speedup.

python gil: Practical Usage and Code Examples | RYUSLOG DEV