Python Closure: How Nested Functions Capture State

python closure requires a clear understanding of the core syntax, runtime behavior, and practical implementation patterns demonstrated in the examples below.

In Python, a closure is a function that retains access to variables from its enclosing scope even after that scope has finished executing. This behavior is fundamental to many patterns like decorators and function factories. Understanding how closures capture state is essential for writing correct and maintainable code.

What Makes a Function a Closure in Python

A closure arises when a nested function references a variable from its enclosing function, and the enclosing function returns that nested function. The nested function carries a reference to the variable, not just its value at creation time. This is possible because Python stores the needed variables in a special cell object that persists after the outer function returns.

Consider the simplest example:

def make_multiplier(factor):
    def multiply(x):
        return x * factor
    return multiply

times_two = make_multiplier(2)
print(times_two(5))  # 10

Here, multiply is a closure because it references factor from make_multiplier's scope. When make_multiplier(2) returns, factor is still accessible to multiply. The closure captures the variable factor, not just the value 2. If factor were mutable, changes would be visible across calls.

Not every nested function is a closure. If the nested function does not reference any variables from the outer scope, it is just a regular function. For example:

def outer():
    def inner():
        return "hello"
    return inner

inner does not close over any variables, so it is not a closure. The distinction matters because closures have additional memory and lifecycle implications.

How Closures Capture Variables from the Enclosing Scope

Python determines which variables are captured at compile time. When a nested function references a name that is not local to it, Python treats that name as a free variable. The compiler creates a cell for each free variable, and the closure stores a reference to that cell.

This means the closure sees the current value of the variable at call time, not the value when the closure was created. This behavior is often surprising when combined with loops, as we will see later.

To inspect which variables a closure captures, you can use the __closure__ attribute:

def outer():
    x = 10
    def inner():
        return x
    return inner

fn = outer()
print(fn.__closure__[0].cell_contents)  # 10

Each cell in __closure__ corresponds to a captured variable. The order matches the order in which the variables appear in the nested function's code. This is rarely needed in production code, but it helps debug closure behavior.

Using nonlocal to Modify Captured State

By default, a closure can read captured variables, but it cannot reassign them. Assigning to a variable inside a nested function makes it local to that function unless you declare it nonlocal. The nonlocal statement tells Python that the variable belongs to an enclosing scope.

def counter():
    count = 0
    def increment():
        nonlocal count
        count += 1
        return count
    return increment

c = counter()
print(c())  # 1
print(c())  # 2

Without nonlocal, the count += 1 line would raise an UnboundLocalError because Python treats count as local to increment. The nonlocal keyword makes the closure stateful, which is useful for creating counters, accumulators, or other stateful callables.

Note that nonlocal works only for variables in an enclosing function scope, not global scope. For global variables, you would use global. Closures are typically used with nonlocal when you need to maintain state across calls.

Practical Use Cases: Function Factories and Decorators

Closures are the foundation of several common Python patterns. Two of the most frequent are function factories and decorators.

A function factory returns a new function with some parameters fixed. The make_multiplier example above is a factory. Factories are useful when you need to generate specialized functions without repeating code.

Decorators are a more elaborate use of closures. A decorator takes a function, wraps it with additional behavior, and returns the wrapper. The wrapper often needs to access the original function and its arguments, which it does via closure:

def log_calls(func):
    def wrapper(*args, **kwargs):
        print(f"Calling {func.__name__}")
        return func(*args, **kwargs)
    return wrapper

@log_calls
def add(a, b):
    return a + b

print(add(2, 3))  # prints "Calling add" then 5

Here, wrapper closes over func. The closure retains the original function object, allowing the wrapper to call it later. This pattern is so common that Python provides functools.wraps to copy metadata from the original function to the wrapper, but the underlying closure mechanism remains the same.

The Late Binding Pitfall in Loops

A well-known issue with closures is late binding: closures capture variables by reference, not by value. When you create closures in a loop, they all see the final value of the loop variable, not the value at each iteration.

def create_functions():
    funcs = []
    for i in range(3):
        def f():
            return i
        funcs.append(f)
    return funcs

for f in create_functions():
    print(f())  # 3 3 3

All three functions return 3 because they all capture the same variable i, which ends the loop at 3. To capture the current value, you need to bind it to a default argument or use a factory function:

def create_functions():
    funcs = []
    for i in range(3):
        def f(i=i):  # default argument captures value
            return i
        funcs.append(f)
    return funcs

for f in create_functions():
    print(f())  # 0 1 2

Alternatively, you can use a nested factory that takes i as an argument:

def make_f(i):
    def f():
        return i
    return f

This is a common source of bugs, especially when building callbacks or event handlers in loops. Understanding that closures capture variables, not values, helps you avoid this pitfall.

Memory and Lifecycle Implications of Closures

Closures keep the captured variables alive as long as the closure object exists. This can lead to unexpected memory retention if you create closures that capture large objects or references to objects that should be garbage collected.

For example, if an outer function creates a large temporary object and returns a closure that references it, that object will not be freed until the closure is garbage collected. This is usually fine, but it can become a problem if you store many closures or if the closure is long-lived.

Consider a closure that captures a reference to a large list:

def process_data(data):
    # data is a large list
    def get_sum():
        return sum(data)
    return get_sum

The get_sum closure holds a reference to data. If the caller does not need data after creating the closure, the memory cannot be reclaimed until the closure is dropped. This is not a memory leak per se, but it is a retention detail to keep in mind when designing APIs that return closures.

In contrast, if you only need a single value, you can copy it into a local variable inside the closure to avoid retaining the entire object:

def process_data(data):
    total = sum(data)  # capture only the result
    def get_sum():
        return total
    return get_sum

This reduces the closure's footprint. While closures are generally efficient, being aware of what they capture helps you write more memory-conscious code.

When to Prefer Closures Over Classes or Lambdas

Closures are not always the right tool. They offer a concise way to create callable objects with a small amount of state, but for more complex state, a class with __call__ may be clearer.

The table below summarizes when to choose each approach:

A closure is ideal when you need a small, self-contained function with a few captured values. A class is better when you need multiple methods or when the state is complex enough to benefit from named attributes and methods. Lambdas are restricted to expressions and cannot use nonlocal or return statements, so they are not suitable for stateful closures.

For example, a closure-based counter is compact:

def make_counter():
    count = 0
    def increment():
        nonlocal count
        count += 1
        return count
    return increment

A class-based version is more verbose but offers additional methods like reset:

class Counter:
    def __init__(self):
        self.count = 0
    def increment(self):
        self.count += 1
        return self.count
    def reset(self):
        self.count = 0

Choose the approach that matches the complexity of the state and the number of operations. Closures are not a replacement for classes; they are a lighter alternative for simple cases.

Closures are a core Python feature that enables many elegant patterns. By understanding how they capture variables, how to modify captured state with nonlocal, and where the common pitfalls lie, you can use them effectively without introducing subtle bugs. The key is to remember that closures capture references, not values, and to design your functions with that behavior in mind.