python functools: partial, lru_cache, wraps, reduce

python functools requires a clear understanding of the core syntax, runtime behavior, and practical implementation patterns demonstrated in the examples below.

The functools module in Python's standard library provides higher-order functions and operations on callable objects. It is not a framework or a flashy feature; it is a collection of utilities that make common functional programming patterns explicit, concise, and less error-prone. When you need to bind arguments, memoize results, preserve decorator metadata, or dispatch on type, functools is the first place to look.

What functools Provides

functools bundles several tools that address recurring problems in Python code. The most frequently used are:

partial() for fixing arguments of a callable
lru_cache() for memoizing function results
wraps() for preserving metadata in decorators
reduce() for folding a sequence into a single value
singledispatch() for type-based function dispatch
cmp_to_key() for adapting comparison functions to key functions

Each tool solves a specific problem. Understanding when to reach for them matters more than memorizing the API.

partial: Binding Arguments Ahead of Time

partial creates a new callable that behaves like the original with some arguments already supplied. It is useful when you have a function that takes several parameters but you always pass the same value for one of them.

from functools import partial

def power(base, exponent):
    return base ** exponent

square = partial(power, exponent=2)
cube = partial(power, exponent=3)

print(square(5))  # 25
print(cube(2))    # 8

The resulting callable still accepts the remaining arguments. You can call it with positional or keyword arguments, but you cannot override an argument that has already been bound unless you pass it explicitly as a keyword at call time. This behavior makes partial ideal for callbacks in event handlers or for customizing library functions without writing a wrapper.

One common mistake is assuming partial freezes the bound argument's value at definition time. It does not. It stores the reference, so if the bound value is mutable and changes later, the callable sees the change. For immutable values like integers or strings, this is not an issue.

lru_cache: Memoization with Automatic Eviction

lru_cache caches function results based on the arguments. It is a decorator that wraps the function and stores the most recent calls in a dictionary, evicting the least recently used entries when the cache reaches a maximum size.

from functools import lru_cache

@lru_cache(maxsize=128)
def fibonacci(n):
    if n < 2:
        return n
    return fibonacci(n - 1) + fibonacci(n - 2)

print(fibonacci(50))  # 12586269025

Without caching, the recursive fibonacci recomputes the same values repeatedly. With lru_cache, each distinct n is computed only once. The maxsize parameter controls how many entries are kept. Setting maxsize=None makes the cache unbounded, which is safe only when the number of distinct arguments is small or memory is not a concern.

The cache keys are based on the positional and keyword arguments. The arguments must be hashable, so lists and dictionaries cannot be used directly. If you need to cache a function that takes a list, convert it to a tuple first.

lru_cache also exposes methods like cache_clear() and cache_info() for manual management and inspection. In long-running processes, a cache that never clears can become a memory leak. Use cache_clear() when the underlying data changes or when you know the cache is no longer needed.

wraps: Preserving Metadata in Decorators

A decorator typically replaces the original function with a wrapper. Without wraps, the wrapper loses the original function's name, docstring, and module. This makes debugging and introspection harder.

from functools import wraps

def my_decorator(func):
    @wraps(func)
    def wrapper(*args, **kwargs):
        print("Before call")
        result = func(*args, **kwargs)
        print("After call")
        return result
    return wrapper

@my_decorator
def greet(name):
    """Return a greeting."""
    return f"Hello, {name}"

print(greet.__name__)  # greet
print(greet.__doc__)   # Return a greeting.

Without @wraps, greet.__name__ would be wrapper, and greet.__doc__ would be None. wraps copies the metadata from the original function onto the wrapper, so tools like help() and debuggers behave correctly. It also updates __wrapped__, which allows inspect.unwrap to traverse the decorator chain.

Always use wraps when writing decorators that wrap functions. It costs nothing and prevents subtle issues in production code where function names are used for logging, serialization, or documentation generation.

reduce: Folding a Collection into a Single Value

reduce applies a binary function cumulatively to the items of a sequence, from left to right, reducing the sequence to a single value. It is part of the functional programming trio along with map and filter, but it is less commonly needed because explicit loops are often clearer.

from functools import reduce

numbers = [1, 2, 3, 4, 5]
total = reduce(lambda x, y: x + y, numbers)
print(total)  # 15

The first argument is a function that takes two arguments. The second is the iterable. An optional third argument provides an initial value. If the iterable is empty, the initial value is returned. Without an initial value and with a single-element iterable, that element is returned directly.

reduce is most useful when the operation is associative and you want to avoid an explicit loop. For example, computing the product of all numbers or finding the maximum. However, Python's built-in sum(), max(), and min() cover many common cases. Use reduce only when the operation is not already available as a built-in.

singledispatch: Function Overloading by Type

singledispatch lets you define a generic function and then register specialized implementations for specific types. The dispatch is based on the type of the first argument.

from functools import singledispatch

@singledispatch
def process(value):
    raise TypeError(f"Unsupported type: {type(value)}")

@process.register(str)
def _(value):
    return f"String: {value.upper()}"

@process.register(int)
def _(value):
    return f"Integer: {value * 2}"

print(process("hello"))  # String: HELLO
print(process(5))        # Integer: 10

The base function is called when no registered implementation matches the argument type. You can register implementations for any type, including custom classes. This pattern is a clean alternative to a chain of isinstance checks.

singledispatch works on the first argument only. If you need dispatch on multiple arguments, consider singledispatchmethod for methods or a custom dispatch mechanism. The dispatch lookup is cached internally, so repeated calls with the same type do not incur a large overhead.

Performance and Memory Considerations

lru_cache is the most performance-sensitive tool in functools. The cache lookup adds a dictionary lookup and a function call overhead. For functions that are cheap to compute, the cache can be slower than recomputing. Use it only when the function is expensive relative to the lookup cost.

Memory usage grows with maxsize. A large cache can consume significant memory, especially if the cached results are large objects. Monitor cache_info() to see hit rates and adjust maxsize accordingly. In a web application, an unbounded cache can cause memory exhaustion. Prefer a bounded cache with a reasonable maxsize.

partial creates a new object but does not duplicate the function. The overhead is minimal. reduce and singledispatch have no special memory implications beyond the normal function call machinery.

Compatibility and Maintenance

functools has been part of Python since version 2.5, but some features have evolved. lru_cache was added in Python 3.2. singledispatch appeared in Python 3.4. Python 3.9 introduced functools.cache, which is an unbounded version of lru_cache with no maxsize argument. If you are on Python 3.9 or later and do not need eviction, cache is simpler and slightly faster.

The wraps function has always updated __wrapped__, but the exact set of copied attributes has grown. Rely on __name__ and __doc__ for basic metadata; do not assume __annotations__ or other attributes are always copied.

When maintaining older codebases, be aware that lru_cache requires hashable arguments. If you encounter a TypeError about unhashable arguments, convert the argument to a tuple or frozenset before calling the cached function. This is a common source of confusion.

Finally, functools is not a replacement for well-designed function signatures. Use partial sparingly; if you find yourself binding many arguments, consider refactoring the function or using a class. The module is a tool for clarity, not a way to hide poor API design.