python cache decorator: Practical Usage and Code Examples

When a Python function is called repeatedly with the same arguments, recomputing the result each time wastes CPU cycles. A python cache decorator solves this by storing previous results and returning them on repeat calls, a technique known as memoization. The standard library provides functools.lru_cache, and you can also write custom decorators when you need more control over expiration or storage. n## What a Cache Decorator Does

A cache decorator wraps a function and adds a layer of result storage. When the wrapped function is called, the decorator computes a key from the arguments, checks whether that key already exists in the cache, and returns the cached value if it does. Otherwise it calls the original function, stores the result under the key, and returns it. This eliminates repeated computation for identical inputs.

The key must uniquely represent the function call. For simple functions with hashable arguments, the arguments themselves form the key. For keyword arguments, you need to include them in a deterministic order. The decorator must also handle edge cases like mutable arguments that cannot be used as dictionary keys.

Using functools.lru_cache

The easiest way to add caching is functools.lru_cache. It is a built-in decorator that uses a least-recently-used eviction policy and is thread-safe. Here is a minimal example:

from functools import lru_cache

@lru_cache(maxsize=128)
def expensive_function(n):
    print(f"Computing {n}...")
    return n * n

print(expensive_function(5))  # Computes and caches
print(expensive_function(5))  # Returns from cache

maxsize limits the number of entries. When the cache exceeds this limit, the least recently used entry is discarded. Setting maxsize=None creates an unbounded cache, which can grow without limit and cause memory pressure. The typed parameter, when True, separates results for different argument types, so f(1) and f(1.0) are treated as distinct calls.

lru_cache also provides a cache_clear() method to empty the cache and a cache_info() method that returns statistics about hits and misses. These are useful for debugging and monitoring.

Writing a Custom Cache Decorator

Sometimes you need behavior that lru_cache does not provide, such as time-based expiration or custom key generation. A simple custom decorator can be written with a dictionary:

import time

def ttl_cache(seconds):
    def decorator(func):
        cache = {}
        def wrapper(*args, **kwargs):
            key = (args, tuple(sorted(kwargs.items())))
            if key in cache:
            entry = cache[key]
            if time.time() - entry["timestamp"] < seconds:
                return entry["value"]
            del cache[key]
            result = func(*args, **kwargs)
            cache[key] = {"value": result, "timestamp": time.time()}
            return result
        return wrapper
    return decorator

This decorator adds a timestamp to each entry and checks whether it is still fresh. It also removes expired entries to prevent the dictionary from growing indefinitely. The key uses args and a sorted tuple of keyword arguments to ensure consistent ordering. Note that this implementation assumes arguments are hashable and does not handle mutable arguments.

Cache Invalidation and Expiration

Caches become stale when the underlying data changes. lru_cache has no built-in expiration; you must call cache_clear() manually when you know the data has changed. For time-based expiration, a custom decorator like the one above is necessary. Another approach is to use a separate cache object that supports explicit invalidation, such as a dictionary that you clear from outside.

When designing a cache, consider the cost of stale data versus the cost of recomputation. If correctness is critical, avoid caching or use short expiration times. If the function is deterministic and the inputs rarely change, a long-lived cache is safe.

Memory Usage and Cache Size Limits

An unbbounded cache can consume all available memory if the function is called with many distinct arguments. lru_cache solves this with maxsize, but custom decorators need their own eviction policy. A simple approach is to store only the most recent N results, or to use a collections.OrderedDict and move entries to the end on access. For large result objects, consider whether caching is worth the memory overhead. Use cache_info() to monitor hit rates and adjust the cache size accordingly.

Thread Safety and Concurrency

functools.lru_cache is thread-safe; it uses a lock internally to protect the cache. A custom decorator that reads and writes a plain dictionary is not safe under concurrent access. Two threads can race to compute the same value, or one thread can read a partially updated entry. To make a custom decorator thread-safe, add a threading.Lock around the cache operations:

import threading

def thread_safe_cache(func):
    cache = {}
    lock = threading.Lock()
    def wrapper(*args, **kwargs):
        key = (args, tuple(sorted(kwargs.items())))
        with lock:
            if key in cache:
                return cache[key]
            result = func(*args, **kwargs)
            cache[key] = result
            return result
    return wrapper

The lock ensures that only one thread computes a missing value at a time. However, it also serializes all calls, which can become a bottleneck if the function is called frequently. For read-heavy workloads, a lock-free approach using functools.lru_cache is simpler and often sufficient.

When to Avoid a Cache Decorator

Not every function benefits from caching. Functions that depend on external state, such as reading from a file or a database, can return stale results if the underlying data changes. Functions with side effects, like sending an email or writing to a log, should not be cached because the side effect would only happen once. Non-deterministic functions that return different results for the same input, such as random.random() or time.time(), are also unsuitable. Finally, if the result object is large and the function is called infrequently, the memory cost of caching may outweigh the performance gain. Use a cache decorator only when the function is deterministic, the inputs are hashable, and the computation is expensive enough to justify the storage overhead.