python redis get set expire ttl and caching: Practical Usage

python redis get set expire ttl and caching requires a clear understanding of the core syntax, runtime behavior, and practical implementation patterns demonstrated in the examples below.

When working with Python and Redis, the get, set, and expire commands form the core of most caching strategies. Understanding how TTL (time-to-live) works is essential for building reliable caches that don't serve stale data or accumulate unused keys. This article covers the practical usage of these commands with the redis-py client, explains how expiration behaves under the hood, and discusses the operational decisions that affect cache performance and consistency.

Setting and Getting Values with redis-py

The redis-py library exposes Redis commands as methods on a client object. The simplest pattern is set followed by get:

import redis

r = redis.Redis(host='localhost', port=6379, decode_responses=True)

r.set('user:123:profile', '{"name": "Alice"}')
value = r.get('user:123:profile')
print(value)  # {"name": "Alice"}

The decode_responses=True parameter makes the client return strings instead of bytes, which is convenient for JSON payloads. Without it, get returns a bytes object that you would decode manually.

set overwrites any existing value at the key. If you need to avoid overwriting when the key already exists, use the nx parameter:

# Only set if the key does not exist
r.set('user:123:profile', '{"name": "Bob"}', nx=True)

For a simple cache, this is rarely needed because you usually want to replace the cached value with fresh data.

Setting Expiration and TTL

Redis supports two ways to attach a TTL to a key: set it at write time with the ex parameter, or apply it later with the expire command.

Setting TTL at Write Time

The set command accepts an ex argument that specifies the lifetime in seconds:

# Cache for 60 seconds
r.set('weather:city:london', '{"temp": 18}', ex=60)

There is also px for milliseconds, and exat/pxat for absolute Unix timestamps. For most caching scenarios, relative ex is the simplest.

Applying Expiration to an Existing Key

If the key already exists and you want to add or change its TTL, use expire:

r.set('session:user:42', 'active')
r.expire('session:user:42', 3600)  # 1 hour

You can also use pexpire for milliseconds. The expire command returns True if the timeout was set and False if the key does not exist.

Checking Remaining TTL

The ttl command returns the remaining time in seconds, and pttl returns milliseconds. A return value of -1 means the key has no expiration, while -2 means the key does not exist.

r.set('temp:key', 'value', ex=120)
remaining = r.ttl('temp:key')
print(remaining)  # approximately 120, then decreases

no_expiry = r.ttl('persistent:key')  # -1 if key exists without TTL
missing = r.ttl('nonexistent:key')   # -2

These checks are useful for debugging cache behavior or for implementing conditional refreshes.

How TTL and Expiration Actually Work

Redis does not immediately delete a key when its TTL reaches zero. Instead, it uses two mechanisms:

Lazy expiration: When a key is accessed via a command like get, Redis checks its TTL and removes it if expired, returning None (or nil).
Active expiration: Redis periodically samples keys with TTLs and removes expired ones in the background, even if they are never accessed.

This means an expired key may still occupy memory for a short time, but it will never be returned to a client. From the application's perspective, an expired key behaves exactly like a missing key.

For example:

r.set('flash:message', 'hello', ex=1)
time.sleep(2)
value = r.get('flash:message')
print(value)  # None

The get returns None because the key was lazily expired. This is the expected behavior for caching: stale data is never served.

Caching Patterns with TTL

The most common pattern is cache-aside (lazy loading). The application checks the cache first, and on a miss, loads from the source of truth and stores the result with a TTL:

def get_user_profile(user_id):
    cache_key = f'user:{user_id}:profile'
    cached = r.get(cache_key)
    if cached is not None:
        return cached

    # Simulate a database query
    profile = fetch_from_database(user_id)
    r.set(cache_key, profile, ex=300)
    return profile
```n
This pattern reduces database load but introduces a window where the cache can be stale for up to the TTL duration. The choice of TTL is a tradeoff between freshness and performance.

For write-heavy workloads, you might invalidate the cache on updates instead of waiting for expiration:

```python
def update_user_profile(user_id, new_data):
    # Update database
    update_database(user_id, new_data)
    # Invalidate cache
    r.delete(f'user:{user_id}:profile')

This ensures the next read repopulates the cache with fresh data. Combining deletion with TTL gives you a safety net if the deletion fails.

Handling Missing Keys and Errors

When get returns None, it means the key is either absent or expired. This is the normal signal for a cache miss. You should not treat it as an error.

However, Redis operations can fail due to network issues, timeouts, or server errors. The redis-py client raises exceptions such as redis.exceptions.ConnectionError or redis.exceptions.TimeoutError. In a caching layer, you often want to degrade gracefully:

try:
    value = r.get(cache_key)
except redis.exceptions.RedisError:
    # Fall back to the source of truth
    value = fetch_from_database(cache_key)

Be careful not to catch too broadly; you may want to distinguish between connection errors and command errors. For a cache, failing open (falling back to the database) is usually preferable to failing closed (returning an error to the user).

Performance and Operational Considerations

Every Redis command is a network round trip. For high-throughput caching, minimize the number of calls. The redis-py client supports pipelining to batch commands:

pipe = r.pipeline()
pipe.set('key1', 'value1', ex=60)
pipe.set('key2', 'value2', ex=60)
pipe.execute()

This sends both commands in a single request, reducing latency. For read-heavy patterns, consider using mget to fetch multiple keys at once.

Connection pooling is another important factor. The default client creates a connection pool, but you should reuse a single client instance across your application rather than creating a new one per request. This avoids the overhead of establishing a new TCP connection each time.

Memory usage is directly affected by TTL. Keys without expiration accumulate until evicted by Redis's maxmemory policy. If you use TTL for all cache keys, expired keys are eventually removed, but active expiration is not immediate. In a large cache, you may see memory usage stay high for a while after keys expire.

Redis eviction policies determine what happens when memory is full. For a pure cache, allkeys-lru is often appropriate because it evicts any key, regardless of TTL. If you mix persistent data with cache data, volatile-lru evicts only keys with an expiration set, protecting non-expiring keys.

Choosing TTL Values and Eviction Policies

The right TTL depends on how quickly your data changes and how tolerant your application is to stale reads. A TTL of a few seconds is suitable for rapidly changing data like stock prices. For user profiles that change rarely, minutes or even hours may be acceptable.

A common mistake is setting a uniform TTL for all keys. Consider the cost of a cache miss: if a miss triggers an expensive database query, you want a longer TTL. If the data is cheap to fetch, a short TTL reduces staleness without much overhead.

Eviction policy interacts with TTL. If you set maxmemory and use noeviction, Redis will return errors on writes when memory is full, which can break your cache writes. For a cache, allkeys-lru is usually the safest choice because it automatically evicts the least recently used keys, even if they have not expired. This prevents write failures at the cost of possibly evicting keys that still have a valid TTL.

When using allkeys-lru, the TTL becomes a secondary mechanism. The primary control is memory pressure. This is fine for a cache where losing a key is acceptable. For data that must persist, use a separate Redis instance or database.

Finally, monitor your cache hit rate. A low hit rate suggests your TTL is too short or your eviction policy is too aggressive. A high hit rate with stale data suggests your TTL is too long. Adjust based on observed behavior rather than guessing.

Understanding how get, set, expire, and TTL work together gives you the tools to build a cache that is both fast and correct. The key is to treat TTL as a first-class design parameter, not an afterthought, and to choose eviction policies that match your data's importance.