Python Redis Pub/Sub: Implementation and Patterns

python redis pub sub requires a clear understanding of the core syntax, runtime behavior, and practical implementation patterns demonstrated in the examples below.

Setting Up a Redis Connection for Pub/Sub

The redis-py library provides a straightforward API for Redis pub/sub. Start by creating a Redis client instance and then call pubsub() to get a PubSub object.

import redis

client = redis.Redis(host='localhost', port=6379, decode_responses=True)
pubsub = client.pubsub()

The decode_responses=True parameter makes Redis return strings instead of bytes, which is usually more convenient for JSON payloads or plain text. If you omit it, every message arrives as bytes and you must decode manually.

The PubSub object is separate from the main client. It maintains its own connection to Redis and manages the subscription state. You can have multiple PubSub instances on the same client, but each one uses a dedicated connection.

Publishing Messages

Publishing is done through the main client, not the PubSub object. The publish method takes a channel name and a message:

client.publish('notifications', 'user-42 signed up')

The message can be any string. If you need to send structured data, serialize it to JSON first:

import json

payload = {'user_id': 42, 'action': 'signup'}
client.publish('events', json.dumps(payload))

publish returns the number of subscribers that received the message. A return value of 0 means no subscriber was listening on that channel at that moment. This is useful for debugging, but remember that Redis pub/sub is fire-and-forget: if no subscriber is connected, the message is simply dropped.

Subscribing to Channels

To receive messages, subscribe to a channel and then read from the PubSub object. The simplest way is to use get_message() in a loop:

pubsub.subscribe('notifications')

while True:
    message = pubsub.get_message()
    if message and message['type'] == 'message':
        print(message['data'])

get_message() returns a dictionary with keys like type, channel, data. It is non-blocking: if no message is available, it returns None (or a subscribe confirmation message on the first call). For a blocking loop, use listen() instead:

for message in pubsub.listen():
    if message['type'] == 'message':
        print(message['data'])

listen() yields messages as they arrive and blocks the current thread. This is the pattern most applications use when they need a dedicated subscriber thread.

Handling Message Delivery and Threading

Because listen() blocks, you typically run it in a separate thread or process. The PubSub object is not thread-safe, so you should not call subscribe() from one thread while another thread is iterating over listen(). A common design is to create the subscription in the same thread that consumes messages.

If you need to stop a subscriber, you can call pubsub.close() from another thread. This unblocks listen() and raises an exception inside the loop, so you must handle it:

import threading

def consume():
    pubsub.subscribe('notifications')
    try:
        for message in pubsub.listen():
            if message['type'] == 'message':
                handle(message['data'])
    except redis.exceptions.ConnectionError:
        pass

thread = threading.Thread(target=consume)
thread.start()

Closing the pubsub connection from the main thread will cause listen() to raise a ConnectionError, which you can catch to exit cleanly.

Pattern Subscriptions

Redis also supports pattern-based subscriptions with psubscribe(). This lets you subscribe to multiple channels that match a glob-style pattern:

pubsub.psubscribe('events:*')

Now every message published to events:user-42 or events:order-7 will be delivered. The message['channel'] field will contain the actual channel name, and message['pattern'] will contain the pattern that matched. This is useful for routing messages by topic without maintaining an explicit list of channels.

Pattern subscriptions have a higher overhead than exact subscriptions because Redis has to evaluate the pattern for every publish. Use them when the set of channels is dynamic or too large to enumerate.

Error Handling and Reconnection

Redis pub/sub connections can fail for many reasons: network timeouts, Redis restarts, or dropped connections. The redis-py client does not automatically reconnect a PubSub object. If the connection drops, listen() will raise a ConnectionError.

A robust subscriber should catch the exception and recreate the PubSub object:

while True:
    try:
        pubsub = client.pubsub()
        pubsub.subscribe('notifications')
        for message in pubsub.listen():
            if message['type'] == 'message':
                handle(message['data'])
    except redis.exceptions.ConnectionError:
        time.sleep(1)

This loop reconnects after a failure, but it does not replay messages that were missed during the outage. Redis pub/sub has no persistence or acknowledgment mechanism, so any message published while the subscriber is disconnected is lost. If you need reliable delivery, you must use a different pattern, such as Redis Streams or a message queue.

Performance and Operational Considerations

Pub/sub is designed for low-latency fan-out, not for durable message processing. The main operational tradeoff is that messages are not stored. If your subscriber goes offline, it misses everything published during that period.

The number of subscribers on a channel affects throughput. Redis delivers each message to every subscriber over a separate connection, so a channel with 100 subscribers requires 100 writes. For high fan-out, consider batching or using a different broker.

On the Python side, the GIL can limit throughput if you process messages in a single thread. For CPU-bound processing, offload the work to a worker pool and let the subscriber thread only dispatch messages.

Finally, monitor the Redis server's memory and connection count. Each PubSub object holds a connection open, and a large number of subscribers can exhaust the connection limit. Use connection pooling carefully and close PubSub objects when they are no longer needed.