RDRYUSLOG DEV
Back to Blog
Python

Using send() with Python Generators

python send generator: Learn how to use the send() method to pass values into a running Python generator, with practical examples and performance considerations.

generatorsyieldcoroutinespython-advanceddata-streams
Illustration of a Python generator with a send arrow entering the yield expression, representing bidirectional data flow.

python send generator requires a clear understanding of the core syntax, runtime behavior, and practical implementation patterns demonstrated in the examples below.

When you call send() on a Python generator, you are not just resuming it; you are also delivering a value back into the generator at the point where it last paused. This is the core mechanism behind coroutine-style behavior in Python, and it is often misunderstood because the first call to send() must be None. This article explains how send() works, when it is genuinely useful, and where it adds unnecessary complexity.

The Basic Syntax of send()

A generator that uses send() must have a yield expression that captures the incoming value. The simplest form looks like this:

def receiver():
    while True:
        received = yield
        print(f"Got: {received}")

To drive this generator, you call next() once to start it, then use send() to pass values:

gen = receiver()
next(gen)          # advance to the first yield

gen.send("hello")  # prints "Got: hello"
gen.send(42)        # prints "Got: 42"

The yield without an expression still returns the sent value. The generator pauses at the yield line, and when send() is called, that value becomes the result of the yield expression.

How send() Interacts with yield

When a generator is paused at a yield, it is waiting for the caller to request the next value. Normally, next() does this and passes None implicitly. send(value) does the same thing but also makes value available inside the generator. This is the key difference: send() is a bidirectional communication channel, while next() only sends None.

The value passed to send() is returned by the yield expression. This means you can assign it to a variable and use it in subsequent logic:

def accumulator():
    total = 0
    while True:
        increment = yield total
        total += increment

Here, the generator yields the current total, then waits for the next increment. The caller can send a new value and receive the updated total:

acc = accumulator()
print(next(acc))       # 0, because the first yield returns the initial total
print(acc.send(5))     # 5
print(acc.send(3))     # 8

The first next() is necessary because the generator has not yet reached the yield statement. You cannot call send() with a non-None value before the generator has started.

Passing Data Into a Running Generator

The most common use of send() is to create a generator that acts as a stateful consumer. For example, you can build a simple command processor:

def command_handler():
    state = "idle"
    while True:
        command = yield state
        if command == "start":
            state = "running"
        elif command == "stop":
            state = "stopped"
        elif command == "reset":
            state = "idle"

You can then drive it from a loop:

handler = command_handler()
next(handler)  # initialize

print(handler.send("start"))  # running
print(handler.send("stop"))   # stopped
print(handler.send("reset"))  # idle

This pattern is useful when you want to maintain state between calls without creating a class. The generator's local variables persist across yield statements, giving you a lightweight state machine.

Handling the First send() Call

A common mistake is calling send(value) on a generator that has not been primed. This raises a TypeError because the generator is not yet at a yield expression. The rule is simple: you must call next() or send(None) first.

gen = receiver()
gen.send("hello")  # TypeError: can't send non-None value to a just-started generator

If you want to avoid an explicit next() call, you can use a decorator or a helper function that primes the generator automatically. However, for clarity, most code simply calls next() once.

Common Use Cases for send()

send() is not needed in everyday generator usage, but it shines in specific scenarios:

  • Coroutines: When you need a function that can pause and resume with input, send() provides a simple coroutine mechanism without the asyncio overhead.
  • State machines: As shown earlier, a generator can hold state and transition based on sent commands.
  • Incremental data processing: If you are streaming data and need to inject control signals between chunks, send() lets you do that without breaking the stream.
  • Test doubles: You can create a fake object that returns different values based on the input sent to it.

For example, a generator that filters a stream and also accepts a threshold change:

def threshold_filter():
    threshold = 10
    while True:
        value = yield
        if value > threshold:
            print(f"Pass: {value}")
        else:
            print(f"Block: {value}")

You can send a new threshold by using a special command, but that requires a more complex protocol. In practice, you might use a separate channel or a class for that.

Error Handling and send()

When you call send(), the generator may raise an exception. This exception propagates to the caller, just like with next(). You should handle exceptions if the generator can fail based on the sent value.

def safe_divide():
    while True:
        divisor = yield
        try:
            result = 10 / divisor
        except ZeroDivisionError:
            yield "error"
        else:
            yield result

Here, the generator catches the division error and yields an error indicator. The caller must be aware that send() can raise StopIteration when the generator is exhausted, so you need to handle that if you are not using a for loop.

Another subtlety: if the generator is closed (i.e., it returns), calling send() raises StopIteration. You should catch it if the generator might terminate based on the sent value.

Performance and Memory Considerations

Generators are lazy, so they do not store the entire sequence in memory. Using send() does not change that; the generator still yields one value at a time. The overhead of send() compared to next() is negligible in most applications. The real performance concern is whether you need bidirectional communication at all. If you only need to pass data in one direction, a simple function or a list comprehension is often faster and clearer.

For example, a generator that accumulates values is slower than a plain loop that maintains a local variable, because each send() involves a generator resume and yield overhead. In performance-critical code, measure before assuming that send() is the right tool.

Memory usage remains constant because the generator does not retain the entire history. This makes send() suitable for long-running streams where you need to adjust parameters on the fly.

When Not to Use send()

send() adds complexity. If you find yourself writing a generator that only receives values and never yields meaningful output, a class with a method might be simpler:

class Accumulator:
    def __init__(self):
        self.total = 0
    def add(self, value):
        self.total += value
        return self.total

This is easier to read and test than a generator with send(). Similarly, if you need to pass multiple arguments or handle complex control flow, a coroutine based on asyncio or a state machine class is often more maintainable.

Use send() when:

  • The state is naturally represented by a single loop.
  • You want to avoid class boilerplate.
  • You need to interleave yielding and receiving in a way that is awkward with a class.

If you are already using asyncio, you should prefer async def and await over send() because they are designed for coroutines and are more readable.

Compatibility and Python Versions

The send() method has been part of Python since version 2.5, so it is available in all modern Python 3.x releases. There are no version-specific differences in behavior. However, the yield from syntax (introduced in Python 3.3) can delegate to a subgenerator and also forwards send() calls, which is useful when composing generators. For example:

def outer():
    inner = inner_gen()
    yield from inner

Calling send() on the outer generator will forward the value to the inner generator. This is a powerful composition technique, but it also means that send() behavior can be hidden inside delegation, so you need to be aware of it when debugging.

Advanced Pattern: Bidirectional Streaming

A more advanced use of send() is to create a generator that both receives and yields values, forming a two-way pipeline. For instance, you can build a running average calculator that accepts new numbers and yields the current average:

def running_average():
    count = 0
    total = 0
    average = None
    while True:
        new_value = yield average
        total += new_value
        count += 1
        average = total / count

Driving it:

average_gen = running_average()
next(average_gen)  # get to first yield

print(average_gen.send(10))  # 10.0
print(average_gen.send(20))  # 15.0
print(average_gen.send(30))  # 20.0

This pattern is useful in data processing pipelines where you want to feed data incrementally and read the current state without restarting the computation. The generator retains its internal state, so you do not need to pass the entire history each time.

The key to using send() effectively is to remember that the generator's yield expression is the only place where you can receive data. If you need to send data before the first yield, you must prime the generator with next(). Once you understand that flow, send() becomes a precise tool for building stateful, bidirectional data processors.

python send generator: Practical Usage and Code Examples | RYUSLOG DEV