Python Generator send() Explained with Examples

The Problem send() Solves

A generator in Python produces a sequence of values using yield. The consumer pulls values with next(). That one-way flow is enough for iteration, but some algorithms need to push data back into the generator while it is is paused. The send() method does exactly that: it resumes the generator and passes a value into the yield expression, which becomes the result of that expression inside the generator.

python generator send is a common search phrase because the behavior is subtle: send() both resumes execution and delivers a value, and the first call has a special restriction.

How yield and send() Interact

When a generator reaches yield, it pauses and emits a value. The yield expression itself evaluates to whatever the consumer sends back when the generator is resumed. If the consumer uses next(), the value is None. If the consumer uses send(value), that value is assigned to the expression.

def echo():
    received = yield "ready"
    print(f"received: {received}")
    yield "done"

gen = echo()
print(next(gen))        # "ready"
print(gen.send("hello")) # prints "received: hello", then yields "done"

The first next() starts the generator and runs it to the first yield. At that point the generator is paused, and the yield expression has not yet been assigned. The first call to send() must be send(None) or next(), because there is no yield expression waiting to receive a value before the generator starts.

Passing Data Into a Running Generator

The common pattern is to use send() to feed data into a generator that processes values incrementally. This turns a generator into a simple coroutine.

def accumulator():
    total = 0
    while True:
        value = yield total
        if value is None:
            continue
        total += value

acc = accumulator()
next(acc)  # prime the generator
print(acc.send(10))  # 10
print(acc.send(5))   # 15
print(acc.send(3))   # 18

Here the generator runs an infinite loop, but it only executes up to the yield each time. The consumer sends a value, the generator adds it to the running total, and yields the new total. This pattern is useful for streaming calculations, stateful processing, and pipeline stages.

The Prime Requirement for send()

A generator that uses send() must be "primed" before the first non-None send. Priming means advancing it to the first yield using next(gen) or gen.send(None). If you call gen.send(5) on a fresh generator, Python raises TypeError: can't send non-None value to a just-started generator. This is a common mistake.

gen = accumulator()
gen.send(10)  # TypeError

The reason is that before the first yield, there is no yield expression to receive the value. The generator must execute to the first yield first, and that execution is triggered by next() or send(None).

Using send() for Two-Way Communication

Beyond simple data feeding, send() enables a generator to act as a coroutine that can receive commands and return responses. The generator can inspect the sent value and change its behavior.

def command_handler():
    state = "idle"
    while True:
        command = yield state
        if command == "start":
            state = "running"
        elif command == "stop":
            state = "stopped"
        elif command == "reset":
            state = "idle"

handler = command_handler()
next(handler)
print(handler.send("start"))  # running
print(handler.send("stop"))   # stopped
print(handler.send("reset"))  # idle

This pattern is the basis for Python coroutines before async/await became standard. Even today, send() is useful for state machines, protocol handlers, and incremental parsers where you need to push data in and get results out.

Error Handling with send() and Generator Exceptions

When a generator is paused at a yield, you can also inject exceptions using throw(). This is related to send() because both resume the generator with an input. throw() raises an exception at the point of the yield, which the generator can catch and handle.

def safe_divide():
    try:
        while True:
            dividend = yield
            divisor = yield
            yield dividend / divisor
    except ZeroDivisionError:
        yield "division by zero"

gen = safe_divide()
next(gen)
gen.send(10)
gen.send(2)   # yields 5.0
next(gen)     # prepare for next pair
gen.send(10)
gen.throw(ZeroDivisionError)  # yields "division by zero"

This allows the consumer to signal errors into the generator without breaking the communication channel. The generator can recover and continue processing. The close() method also exists to stop a generator and raise GeneratorExit inside it.

Performance and Memory Considerations

Generators are lazy: they compute values on demand and hold only their local state between yields. Using send() does not change that fundamental property. The generator still suspends and resumes without building a full result set in memory. This makes send() suitable for large or infinite sequences where you want to feed data incrementally.

However, each send() call has a small overhead compared to a plain function call because it involves resuming a frame and evaluating the yield expression. For most applications this overhead is negligible. The real benefit is the ability to maintain state between calls without explicit class attributes or global variables.

If you need to pass data into a generator many times per second, measure whether the generator overhead is acceptable. In performance-critical loops, a simple function that returns a closure might be faster. But for most I/O-bound or stateful processing tasks, send() is clean and maintainable.

When to Use send() Instead of Other Patterns

send() is the right tool when you need two-way communication with a stateful processing unit. If you only need to iterate over a sequence, next() is sufficient. If you need to pass data in and out repeatedly, send() is more direct than using a class with methods.

A class might be clearer for complex state machines with many methods. A generator with send() keeps the logic in one place and avoids self attribute management. For simple state machines, the generator version is often shorter and easier to follow.

If you are building a pipeline where data flows in one direction, send() is not necessary; use regular generator iteration or yield from to compose generators. send() shines when you need to push data back into a paused generator, such as feeding tokens to a parser or commands to a state machine.

The Interaction of send() with yield from

In Python 3, yield from delegates to a subgenerator. When you use send() on a delegating generator, the value is forwarded to the subgenerator, and the result of the yield from expression is the value that the subgenerator returns (via StopIteration). This allows building nested coroutines that can communicate across levels.

def sub():
    received = yield "sub ready"
    yield f"sub got {received}"

def main():
    result = yield from sub()
    yield f"main result: {result}"

gen = main()
print(next(gen))        # "sub ready"
print(gen.send("data")) # "sub got data"
print(next(gen))        # "main result: None" (sub returned None)

This is an advanced pattern that can simplify complex generator hierarchies. However, it adds indirection, and debugging can become harder. Use it when you have a clear composition of coroutines.

A Practical Example: Incremental Parser

To see send() in a realistic scenario, consider a simple tokenizer that receives characters and yields tokens.

def tokenizer():
    token = []
    while True:
        char = yield
        if char is None:
            continue
        if char.isalpha():
            token.append(char)
        else:
            if token:
                yield ''.join(token)
                token = []
        if char == '.':
            yield "END"

tok = tokenizer()
next(tok)
for ch in "hello.world":
    result = tok.send(ch)
    if result:
        print(result)

This generator processes characters one at a time and emits tokens when a delimiter is encountered. The consumer pushes characters via send() and receives tokens as yields. This pattern is memory-efficient for streaming input and demonstrates the power of send() for incremental data processing.

Compatibility and Version Notes

The send() method has been part of Python generators since Python 2.5. The behavior is consistent across Python 3.x. The yield from delegation was added in Python 3.3. If you are supporting older Python versions, avoid yield from but send() itself is safe.

One subtlety: when a generator is closed with close(), a GeneratorExit exception is raised at the suspended yield. If the generator catches GeneratorExit and yields again, Python raises RuntimeError. This is a known limitation. Ensure that your generator does not attempt to yield after being closed.

send() is a low-level tool. For new code that needs coroutine behavior, consider async/await for concurrency. But for synchronous stateful processing, send() remains a valid and readable option.