Python Generator Delegation with yield from

python generator delegation requires a clear understanding of the core syntax, runtime behavior, and practical implementation patterns demonstrated in the examples below.

When a generator needs to yield values produced by another generator, the obvious approach is a nested loop:

def read_lines(paths):
    for path in paths:
        for line in open(path):
            yield line

This works, but it only forwards the yielded values. Calls to send(), throw(), and close() made by the caller stop at the outer generator and never reach the inner one. Python's yield from expression exists specifically to solve this. It delegates the entire iteration protocol to a subgenerator, which is what python generator delegation refers to in practice.

The Problem: Yielding From a Nested Generator

A generator that consumes another generator often looks like this:

def outer():
    for value in inner():
        yield value

The loop is correct for simple iteration, but it is not full delegation. When a caller uses .send(value), the value is delivered to the yield expression inside outer, not to the yield inside inner. The same applies to .throw() and .close(). For a plain data pipeline this rarely matters, but for generators that act as coroutines, the distinction is critical.

yield from replaces the loop and delegates the whole protocol:

def outer():
    yield from inner()

The caller now talks directly to inner. Values sent by the caller arrive at the yield inside inner, exceptions thrown by the caller are raised inside inner, and closing the outer generator closes inner as well.

How yield from Delegates to a Subgenerator

yield from accepts any iterable, not only generators. The delegating generator suspends at the yield from expression and lets the subgenerator drive the conversation.

The return value of the subgenerator is delivered to the delegating generator when the subgenerator finishes:

def inner():
    yield 1
    yield 2
    return "complete"

def outer():
    result = yield from inner()
    print(f"inner returned: {result}")

A generator's return statement does not produce a yielded value; it sets the value carried by StopIteration. yield from captures that value and binds it to result. A plain for loop has no way to access this value.

Bidirectional Communication Through the Delegation Chain

The most important difference between yield from and a manual loop is that the full generator protocol is forwarded.

def accumulator():
    total = 0
    while True:
        value = yield total
        if value is not None:
            total += value

def delegator():
    yield from accumulator()

gen = delegator()
next(gen)        # starts the accumulator, returns 0
gen.send(5)      # delivered directly to accumulator
gen.send(7)

Without delegation, send() would deliver values to the yield inside delegator, and the accumulator would never see them. With yield from, the delegator becomes a transparent pass-through. The same forwarding applies to throw() and close(), which means cleanup logic inside the subgenerator's finally block runs when the caller closes the outer generator.

Practical Use: Composing Generator Pipelines

Generator delegation is the natural way to compose lazy pipelines where each stage is itself a generator.

def numbers():
    for i in range(10):
        yield i

def double(source):
    for value in source:
        yield value * 2

def even(source):
    for value in source:
        if value % 2 == 0:
            yield value

def pipeline():
    yield from even(double(numbers()))

Each stage stays independent and testable. The pipeline remains lazy: nothing is materialized until the caller iterates. If a stage needs to emit values from several sources in sequence, delegation keeps the code flat:

def combined():
    yield from first_source()
    yield from second_source()

This is clearer than nesting loops and preserves the order of the sources.

Runtime and Memory Considerations

yield from keeps the pipeline lazy, so memory usage stays proportional to the current item rather than the full sequence. That is the main operational benefit for large inputs.

The delegation also reduces the number of Python-level frames involved per value. With a manual loop, every value passes through the delegating generator's frame; with yield from, the interpreter routes values directly between the caller and the subgenerator, removing one layer of Python-level work per item. The exact difference depends on the interpreter version and the shape of the pipeline, but the mechanism itself is what matters for understanding the cost.

Error Handling and Edge Cases

Exceptions raised inside the subgenerator propagate to the delegating generator. If the delegating generator has a try/except around the yield from, it can handle failures from the subgenerator:

def risky():
    yield 1
    raise ValueError("bad value")

def wrapper():
    try:
        yield from risky()
    except ValueError as exc:
        yield f"recovered: {exc}"

When the caller uses .throw(), the exception is raised at the point where the subgenerator is suspended. If the subgenerator does not handle it, the exception propagates up through the yield from expression.

One edge case worth knowing: yield from on a non-generator iterable simply iterates it. The delegation protocol only applies when the target is a generator. For a list or tuple, yield from behaves like a for loop, and there is no send or throw forwarding to speak of.

When Not to Use Generator Delegation

Delegation is not always the right tool. If the outer generator must transform each value before yielding it, a loop is the clearer choice:

def with_prefix(prefix, source):
    for value in source:
        yield f"{prefix}{value}"

If the outer generator needs to interleave values from multiple subgenerators, yield from cannot express that; you need explicit loops or itertools.chain. And if the subgenerator must be closed explicitly before the outer generator continues, a try/finally around the delegation is required, because yield from only forwards close() when the outer generator itself is closed.