python next generator: Practical Usage and Code Examples

The phrase python next generator describes the core interaction between the built-in next() function and generator objects. A generator is a function that uses yield to produce a sequence of values lazily, and next() is the function that asks it for the next value. Understanding this interaction is essential for writing efficient iterators, streaming data pipelines, and cooperative coroutines.

The Generator Execution Model

When you define a function with yield, calling that function does not execute the body. It returns a generator object. The body runs only when you advance the generator with next() or iterate over it with a for loop.

def count_up_to(limit):
    current = 1
    while current <= limit:
        yield current
        current += 1

gen = count_up_to(3)
print(gen)  # <generator object count_up_to at 0x...>

At this point no print or other code inside count_up_to has run. The generator object stores the function's local state, including the current value of current, and the position of the yield statement.

Calling next() on a Generator

next(gen) executes the generator body until the next yield expression, returns the yielded value, and then suspends execution again. Each call resumes from where the previous call left off.

print(next(gen))  # 1
print(next(gen))  # 2
print(next(gen))  # 3

The first call starts the function, hits yield current with current == 1, returns 1, and pauses. The second call resumes after that yield, increments current to 2, reaches the loop condition, and yields 2. The same pattern continues until the loop ends.

This is different from a regular function that runs to completion. A generator can be paused indefinitely, which makes it useful for representing infinite sequences or streams where you only want to compute values on demand.

What Happens When the Generator Is Exhausted

A generator is exhausted when its body returns without yielding another value. After that, every call to next() raises StopIteration.

gen = count_up_to(2)
print(next(gen))  # 1
print(next(gen))  # 2
try:
    next(gen)
except StopIteration:
    print("Generator exhausted")

The for loop handles this internally. It calls next() repeatedly and stops when StopIteration is raised. If you call next() directly, you need to handle StopIteration yourself unless you know the exact number of values.

The next() function also accepts a default argument. When the generator is exhausted, it returns that default instead of raising an exception.

gen = count_up_to(1)
print(next(gen, None))  # 1
print(next(gen, None))  # None

This is convenient when you want to treat an exhausted generator as an optional value rather than an error.

Passing Values Back with send()

Generators support two-way communication. The send() method resumes the generator and passes a value into it, which becomes the result of the yield expression. Calling next(gen) is equivalent to gen.send(None).

def echo():
    received = yield "ready"
    yield received

gen = echo()
print(next(gen))          # "ready"
print(gen.send("hello"))  # "hello"

The first next(gen) starts the generator and runs to the first yield. The value "ready" is returned, and the generator pauses with the expression received = yield "ready" incomplete. When you call gen.send("hello"), that string becomes the value of the yield expression, so received is set to "hello", and the generator continues to the next yield, returning "hello".

This pattern is the foundation for cooperative coroutines. You can send commands or data into a long-running generator and receive a response at each pause point.

Using next() with yield from

The yield from expression delegates iteration to another iterable or generator. When you use yield from, the outer generator automatically handles all values from the inner generator, and next() on the outer generator drives the inner one.

def inner():
    yield 1
    yield 2

def outer():
    yield "start"
    yield from inner()
    yield "end"

for value in outer():
    print(value)

This prints start, 1, 2, end. The yield from expression forwards next() calls to the inner generator and forwards values back. It also forwards exceptions and send() values, which makes it the preferred way to compose generators.

Performance and Memory Characteristics

Because a generator computes one value at a time, it does not allocate a list or other container for the full sequence. This matters when you process large files, network streams, or infinite sequences. The memory cost is roughly constant, regardless of how many values the generator can produce.

Calling next() has a small per-call overhead because it resumes a suspended frame and updates local state. This overhead is usually acceptable for I/O-bound or streaming workloads. If you need the fastest possible iteration over an already-materialized sequence, a list or tuple may be faster, but it requires storing all values in memory. The right choice depends on whether you can afford to hold the data at once.

Generators are single-use. Once exhausted, they cannot be restarted. If you need to iterate more than once, you must create a new generator object or materialize the values into a reusable container.

Common Pitfalls with next() and Generators

One common mistake is assuming a generator can be reused. After StopIteration is raised, the generator is permanently exhausted. Another mistake is calling next() without handling StopIteration in code that may receive an unexpected number of values. Using the default argument or a for loop avoids this.

Another pitfall is mixing next() and send() incorrectly. You cannot call send() with a non-None value before the generator has started. The first call must be next(gen) or gen.send(None). Attempting gen.send("value") first raises TypeError: can't send non-None value to a just-started generator.

Finally, be careful when a generator yields from a mutable object. The generator holds a reference to that object, and changes made outside the generator will be visible inside it. This is not a bug, but it can lead to surprising behavior if you mutate a list while iterating.

items = [1, 2, 3]
def gen():
    for item in items:
        yield item

g = gen()
items.append(4)
print(next(g))  # 1
print(next(g))  # 2
print(next(g))  # 3
print(next(g))  # 4

The generator reads items lazily, so appending before the generator reaches the end changes the output. If you need a snapshot, create a copy before building the generator.