Python Async Yield: How Async Generators Work

When a function body contains both async def and yield, Python does not treat it as a coroutine. The function becomes an async generator function, and calling it returns an async generator object. That object cannot be awaited and cannot be iterated with a plain for loop. It is driven with async for, and it may contain await expressions between yields. This combination is what python async yield refers to in practice, and understanding what it produces is the first step to using it correctly.

What an async def with yield Actually Creates

Python has three distinct function forms that look similar but behave very differently:

The presence of yield is what switches an async def from coroutine to async generator. This is not a stylistic choice: the two forms have incompatible protocols. A coroutine returns a single value when awaited. An async generator produces a sequence of values over time, and each value may be preceded by asynchronous work.

A minimal async generator:

async def countdown(n: int):
    while n > 0:
        yield n
        n -= 1

Calling countdown(3) returns an async generator object immediately. No code in the body runs until iteration begins.

Iterating with async for

The async for loop drives the async generator. Each iteration resumes the generator until the next yield, runs the loop body, then suspends again.

import asyncio

async def ticker():
    for i in range(1, 4):
        await asyncio.sleep(0.1)
        yield i

async def main():
    async for value in ticker():
        print(value)

asyncio.run(main())

Between each yield, the generator can perform await operations, so the values are produced lazily as the consumer requests them. The consumer controls the pace: the generator does not run ahead and buffer values. This pull-based behavior is the defining characteristic of an async generator.

The Async Generator Protocol

Beyond __anext__, which async for uses internally, async generators expose three async methods that mirror the generator protocol:

• asend(value) resumes the generator and makes yield evaluate to value.
• athrow(exc) raises exc at the suspended yield point.
• aclose() raises GeneratorExit at the suspended yield point and releases the generator.

async def receiver():
    received = yield "ready"
    yield f"got: {received}"

async def main():
    gen = receiver()
    await gen.asend(None)          # start the generator
    result = await gen.asend("ping")
    print(result)                  # got: ping
    await gen.aclose()

The first asend(None) is required to advance the generator to its first yield, matching how send(None) starts a regular generator. athrow is useful when you want the generator's own exception handling to observe an error from the consumer side.

Where await and yield Can Appear

An async generator may contain await before or after yield, and the placement changes when code executes relative to the consumer. In the following examples, get_data() stands in for any awaitable operation.

async def fetch_first():
    data = await get_data()   # runs before the first value is delivered
    yield data

async def fetch_later():
    yield "immediate"          # delivered before any await runs
    data = await get_data()
    yield data

In fetch_first, the consumer waits for the asynchronous operation before receiving anything. In fetch_later, the first value is delivered immediately, and the await happens only when the consumer requests the second value. Choosing between these shapes controls latency and resource usage: deferring awaits keeps the first value cheap, while awaiting up front lets you validate or transform data before exposing it.

Common Failure Modes

The most frequent errors come from treating an async generator as one of the other two forms.

Using a plain for loop raises a TypeError because an async generator is not iterable in the synchronous sense:

for value in ticker():   # TypeError: 'async_generator' object is not iterable
    print(value)

Awaiting the generator directly also fails, because the object is not a coroutine:

await ticker()   # TypeError: object async_generator can't be used in 'await' expression

A return statement with a value inside an async generator attaches that value to StopAsyncIteration. This is rarely what you want; if you need a final result, collect values in the consumer or use a separate coroutine.

A subtler issue is failing to close the generator. If you hold an async generator and stop consuming it, Python emits a RuntimeWarning when the object is garbage collected, and any finally block in the generator never runs. The async for loop closes the generator automatically when it exits, but manual iteration with __anext__ or asend does not.

Async Generators vs. asyncio.Queue

Both async generators and asyncio.Queue can stream values between asynchronous producers and consumers, but they have opposite control flow.

An async generator is pull-based: the consumer requests each value, and the producer code runs only as needed. This is ideal when the producer is cheap to run incrementally, such as reading from a file or an HTTP stream, and when you want backpressure to propagate naturally to the producer.

An asyncio.Queue is push-based: a producer task puts values into the queue, and a consumer task gets them. This decouples the two sides and allows the producer to run ahead, buffering values. Use a queue when the producer and consumer are independent tasks, when you need bounded buffering, or when multiple consumers must share the same stream.

For a one-to-one streaming pipeline where the consumer controls the pace, an async generator is simpler and avoids the overhead of queue management.

Resource Cleanup and aclose()

Because an async generator can hold files, connections, or locks across awaits, cleanup matters. The generator's finally block runs when aclose() is called, when the async for loop finishes or is broken out of, or when an exception propagates through the generator.

async def guarded_stream(lock: asyncio.Lock):
    await lock.acquire()
    try:
        for i in range(10):
            yield i
    finally:
        lock.release()

If you abandon an async generator without closing it, the finally block does not run, and the warning from the garbage collector is the only signal. In long-running services, this leaks resources silently. The reliable pattern is to let async for manage the lifecycle, or to wrap manual iteration in a try/finally that calls aclose() explicitly.