c# async yield return with IAsyncEnumerable

Developers searching for c# async yield return are usually trying to write a method that both awaits asynchronous operations and produces a sequence of values. The two features appear to be incompatible at first, and in one sense they are: you cannot use yield return inside a method that returns Task<IEnumerable<T>>. The solution is IAsyncEnumerable<T>, which has been available since C# 8.0 and is the correct tool for this exact scenario.

The Problem: async and yield return Don't Mix Directly

A method that uses yield return becomes an iterator method. The compiler rewrites it into a state machine that produces values one at a time as the caller requests them. An async method, on the other hand, returns a Task or Task<T> that represents an operation that completes later. These two transformations conflict.

// This does not compile.
public static async Task<IEnumerable<int>> GetNumbersAsync()
{
    for (int i = 0; i < 10; i++)
    {
        await Task.Delay(100);
        yield return i;
    }
}

The compiler rejects this because yield return cannot appear in a method that has the async modifier. The async modifier requires the return type to be Task, Task<T>, or another awaitable type, while yield return requires the return type to be IEnumerable<T>, IEnumerator<T>, or a non-generic equivalent. No type satisfies both requirements.

The common workaround is to collect values into a List<T> and return Task<List<T>>. That compiles, but it changes the behavior: the caller receives nothing until the entire operation completes.

IAsyncEnumerable<T>: The Bridge Between async and Iterators

C# 8.0 introduced IAsyncEnumerable<T>, which allows a method to be both asynchronous and an iterator. The method keeps the async modifier, uses yield return to produce values, and returns IAsyncEnumerable<T>.

public static async IAsyncEnumerable<int> GetNumbersAsync()
{
    for (int i = 0; i < 10; i++)
    {
        await Task.Delay(100);
        yield return i;
    }
}

The compiler generates a state machine that combines both behaviors. Each time the caller requests the next value, the state machine resumes, runs until the next await or yield return, and then suspends. If it hits an await, the caller receives a ValueTask<bool> from MoveNextAsync() that completes when the awaited operation finishes. If it hits a yield return, the value is available immediately.

The underlying interface is different from IEnumerable<T>. IAsyncEnumerable<T> exposes GetAsyncEnumerator(), which returns an IAsyncEnumerator<T>. That enumerator has a MoveNextAsync() method returning ValueTask<bool>, rather than the synchronous MoveNext() used by IEnumerator<T>.

Consuming an Async Iterator with await foreach

To consume an IAsyncEnumerable<T>, use await foreach:

await foreach (int number in GetNumbersAsync())
{
    Console.WriteLine(number);
}

The await foreach statement calls MoveNextAsync() on the enumerator, awaits the returned ValueTask<bool>, and then evaluates the loop body with the current Current value. This is the async equivalent of the standard foreach loop.

You can also enumerate manually if you need more control:

await using (IAsyncEnumerator<int> enumerator = GetNumbersAsync().GetAsyncEnumerator())
{
    while (await enumerator.MoveNextAsync())
    {
        Console.WriteLine(enumerator.Current);
    }
}

The await using statement disposes the enumerator asynchronously, which matters when the iterator holds resources such as a database connection or a network stream.

How the Async Iterator State Machine Works

The compiler transforms an async iterator into a single state machine that tracks two kinds of suspension points. An await suspends the method until the awaited operation completes. A yield return suspends the method until the caller asks for the next value. Both suspension points are represented as states in the generated state machine.

When the caller invokes MoveNextAsync(), the state machine resumes from its current state and runs until it reaches the next await or yield return. If it reaches an await, it returns a ValueTask<bool> that completes when the awaited operation finishes. If it reaches a yield return, it stores the value in Current and returns a completed ValueTask<bool> with the value true.

This design means the method body does not run eagerly when you call it. Calling GetNumbersAsync() only creates the state machine. The first line of the method body runs on the first call to MoveNextAsync(). This is the same deferred-execution behavior as a synchronous iterator, extended with async suspension.

Cancellation and Async Iterators

Async iterators often run for a long time, especially when they wrap paginated API calls or database queries. Cancellation should be passed through the iterator, not handled only at the call site.

public static async IAsyncEnumerable<int> GetNumbersAsync(
    [EnumeratorCancellation] CancellationToken cancellationToken = default)
{
    for (int i = 0; i < 10; i++)
    {
        cancellationToken.ThrowIfCancellationRequested();
        await Task.Delay(100, cancellationToken);
        yield return i;
    }
}

The [EnumeratorCancellation] attribute marks the parameter that the enumerator's WithCancellation() method will populate. This lets the caller pass a token through the enumeration itself:

await foreach (int number in GetNumbersAsync().WithCancellation(cancellationToken))
{
    Console.WriteLine(number);
}

Without this attribute, a token passed to the method directly is captured when the method is called, but a token passed via WithCancellation() would not reach the iterator body. The attribute bridges that gap.

Streaming vs Buffering: Memory and Latency Behavior

The main reason to use IAsyncEnumerable<T> instead of Task<List<T>> is that results are streamed. The caller can process the first value as soon as it is produced, without waiting for the entire operation to finish.

// Buffered: the caller waits for all results.
public static async Task<IEnumerable<int>> GetBufferedAsync()
{
    var results = new List<int>();
    for (int i = 0; i < 10; i++)
    {
        await Task.Delay(100);
        results.Add(i);
    }
    return results;
}

// Streamed: the caller sees each value as it arrives.
public static async IAsyncEnumerable<int> GetStreamedAsync()
{
    for (int i = 0; i < 10; i++)
    {
        await Task.Delay(100);
        yield return i;
    }
}

The buffered version allocates a List<int> that grows as values are added. The streamed version allocates only the state machine and the current value. For a small fixed set of values, the difference is negligible. For a large or unbounded sequence, streaming avoids holding all results in memory at once.

There is a latency difference too. With the buffered version, the caller sees nothing until all ten delays have elapsed. With the streamed version, the caller sees the first value after the first delay. This matters in scenarios like a UI that displays rows as they load, or a pipeline that processes items as they arrive.

Error Handling in Async Iterators

Exceptions in an async iterator are deferred until the caller enumerates the sequence. The method body does not run when the method is called, so an exception thrown on the first line does not surface at the call site.

public static async IAsyncEnumerable<int> GetNumbersAsync()
{
    throw new InvalidOperationException("Failed to start");
    yield return 1;
}

Calling GetNumbersAsync() succeeds. The exception is thrown when the caller first calls MoveNextAsync(). This is consistent with synchronous iterators, but it can surprise developers who expect validation errors to surface when the method is called.

If you need eager validation, separate the validation from the iterator. Have a public method that validates arguments and returns an iterator, or use a local iterator function:

public static IAsyncEnumerable<int> GetNumbersAsync(int count)
{
    if (count < 0)
    {
        throw new ArgumentOutOfRangeException(nameof(count));
    }

    return GetNumbersCore(count);

    static async IAsyncEnumerable<int> GetNumbersCore(int count)
    {
        for (int i = 0; i < count; i++)
        {
            await Task.Delay(100);
            yield return i;
        }
    }
}

The outer method runs eagerly and validates count. The inner local function is the actual async iterator. This pattern keeps argument validation at the call site while preserving streaming behavior.

When to Choose IAsyncEnumerable<T> Over Task<IEnumerable<T>>

Use IAsyncEnumerable<T> when:

The sequence is large or unbounded, and buffering all results in memory would be wasteful.
The caller can begin processing before the entire sequence is available.
The sequence involves per-item asynchronous work, such as fetching a page of data or reading a stream.
You want to support cancellation during iteration.

Use Task<IEnumerable<T>> or Task<List<T>> when:

The caller needs the complete result set before doing anything with it.
The result set is small and fixed.
The method performs one asynchronous operation and then returns all results.
The consuming code is not written for await foreach and would need to be changed.

A method that loads a configuration file, parses it, and returns a list of settings is a poor fit for an async iterator. A method that pages through a remote API and yields each page as it arrives is a good fit.