c# await foreach: Consuming IAsyncEnumerable Streams

c# await foreach requires a clear understanding of the core syntax, runtime behavior, and practical implementation patterns demonstrated in the examples below.

C# 8 introduced await foreach to consume asynchronous streams. Before this feature, working with a sequence of data that arrived asynchronously required buffering the entire collection or writing custom pull-based loops. await foreach works with IAsyncEnumerable<T> and gives you a natural, familiar syntax that mirrors the synchronous foreach while preserving the non-blocking behavior of async code.

What `await foreach` Solves

A synchronous foreach blocks the current thread while the sequence is produced. If the data comes from a network call, a database query, or a background service, that blocking wait wastes a thread and can stall the entire application. await foreach lets you iterate over a sequence where each element is produced asynchronously. The loop awaits the next element without occupying a thread while waiting.

Consider a service that returns a list of records from a remote API. With a synchronous List<T>, you wait for the entire response before you can process the first record. With an IAsyncEnumerable<T>, you can process each record as soon as it arrives, reducing latency and memory pressure.

Consuming an IAsyncEnumerable

The simplest usage of await foreach looks like this:

await foreach (var item in GetItemsAsync())
{
    Console.WriteLine(item);
}

The GetItemsAsync method returns IAsyncEnumerable<string>. The compiler transforms the loop into a state machine that calls MoveNextAsync() on the enumerator and awaits the result. Each iteration resumes on the captured synchronization context, just like await in a normal async method.

You can use await foreach inside an async method. The method must be marked async and return Task, Task<T>, or another awaitable type. You cannot use await foreach in a synchronous method.

Writing an IAsyncEnumerable with yield return

Producing an asynchronous stream is straightforward. Use yield return in an iterator method that returns IAsyncEnumerable<T>. The method can also use await before yielding a value, which is the key difference from a synchronous iterator.

async IAsyncEnumerable<int> GenerateNumbersAsync()
{
    for (int i = 0; i < 10; i++)
    {
        await Task.Delay(100); // Simulate async work
        yield return i;
    }
}

Each time the consumer calls MoveNextAsync, the iterator resumes after the previous yield return. The await Task.Delay runs asynchronously, so the thread is free while waiting. The compiler generates a state machine that handles both the async operation and the iterator state.

You can combine await with yield return in any order. For example, you might fetch a page of results from an API, yield each item, then fetch the next page. This pattern is common when paginating through large datasets.

Cancellation and Timeouts

Asynchronous streams often represent long-running operations. You should support cancellation so the consumer can stop the iteration early. The standard approach is to pass a CancellationToken to the producer method and check it inside the iterator.

async IAsyncEnumerable<int> GenerateNumbersAsync(CancellationToken cancellationToken)
{
    for (int i = 0; i < 100; i++)
    {
        cancellationToken.ThrowIfCancellationRequested();
        await Task.Delay(100, cancellationToken);
        yield return i;
    }
}

The consumer can then use CancellationTokenSource and cancel when needed. The await foreach loop itself does not accept a cancellation token directly; you must handle cancellation inside the producer. However, you can break out of the loop manually, and the iterator's DisposeAsync will be called, which allows the producer to clean up resources.

If the producer uses Task.Delay with the token, cancellation will throw OperationCanceledException inside the iterator. You can catch that exception in the producer and exit gracefully, or let it propagate to the consumer. The consumer can catch OperationCanceledException around the await foreach loop to handle cancellation.

Error Handling in Asynchronous Streams

Errors can occur at any point during the iteration. The producer might throw an exception when fetching the next element, or the consumer might throw while processing an element. The behavior is similar to a synchronous iterator: an exception thrown inside the iterator propagates to the consumer at the MoveNextAsync call.

async IAsyncEnumerable<string> ReadLinesAsync()
{
    using var reader = new StreamReader("data.txt");
    string? line;
    while ((line = await reader.ReadLineAsync()) != null)
    {
        yield return line;
    }
}

If ReadLineAsync throws, the exception surfaces at the await foreach in the consumer. You can wrap the loop in a try-catch to handle it. Because the iterator is a state machine, the finally blocks inside the iterator run when the consumer disposes the enumerator, even if an exception occurs.

One important detail: if the consumer breaks out of the loop early, the enumerator's DisposeAsync is called. The iterator's finally blocks execute asynchronously. This allows you to release resources like file handles or database connections without leaking them.

Performance Considerations

await foreach avoids buffering the entire sequence, which reduces memory usage when working with large datasets. It also avoids blocking threads, which improves scalability in server applications. However, there is overhead compared to a synchronous foreach because each iteration involves an asynchronous state machine and potentially a context switch.

For most I/O-bound scenarios, the overhead is negligible compared to the cost of the underlying operation. But if you are iterating over a hot loop with millions of elements and the producer is purely CPU-bound, a synchronous List<T> or an array will be faster. Use IAsyncEnumerable when the data source is genuinely asynchronous, such as a network stream, a database cursor, or a message queue.

Another consideration is the synchronization context. In UI applications, each await resumes on the UI thread, which can cause re-entrancy issues if the producer does heavy work. In ASP.NET Core, there is no synchronization context, so continuations run on thread pool threads. Be aware of these differences when designing your producer.

Compatibility and Runtime Requirements

await foreach requires C# 8 or later and a runtime that supports IAsyncEnumerable<T>. The interface is part of .NET Standard 2.1, so it is available in .NET Core 3.0 and later, .NET 5 and later, and .NET Framework 4.8 when using a compatibility shim. If you are targeting older frameworks, you can use the Microsoft.Bcl.AsyncInterfaces NuGet package to get the necessary types.

The compiler also requires the System.Threading.Tasks.Extensions package for older targets. In modern .NET, these types are included in the base library. When targeting .NET Framework, you may need to install the package explicitly.

If you are using a library that returns IAsyncEnumerable<T>, you can consume it with await foreach regardless of the library's target framework, as long as your project is compatible. Conversely, if you write a method that returns IAsyncEnumerable<T>, consumers on older runtimes may not be able to use await foreach unless they also upgrade or use the compatibility package.

Advanced Pattern: Combining Multiple Streams

You can use await foreach in combination with other async features. For example, you might want to merge two asynchronous streams into one. The simplest approach is to iterate over them sequentially, but that loses the benefit of parallel production. A more advanced pattern uses Task.WhenAll with a channel to merge streams concurrently. This is not directly supported by the language, but you can build a helper method that reads from multiple IAsyncEnumerable sources and writes to a Channel<T>. The consumer then uses await foreach to read from the channel.

async IAsyncEnumerable<T> MergeAsync<T>(IAsyncEnumerable<T> first, IAsyncEnumerable<T> second)
{
    var channel = Channel.CreateUnbounded<T>();
    var writer = channel.Writer;
    var tasks = new[] { WriteAllAsync(first, writer), WriteAllAsync(second, writer) };
    _ = Task.WhenAll(tasks).ContinueWith(_ => writer.TryComplete());

    await foreach (var item in channel.Reader.ReadAllAsync())
    {
        yield return item;
    }
}

static async Task WriteAllAsync<T>(IAsyncEnumerable<T> source, ChannelWriter<T> writer)
{
    await foreach (var item in source)
    {
        await writer.WriteAsync(item);
    }
}

This pattern preserves the streaming behavior while allowing both producers to run concurrently. The channel buffers items until the consumer is ready, which adds a small memory overhead but avoids blocking the producers. This is a useful technique when you need to combine data from multiple sources without waiting for all of them to complete.

What await foreach Solves