Using C# IAsyncEnumerable for Async Streams

C# IAsyncEnumerable<T> is the interface for async streams: sequences of values that are produced asynchronously and consumed one at a time. It was introduced in .NET Core 3.0 and C# 8.0, and it fills a gap that previously forced developers to choose between collecting an entire result set into a list or using synchronous IEnumerable<T> with blocking I/O.

The Problem Async Streams Solve

Before IAsyncEnumerable, when a method needed to return a sequence of values that required asynchronous work to produce, the options were limited. You could return Task<List<T>>, which forces the entire collection to be materialized in memory before the caller sees any element. Or you could return IEnumerable<T> backed by a method that blocks on each async operation, which wastes threads and risks deadlocks in synchronization contexts.

IAsyncEnumerable<T> solves this by representing a sequence where each element can be produced asynchronously. The consumer pulls elements one at a time, and the producer performs async work between elements without blocking a thread.

Producing an Async Stream with yield return

The simplest way to create an IAsyncEnumerable<T> is to write an async iterator method. The syntax combines async methods with iterator blocks:

public async IAsyncEnumerable<int> ReadNumbersAsync()
{
    for (int i = 0; i < 10; i++)
    {
        await Task.Delay(100);
        yield return i;
    }
}

The method signature uses async IAsyncEnumerable<int> and the body uses yield return just like a synchronous iterator. The difference is that the method can contain await expressions before each yield return. The compiler transforms this into a state machine that produces each element only when the consumer requests it.

This is important: the delay happens between elements, not upfront. The first element is produced after the first delay, the second after the second delay, and so on. The caller never sees the entire collection at once.

Consuming an Async Stream with await foreach

To consume an async stream, use await foreach:

await foreach (int number in ReadNumbersAsync())
{
    Console.WriteLine(number);
}

The await foreach statement requests each element from the stream, awaiting the asynchronous production of each one. The loop body runs only after the element is available. This is the async counterpart of foreach.

One detail worth understanding is that await foreach does not buffer elements. The consumer and producer operate in lockstep: the consumer requests element N, the producer computes it, the consumer processes it, then the consumer requests element N+1. This backpressure behavior is what makes async streams suitable for large or infinite sequences.

Cancellation in Async Streams

Async streams support cooperative cancellation through CancellationToken. The consumer can pass a token to the stream, and the producer should observe it:

public async IAsyncEnumerable<int> ReadNumbersAsync(CancellationToken token)
{
    for (int i = 0; i < 10; i++)
    {
        token.ThrowIfCancellationRequested();
        await Task.Delay(100, token);
        yield return i;
    }
}

The consumer passes the token via WithCancellation:

using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(2));

await foreach (int number in ReadNumbersAsync(cts.Token)
    .WithCancellation(cts.Token))
{
    Console.WriteLine(number);
}

The WithCancellation extension method associates the token with the enumeration itself, so cancellation interrupts the await foreach loop even if the producer does not observe the token directly. The producer should still observe the token in its own async operations to avoid wasted work.

Error Handling in Async Streams

Exceptions in async streams follow the same model as async methods. If the producer throws, the exception propagates to the consumer at the point where the element was being produced. This means exceptions can surface at any await foreach iteration, not just at the start or end of the enumeration.

public async IAsyncEnumerable<string> ReadLinesAsync()
{
    using var reader = new StreamReader("data.txt");
    while (await reader.ReadLineAsync() is string line)
    {
        yield return line;
    }
}

If the file does not exist, the exception is thrown when the consumer requests the first element, not when the method is called. The method itself returns immediately; the body executes lazily. This is a common source of confusion: wrapping the method call in a try/catch does not catch the exception unless the enumeration is also inside the try block.

The correct pattern is to wrap the entire await foreach loop:

try
{
    await foreach (string line in ReadLinesAsync())
    {
        Process(line);
    }
}
catch (FileNotFoundException ex)
{
    Logger.LogError(ex, "Input file missing");
}

Performance and Resource Considerations

The main performance benefit of IAsyncEnumerable is memory efficiency. A Task<List<T>> approach materializes the full result set in memory. An async stream produces one element at a time, so memory usage stays constant regardless of how many elements the sequence contains.

There is a cost, though. Each element in an async stream involves an async state machine, and each await in the producer adds overhead compared to a synchronous loop. For sequences with a small, bounded number of elements, a Task<List<T>> may be simpler and faster. Async streams pay off when the sequence is large, unbounded, or when each element requires significant async work such as network or disk I/O.

Another consideration is that async streams do not support random access or re-iteration. An IAsyncEnumerable<T> is single-use in practice: once you enumerate it, you cannot rewind it. If the consumer needs to iterate the sequence twice, the producer must be called again or the results must be buffered.

Choosing Between IAsyncEnumerable and Alternatives

The choice depends on how the data is produced and consumed:

Use IAsyncEnumerable when the producer performs async work per element and the consumer wants to process elements incrementally. Use Task<List<T>> when the result set is small and the consumer needs the complete collection. Use IObservable when the data arrives on its own schedule and the consumer cannot request elements.

Production Considerations for Async Streams

In production code, async streams interact with other infrastructure in ways worth planning for. Database providers such as Entity Framework Core expose IAsyncEnumerable<T> for query results, which lets a query stream rows without loading the entire table into memory. That works well for reporting queries, but the underlying connection stays open for the duration of the enumeration, so the stream should be consumed within a scope that owns the connection.

Logging and telemetry also deserve attention. An async stream that fails halfway through may leave the consumer with a partial result set. If the consumer is a batch job, it should track how many elements were processed before the failure so the job can resume from the correct position.

Finally, be deliberate about where the enumeration happens. An await foreach loop that performs slow work in the loop body holds the producer's resources open. If the producer holds a database connection or a file handle, a slow consumer keeps that resource allocated for longer than a materialized list would.