C# Iterator: How yield return Works

When you write a method that returns IEnumerable<int> and use yield return, the C# compiler generates a state machine that executes your method lazily. This is the core of the C# iterator pattern. Iterators let you produce sequences on demand without materializing the entire collection in memory. In this article, you'll see how iterators work, how to implement them correctly, and where they can cause subtle performance problems.

What Makes a Method an Iterator

A method becomes an iterator when it contains a yield return or yield break statement and its return type is IEnumerable, IEnumerable<T>, IEnumerator, or IEnumerator<T>. The compiler detects the yield keyword and rewrites the method body into a private class that implements the appropriate interface. You don't write that class yourself; the compiler does it for you.

public IEnumerable<int> GetNumbers()
{
    yield return 1;
    yield return 2;
    yield return 3;
}

This method returns an IEnumerable<int> that, when enumerated, produces 1, 2, and 3. The method body does not execute until someone calls GetEnumerator() and then calls MoveNext() on the resulting enumerator. That's the deferred execution behavior that defines iterators.

How the Compiler Transforms yield return

The compiler turns the iterator method into a state machine. Each yield return becomes a state in that machine. When MoveNext() is called, the machine runs from the last suspended point until the next yield return or the end of the method. Local variables are preserved across calls because they are hoisted into fields of the generated class.

Consider this method:

public IEnumerable<int> CountTo(int limit)
{
    for (int i = 1; i <= limit; i++)
    {
        yield return i;
    }
}

The compiler generates a nested class that holds i, limit, and the current state index. Each call to MoveNext() executes the loop body until it hits yield return i, saves the state, and returns true. The next call resumes the loop with the saved i. This is why you can't use ref or out parameters in iterator methods—they can't be stored in the generated class.

Deferred Execution and Materialization

Deferred execution means that the code inside the iterator doesn't run until you start enumerating. This is powerful, but it also means that if the iterator captures external state, that state is read at enumeration time, not at method call time. For example:

var items = new List<int> { 1, 2, 3 };
IEnumerable<int> iterator = items.Where(x => x > 1);
items.Add(4);
foreach (var item in iterator)
{
    Console.WriteLine(item);
}

The output is 2, 3, 4 because the iterator reads the list when foreach starts, not when Where is called. This is a common source of bugs when the underlying collection changes between the iterator creation and enumeration. If you need a snapshot, call ToList() or ToArray() to materialize the sequence immediately.

Using Iterators with foreach and LINQ

Iterators are the backbone of LINQ. Methods like Where, Select, and Take return iterators that compose with each other. Each LINQ method that returns IEnumerable<T> uses the same deferred execution model. This allows you to build complex queries without intermediate allocations.

IEnumerable<string> names = GetNames();
var shortNames = names.Where(n => n.Length < 5).Select(n => n.ToUpper());
foreach (var name in shortNames)
{
    Console.WriteLine(name);
}

Each foreach iteration calls MoveNext() on the composed iterator chain. The Where iterator pulls from the source, filters, and yields to Select, which transforms and yields to the loop. No intermediate list is created. The cost is a set of nested state machines, each with its own MoveNext call.

Exception Handling Inside Iterators

Exception handling in iterators is tricky because the code runs in chunks. A try block that contains yield return is allowed, but the catch block cannot contain a yield return. The compiler enforces this because the state machine cannot represent a yield inside a catch without losing the exception context. Also, a finally block always runs when the enumerator is disposed, which is important for resource cleanup.

public IEnumerable<int> ReadLines(string path)
{
    using var reader = new StreamReader(path);
    string? line;
    while ((line = reader.ReadLine()) != null)
    {
        yield return int.Parse(line);
    }
}

Here, the using statement generates a finally that calls Dispose() on the reader. The compiler ensures this finally runs when the enumerator is disposed, which happens when foreach completes or when the caller calls Dispose() on the enumerator. If you manually enumerate with MoveNext() and stop early, you must dispose the enumerator to trigger cleanup.

Performance Considerations for Iterators

Iterators are not free. Each MoveNext() call has overhead compared to a simple for loop. The generated state machine allocates an object on the heap, and each iteration involves a virtual call to MoveNext(). For most applications, this overhead is negligible. But in hot paths that process millions of items, a hand-written IEnumerator<T> or a direct List<T> enumeration can be measurably faster.

Another cost is the allocation of the iterator object itself. If you create an iterator method that is called frequently, you may see increased garbage collection pressure. In .NET, you can mitigate this by returning a cached iterator when the sequence is empty or constant. For example:

public IEnumerable<int> GetEmpty()
{
    return Enumerable.Empty<int>();
}

Avoid using iterators to wrap a collection that is already materialized. If you have a List<int> and you return it directly, you avoid the state machine entirely. Only use an iterator when you need lazy evaluation or when the sequence is generated on the fly.

Common Iterator Mistakes and How to Avoid Them

One common mistake is assuming that the iterator method executes immediately. Code before the first yield return does not run until the first MoveNext() call. This can cause unexpected behavior if you validate arguments at the top of the method:

public IEnumerable<int> GetNumbers(int count)
{
    if (count < 0) throw new ArgumentOutOfRangeException(nameof(count));
    for (int i = 0; i < count; i++) yield return i;
}

The exception is not thrown when GetNumbers is called; it is thrown when the caller starts enumerating. If you want eager validation, split the method into two: a normal method that validates and returns the iterator, and a private iterator method that does the work.

Another mistake is using yield return inside a try block with a catch that tries to yield a fallback value. That is not allowed by the compiler. Instead, you can move the try inside a helper method or use a different control flow.

When to Use a Custom IEnumerator Instead

Sometimes you need more control than a compiler-generated iterator provides. For example, if you are implementing a custom collection that must track its position without allocating a new state machine each time, you can write a struct that implements IEnumerator<T>. This is common in high-performance libraries where allocation avoidance matters.

public struct ListEnumerator : IEnumerator<int>
{
    private readonly List<int> _list;
    private int _index;
    private int _current;

    public ListEnumerator(List<int> list) { _list = list; _index = 0; _current = 0; }

    public int Current => _current;
    object IEnumerator.Current => Current;

    public bool MoveNext()
    {
        if (_index >= _list.Count) return false;
        _current = _list[_index];
        _index++;
        return true;
    }

    public void Reset() => _index = 0;
    public void Dispose() { }
}

A struct enumerator avoids heap allocation when used with a foreach loop, because the compiler can avoid boxing if the collection exposes a strongly typed GetEnumerator method. The tradeoff is more manual code and the risk of subtle bugs. For most code, the compiler-generated iterator is the right choice. Reserve custom enumerators for performance-critical scenarios where profiling shows a real bottleneck.