RDRYUSLOG DEV
Back to Blog
C#

Creating a C# Custom Iterator with yield

c# custom iterator: Learn how to build a C# custom iterator using yield return, when it's useful, and how it affects performance and memory.

yield returnIEnumerable<T>state machinelazy evaluationiterator methods
Illustration of a C# custom iterator concept, showing a sequence being generated lazily with a state machine and yield return.

When you write a yield return statement inside a method, the compiler transforms that method into a state machine that implements IEnumerable<T> or IEnumerator<T>. This is the foundation of a C# custom iterator: you don't manually implement MoveNext, Current, or Dispose. You write the sequence logic as straight-line code, and the compiler handles the state transitions for you.

Consider a method that returns even numbers up to a limit:

public static IEnumerable<int> GetEvenNumbers(int max)
{
    for (int i = 0; i <= max; i += 2)
    {
        yield return i;
    }
}

Each call to MoveNext resumes execution after the previous yield return. Local variables like i retain their values across resumptions because the compiler hoists them into fields of a generated state machine class. This lazy behavior is the core reason iterators are efficient for large or infinite sequences: each element is produced on demand.

How the Compiler Generates the State Machine

To understand where the state machine matters, think about what the compiler generates when it sees the yield keyword. The method becomes a nested class that implements both IEnumerable<T> and IEnumerator<T>. The class has fields for the current state, the current value, and any local variables.

The state field tracks which yield point to resume at. When you call GetEnumerator, the compiler returns a new instance of the state machine, or if the iterator is already an enumerator, it may return this. The MoveNext method uses a switch statement on the state to jump to the correct location.

// Conceptual transformation - not actual compiler output
private sealed class <GetEvenNumbers>d__0 : IEnumerable<int>, IEnumerator<int>
{
    private int state;
    private int current;
    private int i;
    private int max;

    public bool MoveNext()
    {
        switch (state)
        {
            case 0:
                i = 0;
                break;
            case 1:
                i += 2;
                break;
            default:
                return false;
        }
        if (i <= max)
        {
            current = i;
            state = 1;
            return true;
        }
        state = -1;
        return false;
    }

    public int Current => current;
}

This is a simplified representation; the real compiler output is more involved, including handling try/finally blocks and thread safety. But the key point is that your yield method is not executed eagerly. The first call to MoveNext runs the code up to the first yield return, then pauses. Subsequent calls continue after that point.

This state machine behavior explains why you cannot use ref or out parameters in an iterator method, and why yield return cannot appear inside a try block that has a catch – the compiler can't resume in a catch handler. These restrictions are a direct consequence of the generated state machine.

Building a Custom Iterator for a Non-Sequential Collection

Sometimes you need to iterate over a collection in a custom order, such as a tree traversal. A C# custom iterator is ideal for depth-first or breadth-first traversal because you can write recursion with yield and get a concise, readable implementation.

public class TreeNode
{
    public int Value { get; set; }
    public List<TreeNode> Children { get; } = new List<TreeNode>();
}

public static IEnumerable<TreeNode> DepthFirst(TreeNode root)
{
    yield return root;
    foreach (var child in root.Children)
    {
        foreach (var node in DepthFirst(child))
        {
            yield return node;
        }
    }
}

Each foreach over DepthFirst(child) creates a new enumerator for that child's state machine. The recursion works naturally because each level of recursion is its own state machine. The caller receives each node as soon as it is produced, so you can stop the traversal early without visiting the entire tree.

For a breadth-first traversal, you could use a queue inside the iterator method, but note that yield return inside a while loop that uses a queue works fine because no try/catch is involved.

Lazy Evaluation and Deferred Execution

The main reason to use a custom iterator is lazy evaluation. The method does not run when you call it; it runs only when you enumerate it. Consider this code:

var numbers = GetEvenNumbers(1000000); // No computation yet
var first = numbers.First(); // Computes only the first element

The First() call only makes the iterator advance once, giving you a significant performance benefit for large sequences. Without an iterator, you would typically build a full list, which allocates memory for all elements even if you only need one.

The downside is that the iterator cannot know its length. Count() on an iterator will enumerate the entire sequence, which is O(n). If you need to know the count multiple times, consider caching the result. Similarly, indexing is not supported directly; you must enumerate to reach an index.

Error Handling in Custom Iterators

Exception handling in iterators has subtle behavior. If the iterator method throws before the first yield return, the exception is thrown when MoveNext is called, not when you call the method. This is important for understanding when resource cleanup happens.

public static IEnumerable<int> SafeSequence()
{
    throw new InvalidOperationException("Cannot start sequence");
    yield return 1;
}

var sequence = SafeSequence(); // No exception here
var enumerator = sequence.GetEnumerator();
var move = enumerator.MoveNext(); // Throws here

This deferred exception behavior can surprise developers who assume controlling code that runs before the first yield executes at the call site. In practice, you should validate arguments in a separate wrapper method if you want exceptions to be thrown immediately.

Performance Implications and Memory Usage

Each iterator invocation allocates a new state machine object. For one-off enumerations this overhead is negligible, but in a hot loop that creates and disposes many enumerators, allocation pressure can matter. The compiler makes a small optimization for foreach loops: if the type implements a public GetEnumerator that returns a struct, it avoids allocating an interface reference. But for a method that returns IEnumerable<T>, each GetEnumerator call allocates a new enumeration object.

If you are writing a custom iterator that is used in performance-critical code, consider these tradeoffs:

  • Iterator methods avoid allocating a full collection, so memory usage is typically lower than a list approach when you only consume part of the sequence.
  • The state machine adds some overhead per element, but it is usually small compared to the cost of building and storing a full collection.
  • If you need to enumerate the same sequence multiple times, each enumeration creates a new state machine, so it may be faster to materialize the sequence into a list once if the sequence is small.

Modern .NET may optimize some iterator patterns by caching the enumerator, but that behavior depends on the runtime version and is not something you should rely on. Write your code to be correct first, then profile if you suspect iterator overhead is a bottleneck.

Advanced Usage: yield break and Early Termination

You can end an iterator prematurely with yield break. This is essential when you need to stop based on a condition that isn't a simple end of collection.

public static IEnumerable<string> ReadUntilEmpty()
{
    while (true)
    {
        string line = Console.ReadLine();
        if (line == "")
        {
            yield break;
        }
        yield return line;
    }
}

yield break is equivalent to reaching the end of the method. It triggers any finally blocks that are in scope, which is important for resource cleanup. Because iterators can hold resources (like an open file), you must ensure you dispose the enumerator correctly. A using statement around the iterator or a foreach loop will handle that automatically.

If you create an iterator that opens a file, you should wrap the yield return logic in a try/finally to ensure the file is closed even if the consumer stops early.

public static IEnumerable<string> ReadLines(string path)
{
    using (var reader = new StreamReader(path))
    {
        string line;
        while ((line = reader.ReadLine()) != null)
        {
            yield return line;
        }
    }
}

The using statement translates to a finally block in the generated state machine, ensuring Dispose is called on the reader when the enumeration ends or is abandoned by the consumer.

Choosing Between an Iterator and a Collection

A custom iterator is the right choice when:

  • The sequence is potentially large or infinite.
  • You want to stream data as it becomes available.
  • You need to define a custom traversal order that is easier to express recursively.
  • You want to delay computation until the consumer asks for the next element.

A collection (like List<T> or an array) is better when:

  • You need random access by index.
  • You need to know the count upfront.
  • The sequence is small and you will enumerate it multiple times.
  • You need to sort or search the data.

In most cases, a foreach loop over an iterator is indistinguishable from a loop over a list. The difference shows up in memory usage and when you stop early. If you only need the first few elements, an iterator avoids the cost of generating the entire collection.

Compatibility and Version Considerations

The yield keyword has been part of the C# language since version 2.0, so any modern .NET project can use it. However, the behavior of iterator methods depends on the runtime and compiler version. For example, newer runtimes may generate more efficient state machines or handle certain edge cases differently.

If you are working with IAsyncEnumerable<T> (introduced for asynchronous streaming), you use await foreach and yield return in an async iterator method. That is a separate but related concept. Async iterators allow you to yield elements that are produced asynchronously, which is useful for streaming data from a network or database. The syntax is similar:

public static async IAsyncEnumerable<int> GetNumbersAsync()
{
    for (int i = 0; i < 10; i++)
    {
        await Task.Delay(100);
        yield return i;
    }
}

When you create an async iterator, the compiler generates a state machine that supports await points inside the method. This is a more advanced feature but builds on the same conceptual foundation.

For most synchronous scenarios, a C# custom iterator with yield return provides a clean, lazy way to define sequences. The state machine compiler transformation is hidden from you, but understanding it helps you reason about when code executes, how exceptions propagate, and why certain restrictions exist. Keep these details in mind as you integrate iterators into your codebase.

c# custom iterator: Practical Usage and Code Examples | RYUSLOG DEV