C# Iterator Deferred Execution

c# iterator deferred execution requires a clear understanding of the core syntax, runtime behavior, and practical implementation patterns demonstrated in the examples below.

When you write an iterator method using yield return, the method body does not run when you call it. Instead, it returns an IEnumerable<T> that executes only when you start enumerating it. This behavior is called deferred execution, and it has significant consequences for performance, error handling, and when side effects occur.

What Deferred Execution Means for Iterator Methods

When you define a method that uses yield return, calling that method does not execute the method body. Instead, it returns an IEnumerable<T> object that represents the sequence. The code inside the iterator runs only when something starts enumerating the sequence, typically with foreach or by calling GetEnumerator() and MoveNext(). This is the core of c# iterator deferred execution.

Consider this example:

public static IEnumerable<int> GetNumbers()
{
    Console.WriteLine("Iterator started");
    yield return 1;
    Console.WriteLine("After first yield");
    yield return 2;
    Console.WriteLine("After second yield");
}

If you call var numbers = GetNumbers();, nothing is printed. The Console.WriteLine calls execute only when you iterate:

foreach (var n in numbers)
{
    Console.WriteLine($"Got {n}");
}

The output order shows that the iterator resumes after each yield return. This deferred behavior is by design and is the foundation of lazy sequences in C#.

How the Compiler Turns Iterators into State Machines

The C# compiler transforms an iterator method into a class that implements IEnumerable<T> and IEnumerator<T>. The local variables in the iterator become fields of that class, and the code between yield statements becomes a state machine with a MoveNext() method. Each call to MoveNext() executes the next chunk of code until it hits another yield return or the method ends.

This transformation is why you cannot use ref or out parameters in iterator methods, and why yield return cannot appear inside a try block with a catch. The state machine must preserve the exact position between calls.

Understanding this mechanism explains why the iterator body does not run at the time of the method call: the method call only creates the state machine object. The first call to MoveNext() runs the code from the beginning until the first yield return.

When the Iterator Body Actually Runs

The iterator body starts executing on the first call to MoveNext(), not when the method is invoked. This timing affects when exceptions are thrown. If an iterator method throws an exception before the first yield return, that exception is not thrown when you call the method. It is thrown when you start enumerating.

For example:

public static IEnumerable<int> GetNumbers()
{
    throw new InvalidOperationException("Boom");
    yield return 1;
}

Calling GetNumbers() succeeds and returns an enumerable. The exception is thrown when you call GetEnumerator().MoveNext() or use foreach. This is different from a regular method, where the exception would be thrown immediately.

This behavior is important for APIs that return sequences: the caller might assume that a method call validates its arguments, but with deferred execution, validation must be done separately if needed.

Side Effects and Exception Timing

Because the iterator body executes lazily, any side effects it performs—writing to a log, modifying a shared field, incrementing a counter—happen during enumeration, not at the method call. If you enumerate the same sequence twice, the side effects happen twice. This is a common source of bugs.

Consider a method that reads from a file or a database. If you return an iterator that reads rows lazily, the file or connection is opened only when you start iterating. If you iterate multiple times, you may get different results if the underlying data changes. This is often desirable for streaming, but it also means you must be careful about resource lifetime.

Performance and Memory Implications

Deferred execution allows you to build pipelines that avoid materializing intermediate collections. For example, you can chain LINQ operations like Where, Select, and Take without creating arrays or lists until the final result is consumed. This can reduce memory usage and improve startup time for large sequences.

However, deferred execution also has costs. Each MoveNext() call involves a state machine transition, which has some overhead compared to a simple loop over an array. For small sequences, the overhead is negligible, but for hot paths with millions of iterations, it can matter. Also, if you enumerate the same iterator multiple times, the code runs again, which can be expensive.

The key tradeoff is between memory and CPU. Materializing with ToList() gives you a snapshot and allows repeated enumeration without re-executing the iterator, but it allocates memory. Deferred execution keeps memory low but may re-run the iterator logic.

Forcing Execution with ToList and ToArray

When you need to execute the iterator immediately, you can call ToList() or ToArray() on the enumerable. This forces the iterator to run to completion and stores the results in a collection. After that, the collection is independent of the original iterator.

var numbers = GetNumbers().ToList();

Now the iterator has run once, and numbers is a List<int> that you can enumerate multiple times without re-running the iterator logic. This is useful when you need a snapshot of the data or when the iterator has side effects that should happen only once.

Be careful with infinite sequences: calling ToList() on an infinite iterator will never return. Deferred execution allows you to take only the first few items with Take, but materializing the whole sequence is impossible.

Common Pitfalls with Multiple Enumeration

A frequent mistake is enumerating an iterator multiple times without realizing it. For example, if you pass an IEnumerable<T> to a method that iterates it twice, the iterator runs twice. If the iterator is expensive or has side effects, this can cause unexpected behavior.

public static void Process(IEnumerable<int> items)
{
    var count = items.Count();
    foreach (var item in items)
    {
        // ...
    }
}

If items is a deferred iterator, Count() forces the first enumeration, and the foreach forces a second. This can be avoided by materializing the sequence once at the start of the method.

Deferred Execution in LINQ and Custom Iterators

LINQ query operators like Where, Select, and OrderBy are implemented as iterator methods, so they also defer execution. When you write var query = source.Where(x => x > 5);, the query is not executed until you enumerate it. This is why you can build a query and later change the source or the predicate without re-creating the query.

Custom iterators follow the same rules. If you implement a method that returns IEnumerable<T> using yield, you inherit all the deferred execution behavior. This is useful for creating streaming data sources, but you must document the behavior so callers understand when side effects occur.

One edge case: if you use yield break, the iterator ends without yielding any items. The method body still does not run until enumeration, and yield break simply signals the end of the sequence.