Python Comprehension Multiple Loops

Python comprehension multiple loops let you build lists, dictionaries, sets, and generators from nested iterations in a single expression. The syntax is compact, but the order of the for clauses matters: they execute in the same order as nested for loops. Misplacing a clause changes the result or raises a NameError when a variable is referenced before assignment.

Basic Syntax of Multiple Loops in Comprehensions

A comprehension with multiple loops looks like this:

pairs = [(x, y) for x in range(3) for y in range(2)]
# [(0, 0), (0, 1), (1, 0), (1, 1), (2, 0), (2, 1)]

The first for clause is the outer loop, and each subsequent for clause is nested inside the previous one. This is equivalent to:

pairs = []
for x in range(3):
    for y in range(2):
        pairs.append((x, y))

The comprehension version produces the same list in the same order. The expression at the front of the comprehension is evaluated once per combination of loop variables.

The same pattern works for dictionary, set, and generator comprehensions:

# Dictionary
matrix = {(i, j): i + j for i in range(3) for j in range(3)}

# Set
unique_sums = {i + j for i in range(3) for j in range(3)}

# Generator
gen = ((i, j) for i in range(3) for j in range(3))

How Nested Loops Map to Comprehension Order

Reading a multi-loop comprehension from left to right mirrors the indentation of equivalent nested loops. The first for is the outermost, and the last for is the innermost. This ordering is not just stylistic; it determines which variables are available at each stage.

Consider flattening a matrix:

matrix = [[1, 2], [3, 4]]
flat = [value for row in matrix for value in row]
# [1, 2, 3, 4]

Here row is defined by the first loop, and the second loop iterates over row. Reversing the loops would raise a NameError because row would not exist yet:

# NameError: name 'row' is not defined
flat = [value for value in row for row in matrix]

This is a common mistake. Always ensure that a variable used in a later for clause is bound by an earlier one.

The order also affects the number of iterations. The total number of combinations is the product of the lengths of all iterables, unless a condition filters some out.

Using Multiple Loops with Conditions

You can add if clauses after any for clause. The condition applies to the current state of the loop variables. For example, to get all pairs where the first element is greater than the second:

pairs = [(x, y) for x in range(5) for y in range(5) if x > y]

The if appears after the second for, so it can use both x and y. You can also filter after the first loop, before the second loop runs:

pairs = [(x, y) for x in range(5) if x % 2 == 0 for y in range(5)]

This only iterates over even x values, reducing the number of inner-loop executions. The placement of if clauses changes both the result and the runtime cost. A condition placed earlier reduces the number of iterations for subsequent loops, which can be significant when the iterables are large.

Performance Considerations for Multi-Loop Comprehensions

Comprehensions are generally faster than an equivalent for loop that builds a list with append() because the list append operation is optimized in the comprehension's bytecode. However, the performance advantage shrinks when you add many loops and conditions, especially if the loop bodies are complex.

The main performance concern with multi-loop comprehensions is memory. A list comprehension materializes the entire result in memory. For a product of two large ranges, the result can be enormous. If you only need to iterate once, use a generator expression instead:

gen = ((x, y) for x in range(1000) for y in range(1000))

This does not create the full list; it yields one pair at a time. The memory footprint is constant, but each iteration still has the same computational cost.

Another subtle cost is repeated evaluation of the iterable in the inner loop. If the inner iterable is a generator, it is consumed once. If it is a list, it is re-iterated for each outer value. For example:

items = [1, 2, 3]
pairs = [(x, y) for x in range(3) for y in items]

items is iterated three times. That is fine for a list, but if items is a generator, the second outer iteration will see an exhausted generator. Always use re-iterable sequences (lists, tuples, ranges) for inner loops unless you intentionally want to consume a generator.

Readability and Maintainability

Multi-loop comprehensions become hard to read when they contain more than two or three loops or when the expression is complex. A nested for loop with clear indentation is often more maintainable, even if it is longer.

Consider this comprehension:

result = [
    (a, b, c)
    for a in range(10)
    for b in range(a)
    for c in range(b)
    if a + b + c > 10
]

It is not immediately obvious what the code does. The equivalent explicit loops make the structure clearer:

result = []
for a in range(10):
    for b in range(a):
        for c in range(b):
            if a + b + c > 10:
                result.append((a, b, c))

Readability is subjective, but a good rule is to use a comprehension when the logic fits on one or two lines and the expression is simple. For anything more involved, prefer explicit loops or break the logic into helper functions.

Another maintainability issue is variable name reuse. In a comprehension, the loop variables are local to the comprehension and do not leak into the enclosing scope (in Python 3). This is usually beneficial, but it can confuse readers if the same name is used elsewhere in the function. Use descriptive names that reflect the domain, even if they make the line longer.

Common Pitfalls and Edge Cases

One common pitfall is using the same variable name in nested loops. Python allows it, but the inner loop overwrites the outer variable for the remainder of the inner iteration:

pairs = [(i, i) for i in range(3) for i in range(2)]
# [(0, 0), (0, 1), (1, 0), (1, 1), (2, 0), (2, 1)]

This works because the second i shadows the first, but the result is confusing. Avoid reusing variable names in the same comprehension.

Another edge case is the interaction with if clauses and variable scope. An if clause placed before a for clause cannot reference the loop variable from that later loop. For example:

# NameError: name 'y' is not defined
[(x, y) for x in range(3) if y > 0 for y in range(3)]

The condition is evaluated before y is bound. Always place if clauses after the for clause that defines the variables they use.

Finally, be careful with comprehensions that have side effects, such as calling functions that modify external state. A comprehension is an expression, and the order of evaluation is guaranteed left-to-right, but the code becomes harder to debug. If you need side effects, use an explicit loop.

Alternatives: itertools.product and Other Tools

For Cartesian products, itertools.product is often clearer and more flexible than a nested comprehension:

from itertools import product

pairs = list(product(range(3), range(2)))
# [(0, 0), (0, 1), (1, 0), (1, 1), (2, 0), (2, 1)]

product accepts any number of iterables and has a repeat parameter for repeated products. It returns an iterator, so you can wrap it in list() or iterate directly. This avoids the nested for syntax entirely and is more readable when you only need a Cartesian product without filtering.

For flattening nested structures, itertools.chain.from_iterable can be combined with a single-loop comprehension:

from itertools import chain
flat = list(chain.from_iterable(matrix))

This is more efficient than a double loop when you are only flattening one level.

When you need to combine two lists element-wise rather than as a product, use zip:

combined = [(a, b) for a, b in zip(list1, list2)]

A multi-loop comprehension is not always the right tool. Choosing the right standard-library function can make the code shorter and the intent clearer.

When to Choose a Comprehension Over an Explicit Loop

The decision depends on three factors: complexity, performance, and readability. A comprehension is appropriate when the logic is a simple mapping or filtering over nested iterations and the result is a container or generator. It is not appropriate when the loop body contains multiple statements, needs to modify external state, or requires exception handling.

If you need to break out of the loop early, a comprehension cannot do that directly. You would need to use a generator function or an explicit loop. Similarly, if you need to accumulate values based on a condition that depends on previous results, a comprehension is not suitable.

In terms of performance, comprehensions are usually faster than explicit loops for small to medium-sized data. For very large data, the memory overhead of a list comprehension can be a problem; use a generator expression or an explicit loop that writes to a file or another sink.

Ultimately, the goal is to write code that is correct, efficient, and easy for the next developer to understand. A multi-loop comprehension is a powerful tool, but it is not always the best choice. Measure and profile when performance is critical, and prioritize readability when the logic is complex.