Python List Comprehension: Syntax, Filtering, and Performance

A python list comprehension builds a new list by applying an expression to each item in an iterable. The syntax follows a compact form:

numbers = [1, 2, 3, 4, 5]
squares = [n * n for n in numbers]

The expression n * n is evaluated for each element in numbers, and the results are collected into a new list. The comprehension reads left to right: first the expression, then the for clause that defines the iteration variable and the source iterable.

This replaces the more verbose equivalent:

squares = []
for n in numbers:
    squares.append(n * n)

The comprehension version is shorter and keeps the transformation visible in a single line. The loop version is still useful when the transformation requires multiple statements, such as accumulating state across iterations or handling exceptions per element.

Filtering with Conditional Expressions

Adding an if clause at the end of the comprehension filters the source iterable before the expression is evaluated:

even_squares = [n * n for n in numbers if n % 2 == 0]

Only elements for which the condition evaluates to True reach the expression. The condition is evaluated for every element, so the order of clauses matters. A comprehension with both a filter and a transformation applies the filter first, then the transformation.

The equivalent loop form is:

even_squares = []
for n in numbers:
    if n % 2 == 0:
        even_squares.append(n * n)

Conditional expressions can also appear inside the expression itself, using the ternary operator:

labels = ["even" if n % 2 == 0 else "odd" for n in numbers]

This is different from an if clause. The ternary selects between two output values for every element, while the if clause removes elements entirely. Mixing both is possible but can reduce readability quickly.

Nested Loops and Flat Iteration

Multiple for clauses in a single comprehension produce a flat list from nested iteration:

pairs = [(x, y) for x in range(3) for y in range(3)]

The first for clause is the outer loop, and each subsequent clause is an inner loop. The result is equivalent to:

pairs = []
for x in range(3):
    for y in range(3):
        pairs.append((x, y))

Nested comprehensions, where a comprehension appears inside another comprehension's expression, produce a different structure. A nested comprehension creates a list of lists:

matrix = [[x * y for y in range(3)] for x in range(3)]

The inner comprehension runs once per outer iteration, producing a new list each time. This is useful for building grids or matrices, but the nesting depth should stay small. Beyond two levels, the comprehension becomes difficult to read and a regular loop structure is easier to maintain.

When to Use a Generator Expression Instead

A list comprehension constructs the entire list in memory before any element is consumed. For large inputs, this can be wasteful when the result is only iterated once. A generator expression uses the same syntax with parentheses instead of brackets:

squares_gen = (n * n for n in numbers)

The generator produces values lazily, one at a time, as it is iterated. This matters when the source iterable is large or when the result is passed directly to a function that consumes it immediately:

total = sum(n * n for n in numbers)

Here the generator avoids building an intermediate list of squares. The sum function consumes each value as it is produced. The memory cost stays proportional to the input size rather than the output size.

The tradeoff is that a generator can only be iterated once. If the result must be accessed multiple times or indexed, a list comprehension is the correct choice.

Performance Characteristics

List comprehensions are generally faster than equivalent for loops with append because the loop mechanics and method lookup are handled internally by the interpreter. The comprehension avoids repeated attribute lookups on the list object and the function call overhead of append.

That said, the performance difference is usually modest for small collections. The real cost differences appear when the transformation itself is expensive, such as calling a function that does significant work per element. In that case, the comprehension's speed advantage is small relative to the cost of the transformation, and readability should guide the choice.

Memory usage is the more significant concern. A list comprehension allocates the full result list. If the input is a large iterator and the output is also large, memory usage can become a problem. A generator expression avoids that allocation but cannot be reused.

When the same transformation is applied repeatedly to the same data, storing the result in a list is often better than regenerating it, because the transformation cost is paid once instead of per iteration.

Common Mistakes and Readability Concerns

One frequent mistake is placing the if clause in the wrong position when combining a filter with a ternary expression. The if clause always appears after the for clause, while the ternary appears inside the expression:

# Correct: filter first, then transform
result = [n * 2 for n in numbers if n > 0]

# Incorrect: this is a syntax error
# result = [n * 2 if n > 0 for n in numbers]

Another issue is using a comprehension for its side effects. A comprehension creates a new list; using it purely to call a function on each element wastes memory and obscures intent. A regular for loop is clearer for side-effect-only operations:

# Unclear: builds a list of None values
[print(n) for n in numbers]

# Clearer
for n in numbers:
    print(n)

Readability degrades quickly when a comprehension spans multiple conditions and transformations. A good rule of thumb is to keep comprehensions to a single for clause and a single if clause. Beyond that, a generator function or a regular loop with intermediate variables is easier to debug and modify.

Maintaining Readability in Complex Comprehensions

When a comprehension grows beyond a simple transformation, refactoring the expression into a named function keeps the comprehension readable:

def classify(n):
    if n < 0:
        return "negative"
    if n == 0:
        return "zero"
    return "positive"

labels = [classify(n) for n in numbers]

The comprehension stays short, and the logic lives in a testable function. This pattern is especially useful when the same classification logic is needed in multiple places.

For deeply nested or multi-step transformations, a generator function using yield provides the same lazy evaluation as a generator expression while allowing intermediate variables and multiple statements:

def process(items):
    for item in items:
        cleaned = item.strip().lower()
        if cleaned:
            yield cleaned

results = list(process(raw_items))

This keeps the transformation logic explicit while still producing a list when needed. The comprehension remains the right tool for simple, single-expression transformations, but the boundary where it becomes a liability is easy to cross.