Python Iterable: The Iteration Protocol Explained

python iterable requires a clear understanding of the core syntax, runtime behavior, and practical implementation patterns demonstrated in the examples below.

When you write for x in something in Python, the interpreter relies on a contract that something must satisfy: it must be an iterable. Understanding what that contract is, how it works under the hood, and how to create your own iterables is a core skill for writing idiomatic Python. This article explains the iteration protocol, the difference between iterables and iterators, and how to build custom iterables both with classes and with generators.

What Is an Iterable in Python?

An iterable is any object that can return an iterator, which is an object that produces a sequence of values one at a time. In Python, an object is iterable if it defines the __iter__() method, or if it defines __getitem__() for sequence-style access. The for loop, list comprehensions, and many other constructs rely on this protocol.

For example, lists, tuples, dictionaries, sets, strings, and ranges are all iterables. They each implement __iter__() and can be used directly in a for loop:

for item in [1, 2, 3]:
    print(item)

Under the hood, Python calls iter() on the list, which returns an iterator. The loop then repeatedly calls next() on that iterator until StopIteration is raised. This is the essence of the iteration protocol.

The Iteration Protocol: iter and next

The iteration protocol consists of two methods:

• __iter__(self) returns an iterator object. For an iterable, this method is required.
• __next__(self) returns the next value from the iterator. When no more values are available, it raises StopIteration.

An iterator is an object that implements both __iter__() and __next__(). The __iter__() method of an iterator typically returns self, making the iterator itself iterable.

Here is a minimal iterator that yields numbers from 0 to a limit:

class Counter:
    def __init__(self, limit):
        self.limit = limit
        self.value = 0

    def __iter__(self):
        return self

    def __next__(self):
        if self.value >= self.limit:
            raise StopIteration
        current = self.value
        self.value += 1
        return current

You can use this iterator directly in a for loop:

for n in Counter(3):
    print(n)

This prints 0, 1, 2. The for loop calls iter(Counter(3)), which returns the same object, then calls next() repeatedly until StopIteration is raised.

Common Built-in Iterables

Python's standard library and built-in types provide many iterables. Knowing which types are iterable and how they behave is important for choosing the right data structure.

Dictionaries are iterable, but iterating over a dict yields its keys. If you need values or key-value pairs, use .values() or .items(), which return view objects that are also iterable.

d = {'a': 1, 'b': 2}
for key in d:
    print(key)  # a, b

for value in d.values():
    print(value)  # 1, 2

for key, value in d.items():
    print(key, value)

Building a Custom Iterable Class

To make your own class iterable, you need to implement __iter__(). The simplest approach is to make the class its own iterator by also implementing __next__(). However, this pattern has a limitation: the object can only be iterated once. If you want to iterate over the same object multiple times, you need __iter__() to return a fresh iterator each time.

A better pattern is to separate the iterable from the iterator. The iterable defines __iter__() and returns a new iterator object. The iterator holds the state and implements __next__().

class RangeIterable:
    def __init__(self, start, end):
        self.start = start
        self.end = end

    def __iter__(self):
        return RangeIterator(self.start, self.end)

class RangeIterator:
    def __init__(self, start, end):
        self.current = start
        self.end = end

    def __iter__(self):
        return self

    def __next__(self):
        if self.current >= self.end:
            raise StopIteration
        value = self.current
        self.current += 1
        return value

Now you can iterate over the same RangeIterable instance multiple times because each call to iter() creates a new iterator.

r = RangeIterable(0, 3)
print(list(r))  # [0, 1, 2]
print(list(r))  # [0, 1, 2]

Using Generators for Lazy Iteration

Writing full iterator classes is verbose. Python's generator functions provide a concise way to create iterators. A generator function is defined with yield instead of return. When called, it returns a generator object, which is an iterator.

def count_up_to(limit):
    value = 0
    while value < limit:
        yield value
        value += 1

Using it:

for n in count_up_to(3):
    print(n)

Generators are lazy: they compute values on demand and do not store the entire sequence in memory. This makes them ideal for large or infinite sequences.

You can also use generator expressions, which are similar to list comprehensions but with parentheses:

squares = (x * x for x in range(10))

This creates a generator that yields squares one at a time. The generator expression does not allocate a list; it computes each value when requested.

Iterators vs Iterables: How iter() Works

Every iterator is an iterable because it implements __iter__() and returns itself. But not every iterable is an iterator. A list is iterable but not an iterator; it has an __iter__() method but no __next__().

The built-in iter() function is the key. When called on an iterable, it returns an iterator. When called on an iterator, it returns the same object. This is why a for loop works on both.

Consider this example:

numbers = [1, 2, 3]
iterator = iter(numbers)
print(next(iterator))  # 1
print(next(iterator))  # 2

The list itself does not keep track of a current position. The iterator does. Once an iterator is exhausted, calling next() raises StopIteration. You cannot reset an iterator; you must create a new one.

Performance and Memory Considerations

One of the main reasons to understand iterables is to manage memory efficiently. When you process a large dataset, loading it all into a list can consume significant memory. Generators and lazy iterables produce items one at a time, reducing peak memory usage.

For example, reading a large file line by line:

with open('huge.log') as f:
    for line in f:
        process(line)

The file object is an iterable that yields lines lazily. It does not read the entire file into memory. Similarly, range is lazy; range(10**9) does not allocate a billion integers.

However, lazy iteration is not always free. Each next() call has overhead, and some operations require random access. If you need to index into a sequence repeatedly, a list or tuple is more appropriate. The choice depends on whether you need the entire collection at once or can process items sequentially.

Common Pitfalls and Edge Cases

A common mistake is to confuse an iterable with an iterator and try to reuse an exhausted iterator. For example:

gen = (x for x in range(3))
print(list(gen))  # [0, 1, 2]
print(list(gen))  # []

Once a generator is exhausted, it cannot be restarted. If you need to iterate multiple times, create a new generator or use an iterable that returns a fresh iterator each time.

Another pitfall is modifying a dictionary while iterating over it. This raises RuntimeError because the dictionary's size changes during iteration. To modify a dict, iterate over a copy of its keys:

d = {'a': 1, 'b': 2}
for key in list(d.keys()):
    if key == 'a':
        del d[key]

Finally, be aware that custom iterables that return self from __iter__() are single-use. If you need multi-pass iteration, follow the separate-iterator pattern shown earlier. Understanding these edge cases helps you avoid subtle bugs in production code.