Python Iterator vs Iterable: Key Differences Explained
python iterator vs iterable: Understand the distinction between iterables and iterators in Python, how the iterator protocol works, and when to use each for efficient...
When working with Python, the distinction between an iterable and an iterator often causes confusion. The two terms are related but refer to different behavior. An iterable is an object that can be looped over, while an iterator is an object that produces values one at a time. The for loop in Python relies on this distinction internally. Understanding python iterator vs iterable helps you write clearer code, debug iteration issues, and design objects that integrate cleanly with Python's iteration machinery.
What Is an Iterable in Python?
An iterable is an object that can be passed to the built-in iter() function to get an iterator. In practice, anything that works in a for loop is iterable. Lists, tuples, strings, dictionaries, sets, and files are all iterables. They implement the __iter__() method, which returns an iterator object.
numbers = [1, 2, 3] iterator = iter(numbers) print(iterator) # <list_iterator object at 0x...>
The iter() call returns an iterator that knows how to traverse the list. The list itself is not an iterator; it is an iterable because it can produce an iterator when requested.
What Is an Iterator in Python?
An iterator is an object that implements the iterator protocol, consisting of two methods: __iter__() and __next__(). The __iter__() method returns the iterator object itself, and __next__() returns the next value in the sequence, raising StopIteration when there are no more values.
class CountUp: def __init__(self, limit): self.limit = limit self.current = 0 def __iter__(self): return self def __next__(self): if self.current >= self.limit: raise StopIteration self.current += 1 return self.current counter = CountUp(3) print(next(counter)) # 1 print(next(counter)) # 2 print(next(counter)) # 3 print(next(counter)) # raises StopIteration
An iterator is its own iterable because __iter__() returns self. This allows an iterator to be used directly in a for loop.
The Iterator Protocol: __iter__ and __next__
The iterator protocol is the contract that makes iteration work. When you call iter(obj) on an iterable, Python looks for the __iter__() method. If found, it calls it to obtain an iterator. If not, Python falls back to the old sequence protocol using __getitem__() with integer indices, but that is less common in modern code.
For an iterator, __next__() is the core method. It returns the next element or raises StopIteration to signal the end. The for loop handles this automatically by calling next() repeatedly until StopIteration is raised.
class ReverseString: def __init__(self, value): self.value = value self.index = len(value) def __iter__(self): return self def __next__(self): if self.index == 0: n raise StopIteration self.index -= 1 return self.value[self.index] for char in ReverseString("abc"): print(char) # c, b, a
Notice that the iterator maintains its own state. Each call to __next__() advances the internal pointer. This statefulness is what distinguishes an iterator from a plain iterable.
How for Loops Use Iterables and Iterators
When you write a for loop, Python performs two steps behind the scenes. First, it calls iter() on the loop target to get an iterator. Then it repeatedly calls next() on that iterator until StopIteration is raised.
for item in [1, 2, 3]: print(item)
This is equivalent to:
iterator = iter([1, 2, 3]) while True: try: item = next(iterator) except StopIteration: break print(item)
This explains why you can iterate over the same list multiple times but not over the same iterator twice. A list is an iterable that produces a fresh iterator each time you call iter(). An iterator is exhausted after one pass.
numbers = [1, 2, 3] for n in numbers: print(n) for n in numbers: print(n) # works again iterator = iter(numbers) for n in iterator: print(n) for n in iterator: print(n) # nothing printed, iterator is exhausted
Creating Your Own Iterable and Iterator
To create an iterable, you define a class with an __iter__() method that returns an iterator. The iterator can be a separate class or the same class if it also implements __next__(). Separating them is often clearer because it allows the iterable to produce multiple independent iterators.
class RangeIterable: def __init__(self, start, stop): self.start = start self.stop = stop def __iter__(self): return RangeIterator(self.start, self.stop) class RangeIterator: def __init__(self, start, stop): self.current = start self.stop = stop def __iter__(self): return self def __next__(self): if self.current >= self.stop: raise StopIteration result = self.current self.current += 1 return result for i in RangeIterable(1, 4): print(i)
Each call to iter() on the RangeIterable creates a fresh RangeIterator. This allows multiple simultaneous loops over the same iterable without interference.
Generators: Iterators Without Boilerplate
Writing a full iterator class requires a lot of boilerplate. Python provides generators as a shortcut. A generator function uses yield instead of return. When called, it returns a generator object, which is an iterator. The generator automatically maintains state between next() calls.
def count_up(limit): current = 0 while current < limit: current += 1 yield current for n in count_up(3): n print(n) # 1, 2, 3
The generator object is both an iterable and an iterator. It has __iter__() returning itself and __next__() that resumes execution until the next yield. Generators are the idiomatic way to create iterators in Python when you don't need a custom class with additional methods.
Generator expressions provide a concise way to create generators inline:
squares = (x * x for x in range(10)) nprint(next(squares)) # 0 print(next(squares)) # 1
Common Mistakes and Edge Cases
One common mistake is assuming that an iterator can be reused. Once an iterator raises StopIteration, it is permanently exhausted. Calling iter() on it again returns the same exhausted iterator.
iterator = iter([1, 2]) list(iterator) # [1, 2] list(iterator) # []
Another mistake is confusing an iterable with an iterator. For example, a list is iterable but not an iterator. Calling next() on a list raises TypeError: 'list' object is not an iterator. You must first call iter() to get an iterator.
numbers = [1, 2, 3] next(numbers) # TypeError
A subtle edge case occurs when an object implements both __iter__() and __next__(). Such an object is its own iterator. This is common for generator objects and for classes that represent a single sequence. In that case, you cannot iterate over it twice without recreating it.
Performance and Memory Implications
The main performance advantage of iterators is lazy evaluation. An iterator computes each value on demand rather than materializing the entire sequence in memory. This is critical when working with large datasets or infinite sequences.
# Without generator: creates a huge list in memory squares_list = [x * x for x in range(10**6)] # With generator: computes on the fly squares_gen = (x * x for x in range(10**6))
The generator uses a constant amount of memory regardless of the range size. The list comprehension, on the other hand, allocates memory proportional to the number of elements. For finite, small collections, the difference is negligible, but for large streams, generators are the right choice.
Iterators also avoid the overhead of building a new list when you only need to process items once. For example, passing a generator to sum() or max() avoids an intermediate list.
However, iterators have a cost: they are single-pass. If you need to iterate over the same data multiple times, you either need to recreate the iterator or store the data in an iterable like a list. This tradeoff is fundamental to the design.
When to Choose Which Approach
Use an iterable when you need to provide multiple independent passes over the same data. Lists, tuples, and custom iterables that return fresh iterators are appropriate when the data is finite and can be revisited.
Use an iterator when you need a single pass over a sequence, especially if the sequence is large or infinite. Generators are the simplest way to create iterators for most use cases. Custom iterator classes are only necessary when you need to maintain additional state or provide methods beyond __next__().
If you are building a container object that holds data, make it an iterable by implementing __iter__() to return a new iterator each time. If you are building a stream or a source of values, implement an iterator directly.
For example, a Deck class that represents a collection of cards should be iterable so you can deal cards multiple times. A FibonacciGenerator should be an iterator because it produces an endless sequence.
class Deck: def __init__(self, cards): self.cards = cards def __iter__(self): return iter(self.cards) def fibonacci(): a, b = 0, 1 while True: yield a a, b = b, a + b
The Deck is iterable and can be looped over repeatedly. The fibonacci function returns an iterator that can be consumed with next() indefinitely.
Understanding the distinction between iterables and iterators allows you to design Python objects that behave naturally with language features like for loops, comprehensions, and the * unpacking operator. It also helps you debug issues where an object appears to be empty on the second loop, which is almost always a sign that you are dealing with an exhausted iterator rather than an iterable.