Topic 3: Iterators and Generators

1. Introduction

In Python, iterators are objects that implement the iterator protocol, which consists of the methods __iter__() and __next__(). Generators, on the other hand, are a simple way to create iterators using functions rather than classes. They employ the yield keyword to yield values, which makes the function return a generator object.

2. Iterators

  • Definition: An iterator is an object that can be iterated upon, meaning that you can traverse through its values.

  • Making an Iterator: To make an object iterable, it must implement two methods, __iter__() and __next__().

    class Counter: def __init__(self, limit): self.limit = limit self.count = 0 def __iter__(self): return self def __next__(self): if self.count < self.limit: value = self.count self.count += 1 return value else: raise StopIteration counter = Counter(3) for c in counter: print(c)


    0 1 2

3. Generators

  • Definition: A generator is a special type of iterator that is defined by functions rather than classes.

  • Creating Generators: Generators are built using the yield keyword.

    def simple_generator(): yield 1 yield 2 yield 3 for value in simple_generator(): print(value)


    1 2 3

4. Generator Expressions

  • They are a more concise way to create simple generators with a syntax similar to list comprehensions:

    squared = (x*x for x in range(5)) for value in squared: print(value)


    0 1 4 9 16

5. Advantages of Generators

  • Memory Efficiency: Since generators generate values on the fly and don’t store them in memory (like lists), they are more memory efficient for large datasets.

  • Lazy Evaluation: Generators produce values one at a time and only when required, which can lead to performance gains especially when dealing with substantial data streams.

6. Use Cases

  • Processing Large Files: Generators can be handy when reading large files, as they can process the file line by line without loading the entire file into memory.

  • Infinite Sequences: Generators are also used to represent infinite sequences, such as generating an infinite sequence of Fibonacci numbers.

7. Conclusion

Iterators and generators are fundamental concepts in Python, especially when it comes to efficient data processing and iteration. While iterators provide a generic way to traverse sequences, generators offer a simpler and more memory-efficient method to create custom iterators. It’s essential to know when to use which based on the problem at hand.