PHP Generators - A useful example

PHP5.5 has support for generators which are a powerful language feature available in other languages like Ruby and Python. While generators in PHP are very much like their Python counterparts, I wanted to give them a spin and try a few simple but useful examples of generators in action.

Generator basics

Building iterators in PHP has never been a fun prospect. The Iterator interface requires XX methods which add a non-trivial amount of boiler plate to your code. Generators allow you to easily build forward only iterators. Any function can be turned into a generator function by including the yield keyword. Once a function becomes generator, it behaves a bit differently than a normal function. When invoked, it returns an instance of Generator which is a special kind of iterator. When iterated the generator stops execution at the first yield. Upon subsequent iterations, execution resumes at the yield resumes and runs until the execution flow hits either another yield or an empty return statement. The fact that generator functions effectively ‘pause’ is really interesting because it allows you to make simple & efficient iterators. The most basic use case for generators are functions that allow you to traverse a large list of numbers. Without generators you might implement it like:

Show Plain Text
  1. function xrange($len) {
  2.     $out = [];
  3.     for ($i = 0; $i < $len; $i++) {
  4.         $out[] = $i;
  5.     }
  6.     return $out;
  7. }
  8. foreach (xrange(1000) as $i) {
  9.     // Do something
  10. }

While this is a slightly contrived example it shows a problem that can be solved much more elegantly and efficiently with generators.

Show Plain Text
  1. function xrange($len) {
  2.     $i = 0;
  3.     while ($i < $len) {
  4.         yield $i;
  5.         $i++;
  6.     }
  7.     return;
  8. }
  9.  
  10. foreach (xrange(1000) as $i) {
  11.     // Do something
  12. }

In the example above we never need to allocate more than 2 integers at any point in time. This gives us a big savings in memory as we don’t need to make a huge array before we can start iterating it.

CSV iterator

While the initial example is a bit contrived and trivial, one real world use case I have for generators is parsing CSV files. I’ve done work with CSV files that would not fit into memory once parsed. In the past I’ve used classes that implement the Iterator interface to allow me to incrementally parse a file CSV file and operate on the data. With generators this task can be done much more simply:

Show Plain Text
  1. class CsvIterator {
  2.  
  3.     protected $file;
  4.  
  5.     public function __construct($file) {
  6.         $this->file = fopen($file, 'r');
  7.     }
  8.  
  9.     public function parse() {
  10.         $headers = array_map('trim', fgetcsv($this->file, 4096));
  11.         while (!feof($this->file)) {
  12.             $row = array_map('trim', (array)fgetcsv($this->file, 4096));
  13.             if (count($headers) !== count($row)) {
  14.                 continue;
  15.             }
  16.             $row = array_combine($headers, $row);
  17.             yield $row;
  18.         }
  19.         return;
  20.     }
  21. }
  22.  
  23. $csv = new CsvIterator('/path/to/huge/file.csv');
  24. foreach ($csv->parse() as $row) {
  25.     // Do something with the row.
  26. }

My previous implementation of this took 60 lines of code, while using a generater used fewer than 20. I’m really excited for generators, as the make creating forward only iterators much simpler than previously possible.

Comments

while ($i < $len; $i++) {

you sure?

prokhor on 12/10/13

prokhor: Yep, that is how the xrange() function in python works as well.

mark story on 12/16/13

> while ($i < $len; $i++) {

I have tested the script on 5.5-dev, seems like that is not supported.
See: http://codepad.viper-7.com/AHGSRH

Maybe it work on latest versions?
If it does, should you remove the extra `$i++;` from line 5?

> while ($i < $len) {

This works as expected.
See: http://codepad.viper-7.com/cyb8YY

Renan Gonçalves on 2/13/14

Renan Gonçalves: You’re totally right, I’ve updated the post.

mark story on 2/17/14

I think that the article need more descriptive example to demonstrate the benefit of generators. I could not understand what it really do.

Said Bakr on 4/13/14

@Said Bakr
I think once one gets a good grasp on iterators and actually works with them, only then can one appreciate the value of generators.

Angel S. Moreno on 5/9/14

Comments are not open at this time.