In the recent bakery article concerning the ongoing development of CakePHP 2.0. The already underway migration from SimpleTest to PHPUnit was introduced. I wanted to go into some of the reasons and motivations for that decision as well as explain some of the long term benefits.
Why the change?
Currently there are over 15,000 assertions and 650 files in the CakePHP test suite. A large application can easily have that many tests as well. Changing the underlying test suite framework for a code base of that size is not a small undertaking. Moving CakePHP away from SimpleTest is not something that was lightly decided upon. But there were several contributing reasons that we decided to move to PHPUnit:
- SimpleTest isn’t
E_STRICT. This impairs our ability to create a code base that follows
E_STRICT. Having to wade through 1000’s of warnings to find the relevant ones is an arduous task. Moving to PHPUnit makes fixing the
E_STRICTwarnings in CakePHP easier as there are fewer of them to read.
- PHPUnit boasts a plethora of ancillary tools. From Continuous integration to build automation. Tooling support has gathered behind PHPUnit, and a switch will allow Cake developers to leverage the great tool support.
- PHPUnit is the de facto standard testing framework now. Currently Zend Framework and Yii as well as many other projects use PHPUnit. Symfony and Doctrine will be moving to PHPUnit with their next releases. The mindshare and market share is clearly behind PHPUnit. Moving CakePHP to PHPUnit helps ease the developer experience, as new developers may already be familiar with how to use and write tests.
But all my tests are now broken!
Switching unit test frameworks can be a painful process. There are many differences between PHPUnit and SimpleTest internally. We are very aware of this pain as we migrate all of CakePHP’s tests to use PHPUnit. Our aim is that if your test cases extend
CakeTestCase you might not even really notice the change. We will be working hard to try and ensure that the various differences between SimpleTest and PHPUnit’s assertion methods are wrapped and just work wherever we can. jose_zap has already ported fixtures over and there should be no difference to you from before. The largest pain point for us so far has been the
Mock class SimpleTest provides. Mock objects are implemented very differently in PHPUnit, and there is no easy way to hide the differences. With that said there will be a section in the migration covering how to update your tests.
The move to PHPUnit has given code coverage report generation a huge boost. Using the features PHPUnit provides has made code coverage generation faster, and provide more information. Code coverage with PHPUnit now reports on all files loaded during a test case. This gives you a more complete view of what files/objects are interconnected. The PHPUnit test suite also features the ability to find out which test methods covered a specific line of code. Hovering your mouse over any covered line to find out which tests ran through that line. This of course requires a browser that correctly handles
I think this is a pretty neat feature, and I hope it shows you some of the improvements we’ll be getting from the switch to PHPUnit. I totally understand that this could be a pain point in upgrading your applications. However, I think in the long run it will be a beneficial change. The ability to access better tooling and a more uniform testing environment with other projects is something that I feel will pay off in the long term.