Unit Testing CakePHP Shells

Shells are one of the more difficult objects to properly unit test. Since they normally run in a CLI context instead of a web context they provide some interesting challenges. The biggest hurdles are separating the Shell from the CLI environment, and simulating the correct arguments and parameters.

Taking the environmental factors out

In order to separate the shell from CLI we use mock objects. We start off by mocking a few classes, mainly our subject shell and the ShellDispatcher. Mocking ShellDispatcher is important and vital to keeping our shell disconnected from its environment. For this example, I’ll use the test cases written for my AclExtras Shell . I use partial mocks for all shell tests as I only want to mock out the methods that directly tie the shell to a CLI environment. We’ll start our test case with the following

Show Plain Text
  1.  
  2. if (!defined('DISABLE_AUTO_DISPATCH')) {
  3.     define('DISABLE_AUTO_DISPATCH', true);
  4. }
  5.  
  6. if (!class_exists('ShellDispatcher')) {
  7.     ob_start();
  8.     $argv = false;
  9.     require CAKE . 'console' .  DS . 'cake.php';
  10.     ob_end_clean();
  11. }
  12.  
  13. Mock::generatePartial(
  14.     'ShellDispatcher', 'AclExtrasMockShellDispatcher',
  15.     array('getInput', 'stdout', 'stderr', '_stop', '_initEnvironment')
  16. );
  17.  

By mocking out the input and output and _stop methods, we disconnect the ShellDispatcher from the stdin and stdout, and allow our test case to run independent of being in a CLI context. Next up we need to mock out some methods on the class we actually want to test.

Show Plain Text
  1. Mock::generatePartial(
  2.     'AclExtrasShell', 'MockAclExtrasShell',
  3.     array('in', 'hr', 'out', 'err', 'createFile', '_stop', 'getControllerList')
  4. );

As seen above the most important methods again are the ones that connect to inputs and outputs. We also mock out _stop so our test continues to run even though the shell has attempted to exit. With both classes mocked we setup the start and end test methods.

Setting up the test case

Shell tests are really no different than regular test cases as this point. Shells need a slightly different startTest and endTest as well. Mine usually look something like:

Show Plain Text
  1. class AclExtrasShellTestCase extends CakeTestCase {
  2. //...
  3. function startTest() {
  4.     $this->Dispatcher =& new AclExtrasMockShellDispatcher();
  5.     $this->Task =& new MockAclExtrasShell($this->Dispatcher);
  6.     $this->Task->Dispatch =& $this->Dispatcher;
  7.     $this->Task->Dispatch->shellPaths = Configure::read('shellPaths');
  8. }
  9. //...
  10. function endTest() {
  11.     unset($this->Task, $this->Dispatcher);
  12.     ClassRegistry::flush();
  13. }

In the startTest creates an instance of our mocked ShellDispatcher and the mocked Shell class. We then manually assign the Dispatcher to the Shell with its Dispatch property. After setting the shellPaths we have simulated the objects and settings shells need to run. The endTest method does normal cleanup so we always start our test methods with new objects. Remember that if your shell uses any other shells to mock and construct those as well. You will also need to set Dispatch on all of them.

Asserting in() and out()

Since in() and out() have been mocked I use setReturnValue() and setReturnValueAt() to simulate user input and, expectAt() to make assertions on the generated output. For example:

Show Plain Text
  1. $this->Task->expectOnce('out');
  2. $this->Task->expectAt(0, 'out', array(new PatternExpectation('/recovered/')));

Sets the expectation that out() is called once and that the single time it is called, it will contain the pattern /recovered/. I find that setting an expectation for the call count as well as the individual calls works well. With only the expectations for the individual calls, you won’t get a fail if the method is never called. With only the call count expectation you don’t know what happened each time the method is called.

More complex examples

So the above works well, but what if you have a more complex shell interaction? In these cases I find that splitting things up into smaller methods works well. If that’s not possible, you just spend more time setting expectations on in() and out(). As an example, the following is part of the tests used for the ViewTask which is part of bake.

Show Plain Text
  1. $this->Task->connection = 'test_suite';
  2. $this->Task->args = array();
  3.  
  4. $this->Task->Controller->setReturnValue('getName', 'ViewTaskComments');
  5. $this->Task->Project->setReturnValue('getAdmin', 'admin_');
  6. $this->Task->setReturnValueAt(0, 'in', 'y');
  7. $this->Task->setReturnValueAt(1, 'in', 'n');
  8. $this->Task->setReturnValueAt(2, 'in', 'y');
  9.  
  10. $this->Task->expectCallCount('createFile', 4);
  11. $this->Task->expectAt(0, 'createFile', array(
  12.     TMP . 'view_task_comments' . DS . 'admin_index.ctp',
  13.     new PatternExpectation('/ViewTaskComment/')
  14. ));
  15. $this->Task->expectAt(1, 'createFile', array(
  16.     TMP . 'view_task_comments' . DS . 'admin_view.ctp',
  17.     new PatternExpectation('/ViewTaskComment/')
  18. ));
  19. $this->Task->expectAt(2, 'createFile', array(
  20.     TMP . 'view_task_comments' . DS . 'admin_add.ctp',
  21.     new PatternExpectation('/Add ViewTaskComment/')
  22. ));
  23. $this->Task->expectAt(3, 'createFile', array(
  24.     TMP . 'view_task_comments' . DS . 'admin_edit.ctp',
  25.     new PatternExpectation('/Edit ViewTaskComment/')
  26. ));
  27.  
  28. $this->Task->execute();

The above test uses multiple mocks, and shows a more complete example of how to set expectations and assertions on shell methods. As well as how to use inline PatternExpectations to assert the inputs of your mocked methods.

So I hope that makes testing shell classes a bit easier, and transparent. If you want to find additional examples there are many test cases for the core shell classes.

Comments

It’s a kick in the butt for me :) On monday I was searching how to test cakephp shells after finding basically nothing I thought to myself “Oh well need to look at cake core tests” and of course never did that…

Let’s say that your post is a sign that I must do that :)
Thank you for clear examples

Rytis Lukoševičius on 24/9/09

Good stuff Mark. :)

Tim Koschuetzki on 7/10/09

I wanted to submit the following modification. It’s a base class for shell tests. To use it, derive your test case from it and define the member variable $shell to be the name of the shell being tested. Since my shells often use tasks and tasks themselves look like shells this allows me to build tests without repeating the same base code from test to test:

@
<?php
/*! This is common code that sets up testing for the shell. This is shamelessly stolen from

http://mark-story.com/posts/view/unit-testing-cakephp-shells */ ?>

<?php

/*! This section of code obtains the dispatcher and creates a partial mock object from it to disconnect it from I/O to real devices: */ if (!defined(‘DISABLE_AUTO_DISPATCH’)) { define(‘DISABLE_AUTO_DISPATCH’, true); } if (!class_exists(‘ShellDispatcher’)) { ob_start(); // This will effectively disable $argv = false; require_once(CAKE.‘console’.DS.‘cake.php’); // output. ob_end_clean(); } if (!class_exists(‘MockShellDispatcher’)) { Mock::generatePartial( ‘ShellDispatcher’, ‘MockShellDispatcher’, array(‘getInput’, ‘stdout’, ‘stderr’, ‘_stop’, ‘_initEnvironment’) ); } /*! Actual tests should subclass this class: startTest and endTest should be chained to: */ class ShellTestCase extends CakeTestCase { /*! Start the test up by creating our partially mocked dispatcher, our partially mocked shell and connecting them together: */ function startTest() { // Create the mock shell here using $shell for the name and // generating a new class named “Mock”.$shell // Mock::generatePartial( $this->shell, ‘Mock’.$this->shell, array(‘in’, ‘hr’, ‘out’, ‘err’, ‘createFile’, ‘_stop’, ‘getControllerList’) ); $this->Dispatcher =& new MockShellDispatcher(); // // Making the target shell is a bit more interesting now // Construct a class name for the mock called // Mock.$this->shell use eval to build it and // assign it to $this->Task where it can then be used as the // real shell. // $command = ‘$this->Task = new Mock’.$this->shell.’();’; eval($command); // $this->Task =& new MockTargetShell(); $this->Task->Dispatch =& $this->Dispatcher; $this->Task->Dispatch->shellPaths = Configure::read(‘shellPaths’); // Now create and insert the models in the task: if(is_array($this->Task->uses)) { $models = $this->Task->uses; foreach($models as $model) { App::Import(‘Model’, $model); // Include the code. $modelObject = &ClassRegistry::Init($model); if ($modelObject null) { trigger_error("Failed to create object for $model", E_USER_ERROR); } $code = '$this->Task->'.$model. '= $modelObject;'; $result = eval($code); if($result = false) { trigger_error(“Could not store $model: $code”, E_USER_ERROR); } } } } /*! End the test by forcing the dispatcher and shell to be destroyed and flushing the cache of loaded classes so that next test will need to rebuild. */ function endTest() { unset($this->Task, $this->Dispatcher); ClassRegistry::flush(); } }

?>
@

Ron Fox on 28/12/09

hmm, that didn’t quite format usably. Apologies for using the space but I’ll give this one more shot:


<?php
/*! This is common code that sets up testing for the shell. This is shamelessly stolen from

http://mark-story.com/posts/view/unit-testing-cakephp-shells */ ?>

<?php

/*! This section of code obtains the dispatcher and creates a partial mock object from it to disconnect it from I/O to real devices: */ if (!defined(‘DISABLE_AUTO_DISPATCH’)) { define(‘DISABLE_AUTO_DISPATCH’, true); } if (!class_exists(‘ShellDispatcher’)) { ob_start(); // This will effectively disable $argv = false; require_once(CAKE.‘console’.DS.‘cake.php’); // output. ob_end_clean(); } if (!class_exists(‘MockShellDispatcher’)) { Mock::generatePartial( ‘ShellDispatcher’, ‘MockShellDispatcher’, array(‘getInput’, ‘stdout’, ‘stderr’, ‘_stop’, ‘_initEnvironment’) ); } /*! Actual tests should subclass this class: startTest and endTest should be chained to: */ class ShellTestCase extends CakeTestCase { /*! Start the test up by creating our partially mocked dispatcher, our partially mocked shell and connecting them together: */ function startTest() { // Create the mock shell here using $shell for the name and // generating a new class named “Mock”.$shell // Mock::generatePartial( $this->shell, ‘Mock’.$this->shell, array(‘in’, ‘hr’, ‘out’, ‘err’, ‘createFile’, ‘_stop’, ‘getControllerList’) ); $this->Dispatcher =& new MockShellDispatcher(); // // Making the target shell is a bit more interesting now // Construct a class name for the mock called // Mock.$this->shell use eval to build it and // assign it to $this->Task where it can then be used as the // real shell. // $command = ‘$this->Task = new Mock’.$this->shell.’();’; eval($command); // $this->Task =& new MockTargetShell(); $this->Task->Dispatch =& $this->Dispatcher; $this->Task->Dispatch->shellPaths = Configure::read(‘shellPaths’); // Now create and insert the models in the task: if(is_array($this->Task->uses)) { $models = $this->Task->uses; foreach($models as $model) { App::Import(‘Model’, $model); // Include the code. $modelObject = &ClassRegistry::Init($model); if ($modelObject null) { trigger_error("Failed to create object for $model", E_USER_ERROR); } $code = '$this->Task->'.$model. '= $modelObject;'; $result = eval($code); if($result = false) { trigger_error(“Could not store $model: $code”, E_USER_ERROR); } } } } /*! End the test by forcing the dispatcher and shell to be destroyed and flushing the cache of loaded classes so that next test will need to rebuild. */ function endTest() { unset($this->Task, $this->Dispatcher); ClassRegistry::flush(); } }

?>

Ron Fox on 28/12/09

Very good article. I will be experiencing a few of these
issues as well..

farewell to arms quotes 5 days, 11 hours ago

Have your say:

*
* You can use Textile markup, but be reasonable