Calculating Documentation Coverage with ApiGenerator

Generating code coverage for test cases is a handy feature, it gives you a quick and easy way to determine how much of your code is running during your tests. It doesn’t ensure that the tests are good or that you have enough assertions, but code that doesn’t run definitely has not been tested. Before code coverage was created it was very difficult to determine how much code was being run. Documentation is another area where knowing what you are missing is really helpful. By being able to quickly identify areas where your API is missing documentation you can easily identify which areas you need to focus on.

Documentation coverage?

Since the bad documentation drum gets beaten a fair bit for open source projects, I made a tool that will analyze doc blocks and use a set of rules to evaluate your documentation based on several criteria. Documentation criteria are pluggable and its quite easy to create your own. Included by default are 4 rules.

• Missing Link This is one we use in CakePHP to ensure that all properties and methods have a page in book.cakephp.org.
• Missing Params This rule looks at all the param entries and ensures that they all have a type and description.
• Empty This rule checks that there is a method string, that provides a description of the method.
• Incomplete Tags Checks all the tags and makes sure that all tags have a description.

Adding a new rule is pretty simple too. Make a class that extends DocBlockRule and when you create a new DocBlockAnalyzer define the rules array. For example $Analyer = new DocBlockAnalyzer(array('Custom', 'Empty')) would include the Custom and Empty rules.

Generating documentation reports

Generating documentation reports is pretty easy. If you setup ApiGenerator towards the end of the install you will be asked to enter usernames and passwords. These username/password combos can be used to access the documentation coverage features. If you already have an active install of ApiGenerator, you will need to update the plugin, re-install the database and regenerate the index. Once you have that done from your console run cake api_index users. From there you can view the existing users as well as create new ones.

Once you have some users in your config file, you can go to /admin/api_generator/class_index This gives you paginated list of the classes in your index, and starts generating coverage reports via ajax. An individual report looks like

You get a big bar telling you the final percentage as well as a detailed report from each method/property that had failing rules. The detailed list gives you a very good idea of what you are missing and what you need to do next in order to bring your documentation coverage up.

Well I hope you find this useful and if you have any ideas for how to improve it definitely let me know. You can find the code on thechaw.