Using ApiGenerator to generate documentation

The release of the new API at http://api.cakephp.org wasn’t quite as smooth as I would have liked it. However, since the initial release things have constantly been improving. The search is now much more effective, and global functions are now in the index. So things are looking better each day. I hope to answer a few questions surrounding ApiGenerator today.

Why did you change it if it wasn’t broken?

This is a question that I’ve been asked a few times in the past few weeks. To be honest the old API was horribly broken. Entire classes were missing. File, and CakeSession for example were simply omitted because doxygen doesn’t properly read conditionally included files. Even classes that did exist in the API were incomplete. Model for example had no documentation for saveAll(). We were also somewhat limited in what formatting could be generated. We did look at other options like PHPDoc however the lack of a search was a deal breaker, as that seems to be one of the most often used features in the API. So the API was broken, and it needed to be fixed, and we needed a solution that worked for our needs.

Why reflection?

As I mentioned last time ApiGenerator does very little file parsing, and instead uses Reflection to generate documentation. This was another design decision. We needed to get the new API together pretty quickly. This made a file parsing strategy unfeasible. I don’t know if you’ve ever thought about how complicated the regular expressions and text processing needs to be, but I have, and it hurt my brain. PHPDoc has 20+ classes just for file parsing. Thankfully PHP has a pretty good Reflection API that affords doc block extraction. Reflection can be used to do lots of other things as well but I was most interested in documentation aspect of it. Furthermore, this all meant that I could cobble together something working in much, much less time. While reflection did give a whole bunch of advantages it has one big draw back. The code is live. In order to parse docs you need to be able to load the files. This also means that you can run into namespacing conflicts. Lastly, you can’t generate docs for old versions of the classes powering the doc extraction.

So how do I use this thing?

Well first there are a few requirements. You must have:

  • PHP 5.2.3 or better. There are parts of the reflection API that didn’t exist before then.
  • A database with room for 1 table. – Yes there is only one table, but its super important.
  • A command line. You need access to a shell, no shell, no API. End of story.

So if you meet the requirements we can get started. As stated in the requirements, you will need to know how to use the cake console. As all the API generation / maintainance / config is all done there.

Installation.

Download / git clone the project into app/plugins/api_generator from http://github.com/cakephp/api_generator. It must go here, sprinkling the files throughout your app, or calling the directory teh_api will not work. Next open up your console and run

cake api_index initdb

This will install the database table. You will be asked a few questions about the schema you are installing, so say yes to dropping and adding the tables. Next up is getting a configuration file.

Configuration

ApiGenerator uses an INI file for its configuration. This file can be written by hand or generated with the console. To generate the configuration run:

cake api_index showfiles

The shell will notice that you don’t have a config file and prompt you to make one.

Show Plain Text
  1. ---------------------------------------------------------------
  2. api_config.ini could not be located.
  3. Answer some questions to build it.
  4. ---------------------------------------------------------------
  5. Enter the path to the codebase.  
  6. [/Users/markstory/Sites/cake_api_gen] >

Is the first prompt, your path will look different, but enter the path to the code base to be documented. It will ask about multiple paths, but they are not fully implemented at this point in time.

Show Plain Text
  1. ---------------------------------------------------------------
  2. Setup some excludes
  3. excludes remove files, folders, properties and methods from the index.
  4. Input a comma separated list for multiple options
  5. to continue, just answer "n"
  6. ---------------------------------------------------------------
  7. Exclude properties of the following types (private, protected, static)  
  8. [private] >
  9. Exclude methods of the following types (private, protected, static)  
  10. [private] >

Next up you have to setup exclusions. Exclusions tell ApiGenerator things it should keep hidden. You can set exclusions based on the visibility of methods & properties.

Show Plain Text
  1. Comma separated list of directories to exclude  
  2. [n] >

Next you should include a list of directory names that are not to be looked in. I use tests, locale, config, webroot, tmp for my apps. This is just a list of directory names and not path fragments.

Show Plain Text
  1. Comma separated list of files to exclude  
  2. [n] >

Next is a list of file names to exclude. These files will not be included in the documentation. I use index.php, empty in my app.

Show Plain Text
  1. ---------------------------------------------------------------
  2. About the files in your codebase
  3. input a comma separated list for multiple options
  4. to continue, just answer "n"
  5. ---------------------------------------------------------------
  6. Extensions to parse (php, ctp, tpl)  
  7. [php] >
  8. Regex for matching files  
  9. [[a-z_\-0-9]+] >

Extensions of files to look at, and file name pattern to use. Pretty simple. The defaults work pretty well.

Show Plain Text
  1. ---------------------------------------------------------------
  2. Do you have some classes that do not map to a filename?
  3. to continue, just answer "n"
  4. ---------------------------------------------------------------
  5. Class to map  
  6. [n] >

This is the most complicated part of the configuration. Because ApiGenerator relies on reflection, it needs all parent classes loaded for before it can load and introspect on a class. Its pretty good at finding things once it has an index built but in order to get there, you need to map out some files that it has trouble finding. If you miss a file its ok, it will skip files it knows it doesn’t have all the parents for. You answer the above question with a class name, for example XmlHelper.

Show Plain Text
  1. Enter the path to the file that holds XmlHelper. this can be relative to the default path,
  2. or add a / in front to use an absolute path  
  3. [/Users/markstory/Sites/cake_api_gen] >

Is the next question. You now need to provide a file path to the mapped class. It can be relative to the codebase path, or an absolute path. You can add as many mappings as you wish at this time.

There you go, you now have a complete config file. You might need to tweak it as you generate the index.

Generating your first index.

The index in ApiGenerator, holds information about the methods, properties and locations of classes and functions. You can preview an index generation with

Show Plain Text
  1. cake api_index showfiles

This will show the files that your index will contain based on the exclusions. Once you are satisfied with the list of files being included you can generate the index with

Show Plain Text
  1. cake api_index update

This will clear any existing index and repopulate the index table. Watch for errors saying files couldn’t be found with the mappings, and then add these files to your config’s mappings. Once you have a complete index, you can start exploring and viewing your API. That’s all there really is to it at this time. If you are interested in helping improve ApiGenerator, please open tickets and submit patches. I hope you find this tool useful in documenting your applications.

Comments

It’s insane that PHPDoc doesn’t have search. I remember using the API for ZF once (they use PHPDoc) and hating every minute of it.

Awesome job!

Matt Curry on 2/12/09

Hey Mark,
Regarding the showdown.js/short open tag thing. I found the easiest way around this is to drop it in /app/webroot/api_generator/js. This way it loads right through apache without going to cake.

Also doesn’t require any code changes to cake or api generator.

Might be worth mentioning in README.

Matt Curry on 2/13/09

Ugh…nevermind the above doesn’t work all that well. The JS will load, but you can’t get to /api_generator anymore. Have to go to /api_generator/index. Not that great of a solution.

Matt Curry on 2/13/09

I’m having a problem installing.
After:
cake api_index initdb
when I run:
cake api_index initdb
I get this error
Error: Missing database table ‘api_files’ for model ‘ApiFile’

Filipe on 3/31/09

i got the same error Filipe, to fix the this, put the project inside the app directory. this way:

myproject/app/cakefiles

Micahel F. on 4/25/09

I’ve installed ApiGenerator a number of times so far, and never run into this issue. It sounds like the model isn’t being correctly located and an auto-model is being used, as ApiFile does not even extend Model.

mark story on 4/26/09

fatal: http://thechaw.com/api_generator/info/refs not found: did you run git update-server-info on the server?

Spencer Westwood on 5/3/09

@Spencer Westwood I don’t run thechaw.com. So I’m not sure what has been run on the server. But I haven’t had an issue using thechaw. I have Git 1.6.2

mark story on 5/3/09

cake api_index initdb show me the help file of SchemaShell, why ?

Philippe David on 5/6/10

Is CakePHP 1.3 supported. Also, I generated the files but I don’t know where to go to in the browser to view the API. I don’t think that was clear in the documentation either.

Jason on 5/18/10

I’m also getting the help file of SchemaShell when I try to run cake api_index initdb. Is there anything I should be looking for or anything that I’m doing wrong?

Courtney on 9/17/10

Where is the step by step guide of installation, from downloading, make the project folder, etc to runing, or just simple make a video, this api_generator is so confusing

eduardo Fernandez on 10/15/10

Jason: 1.3 is supported by the master branch in the repo on github.
Eduardo: The docs are what they are, unfortunately I don’t have time to create a video of how to install the plugin. Its mostly just a matter of getting the schema in place and building an index most times.

mark story on 10/16/10

Hey Mark

Maybe you should point out a bit more specific how inheriting classes should be nested in files and folders, e.g. that

- class A is in (e.g.) libs/a.php – class B extends A is in libs/a/b.php

etc. I guess for most people this is absolutely clear, but for the few of us who don’t know such conventions too well it would be very helpful.

Great tool, anyway! Thanks! :-)

Joshua on 1/18/11

I am trying to run the initdb command but all I get is the help message for “cake schema”. Any ideas?

Weydson on 8/9/11

Is this usable with CakePHP 3.x?

Mike on 2/6/15

Comments are not open at this time.