Enter Custom Builds
After fielding a few package requests, I knew there had to be a better way. Stickler does all of its reviews inside docker containers with temporary filesystems. The solution seems simple enough. Before running each job find and install the additional packages required by the current repository. Then once the review is complete, docker could discard the container filesystem, leaving a pristine state for the next review.
This initially seemed simple enough. However, in addition to reviews, Stickler CI can also automatically apply any fixes from your linting tools. In order to avoid wasting a significant amount of time downloading and installing packages multiple times, I needed a way to share state between the fixer and review processes.
To solve this performance issue I chose to create temporary images. When a named container, docker will retain the container state. This state can then be turned into an image with
docker commit. With a custom per-job image, I would be able to offer custom builds without wasting CPU and network. I chose to start with
eslint as all of the additional package requests were for
Installing Additional Packages
Installing additional packages should be simple, but the real world is messy. As a GitHub App, Stickler is unable to access private NPM registries or pull dependencies from other git repositories. Furthermore, installing all of an application’s dependencies could be slow and expensive time wise. To workaround this only, the package installer searches for packages matching
eslint-(config|preset). Only packages matching this pattern are installed. Thankfully most ESLint configuration packages are open-source and small enough that they don’t take long to download.
The end flow roughly looks like:
- Create a named container that uses a tool specific install script.
- The install script uses regexp to find package names & versions.
- Each package is then installed into the image.
- The resulting container filesystem is committed into a new image.
- The fixer command is run in our new image, and any changed files are committed.
- The check command is run and errors are reported to GitHub.
- The image created in step 3 is deleted.
Once I had this flow implemented for
eslint I started beta testing it with customers. When a customer contacted me about additional
eslint packages, I would give them instruction on how to use the plugin installer. I used this beta group to gain confidence in the approach, monitor how custom images operated, and fix a lingering bug or two.
Once I was happy with the solution, I implemented a similar flow for the
flake8 plugin installer require users to provide a list of plugins they use. Having an explicit list requires a bit more work from the user, but ensure that reviews are always deterministic and work the first time.
I’m quite happy with the solution I arrived at. It has been operating quite well throughout its beta and public releases. In the future, I would like to improve a few things. First, would be to keep a pool of frequently used custom containers around to help further improve review times. I’d also like to expand the number of linters that include custom builds. Ruby and PHPCS are the next likely candidates.