Using bitmasks to indicate status

At some point or another we’ve all had to make a data model that involved various flags to indicate different statuses / modes for an object. Often the schema for such a data model may end up looking like

Show Plain Text
  1. CREATE TABLE nodes (
  2.     id INT(11) NOT NULL PRIMARY KEY AUTO_INCREMENT,
  3.     name VARCHAR(100),
  4.     description TEXT,
  5.     published BOOLEAN DEFAULT '0',
  6.     needs_review BOOLEAN DEFAULT '0',
  7.     comments_allowed BOOLEAN DEFAULT '0',
  8.     promoted BOOLEAN DEFAULT '0',
  9.     created DATETIME,
  10.     modified DATETIME
  11. );

This makes logical sense and translates nicely into forms. However, I always felt that it was kind of inefficient to do things this way. As you often end up having to type long comparisons out when checking more than one of these flags. To make comparisons easier you can either write some methods that handle the flags or you can do what I’ve been doing lately and lump them into one field.

Enter the status

In order to lump them together we need use bitmasks and bitwise operators to save space and time. While bitmasks and bitwise operators are nothing new to programmers with a computer science background, those of us coming from design backgrounds they can be a bit daunting. Since I fall into the latter, I thought I would share what I’ve learned. First thing we would need to do is merge all the flag fields into a single status field.

Show Plain Text
  1. CREATE TABLE nodes (
  2.     id INT(11) NOT NULL PRIMARY KEY AUTO_INCREMENT,
  3.     name VARCHAR(100),
  4.     description TEXT,
  5.     STATUS INT(3),
  6.     created DATETIME,
  7.     modified DATETIME
  8. );

I make this an INT(3) field as I’ve yet to run into a situation where I needed a longer value. Next up is wrapping you head around binary numbers & math, and bitwise operators. Binary expresses values as a series of 0’s and 1’s.

Show Plain Text
  1. 00000 = 0
  2. 00001 = 1
  3. 00010 = 2
  4. 00100 = 4
  5. 01000 = 8
  6. 10000 = 16

Each column is referred to as a bit, and as you move left each columns value grows in integer value by 2. It is sometimes easier to think of them as analogous to your old columns, and forget that they are also integers. In our example class we will map each status column in the old table definition to a bit column. Ending up with something like:

published needs_review comments_allowed promoted
0001 0010 0100 1000

With each flag in a separate bit column we can use Bitwise operators to add subtract and combine the different flags. Bitwise operators work on the bits in a value rather than the value itself. The most common operators I use are |, ~ and &. These operators perform your basic addition, subtraction and masking operations. I normally set up each bit status flag as a class constant. Something like:

Show Plain Text
  1. class Node extends AppModel {
  2.  
  3.     const PUBLISHED = 1;
  4.     const NEEDS_REVIEW = 2;
  5.     const COMMENTS_ALLOWED = 4;
  6.     const PROMOTED = 8;
  7. }

I find using class constants makes the most semantic sense and stops me from accidentally changing the values. As shown above, each one of the chosen values represents a single 1 in a different column. It is important to do this, as it makes all the following operations work.

Using bitmasks and bitwise operators on your status field.

Now that we have our different statuses set up, we can start doing some bitwise math with them. You can add different status flags together with the | operator.

Show Plain Text
  1. $publishedAndCommentsAllowed = Node::PUBLISHED | Node::COMMENTS_ALLOWED; //value = 5
  2.  
  3. $commentsAllowedPublishedPromoted = Node::PUBLISHED | Node::COMMENTS_ALLOWED | Node::PROMOTED; //value = 13

In the above examples the binary equivalent of $publishedAndCommentsAllowed is 0101. As you can see two of our columns have ones or are ‘checked’. You can subtract status flags using & ~. Expanding on the examples from above:

Show Plain Text
  1. $published = $publishedAndCommentsAllowed & ~Node::COMMENTS_ALLOWED; //value = 1
  2.  
  3. $published = $commentsAllowedPublishedPromoted & ~Node::COMMENTS_ALLOWED & ~Node::PROMOTED;

Checking status flags with bitmasks

Now that we can change the value of our status field we need to be able to check it for specific bits. We do this with bitmasking and the & operator. If you & two values together only bits (columns) present in both values will be part of the result. So we do checks like so.

Show Plain Text
  1. $status = 5; // (0101) published and comments_allowed set.
  2. if ($status & Node::PUBLISHED) {
  3.     echo 'published';
  4. }
  5.  
  6. $status = 13; // (1101) published, promoted and comments_allowed set.
  7. if ($status & Node::PROMOTED) {
  8.     echo 'promoted';
  9. }

This is all and well, but how do I do database finds it all my status columns are mashed together? Well you can use bitwise in MySQL and other RDMS as well. SELECT * FROM nodes WHERE status & 1 = 1 would select all the published articles. You can express this in CakePHP model find() as

Show Plain Text
  1. $conditions = array(
  2.     '(Node.status & ? = ?)' => array(Node::PUBLISHED, Node::PUBLISHED),
  3. );

So there you have it. The next time you face a pile of flags, you can use bitwise operators to combine them, and hopefully make your life simpler.

Comments

Very cool, I definitively see myself using this.

Thanks

anonymous user on 10/26/08

Wow, what a refreshing read. Thanks. The resulting code is very elegant and readable.

anonymous user on 10/26/08

Wow this is impressive, it is nice to go through your site as there is always something new and cool you are popping on here.

anonymous user on 10/28/08

Jason: Thanks :) I try to make regular posts on what I’ve found glad you find them helpful.

mark story on 10/29/08

Cool stuff… very cool!
Thanks for sharing.

anonymous user on 10/29/08

Thank you! I have definitely from this. This is a great method.

anonymous user on 10/30/08

cool stuff! ive been using it a long time now… but instaed of | and & ~ you could just use + and – …. its nice to read since actually you are adding and subtracting the flags

anonymous user on 11/8/08

skiedr: How would that change Marks code above? Would it make the last select condition prettier?

anonymous user on 11/14/08

A really elegant solution, thanks! Can i assume that the addition of a new flag is as simple as adding the a new const to the Model. For your example const ARCHIVED = 16;

anonymous user on 11/19/08

I’ve found this article so useful over the past few months. One little tip I have regarding it is that if you’re using a multiple select box in CakePHP, you can loop through the selected values and add the bitmasks together using the += operator:

$bitmask = 0;
foreach($this->data['Model']['bitmask_field'] as $b) {
$bitmask += $b;
}
$this->data['Model']['bitmask_field'] = $bitmask;

Rich on 9/15/10

Awesome stuff and clear explanation. Thanks much for sharing with the world. All the best.

Claudio on 2/17/11

a word of note: instead of using += to add together bits, you really should use |= (which will add them bitwise).

Joshua McNeese on 4/12/11

This is a really brilliant explanation. It’s really helped, cheers!

Mike on 8/2/11

I happened to run into the same issue for a legacy dropdown field which was upgraded to a multiple select.
So I ended up doing the exact same thing – and ran into your code after that :)
I was wondering if anyone tried to form a nice little behavior for this. this could take care of validation and basic operations as well as transforming into an array from db and into the bitmask to db.

mark on 2/22/12

mark, as a matter of fact… https://github.com/jmcneese/bitmasked

Joshua McNeese on 2/22/12

I already stumbled upon that one.
But I think that’s actually a different approach using a second table (instead of sticking to a field in the working table).

mark on 2/22/12

I gave a behavioral approach a try the last couple of days:
http://www.dereuromark.de/2012/02/26/bitmasked-using-bitmasks-in-cakephp/
let me know what you think.

The main difference between Joshua’s and mine is that it works with the same table.

mark on 2/26/12

Comments are not open at this time.