Trunk Based — A How-To

Feature toggles

The main idea with TBD, is that instead of putting an alternative code path in a separate branch in your version control system (like git), you put that code path right into the master code where everybody can see it. And where everybody will run it.

A separate code path is, of course, done with a simple if-statement. The condition to test for, is called a feature toggle:

1if (Toggles.isEnabled(validatedUser, "Some Feature")) {
2   doTheNewStuff();
3} else {
4   doTheOldStuff();
5}

- It really is that simple

The toggle is there to protect the new code from interfering with the old code.

It isn’t only used to introduce a new feature. Quite often, you’ll introduce an alternative implementation and measure the impact in a real world environment.

Refactor, Commit and Push

One important benefit of a single code base with feature toggles, is that you can always push the code to master. In the example above, you can easily push the introduction of the method doTheOldStuff() to master. Next, you introduce the feature toggle. By having it default to false, it’s also safe to push. Nobody will run doTheNewStuff(), so you don’t actually have to implement anything before committing. This shows all the other coders what part of the code you are working on. If they need to modify doTheOldStuff(), they will quickly see your feature toggle and know that the two of you need to talk. If you manage to create a merge conflict here, the same thinking applies: The two of you need to talk. Luckily these merge conflicts will be really easy to sort out. Your five lines and my five lines. That’s easily managed.

Usually, the code you need to modify is located in multiple files in your application. That means a lot of checks for that same feature toggle. Instead of that mess, you might need to refactor the original code to minimize the number of needed checks.

Make the change easy, then make the easy change
- Kent Beck

Each refactoring should be committed and pushed to master. By definition, refactoring does not change the behavior of your application. As long as your test coverage is good, this will be a safe exercise.

Properties of a feature toggle

The Toggles class using properties of the validated user to determine if a feature is on or off. That means the feature can be enabled for individual users. In the beginning, only the developer will see this code running. Later, product owners and testers can be added to the list. Even further down the road, you can enable this for a department or office to enable piloting the new solution. At any point, you can bail out by switching the toggle off. In the code example mentioned above, I’ve added the user as a parameter. The Toggles class could easily pick the user object from a session store, or through some dependency injection framework. But the user object and the users roles must be available all the way into the leaf nodes of your call three.

When a feature is deemed done, it can be rolled out in waves. First 1% of the users, then 5%, then 20% and so on. If you notice problems, you can turn the feature off to run the old code. This requires quite a bit of monitoring of the state of our application. But I think your application should have that anyways.

A/B testing is also a use case. You turn the feature on for some users, and then monitor their behavior, the performance profile of the application or other metrics.

Where the toggles are stored, and how you configure them, is up to you. It might be a Virtual Machine (VM) or environment variable. That requires a restart when a feature is toggled, but it’s an easy implementation. In our app, we used the same REST-service for all our configurations, including feature toggles. Implementation was again very easy, but it made caching harder. It is also possible to combine these, turning the feature on on your development machine through a VM parameter, and then use a service if that parameter doesn’t exist.

There are companies selling this as a service as well. Launch Darkly and Split are two I know of.

Automation

To make this work effectively, we need to eliminate manual steps. There needs to be an automatic pipeline that kicks off on each commit, builds a new version of the software, runs all the automated tests in all the various environments and configurations necessary, and then finally deploys it to production. All automatically. If a test fails at any step, the pipeline aborts. This will leave the production code safe, but several test environments now has a known bug that must be fixed quickly. The easiest is to revert the last change or turning off the feature toggle. But sometimes it really is quicker to actually fix the bug.

Automatic deployment to production can be tricky, since you need a load balancer and the ability to upgrade each node separately and safely. HAProxy is made for this, as is NGINX. But the easiest way, is using a tool like Heroku. If you need more power, various cloud service providers’ implementations of Kubernetes or OpenShift might help you.

Benefits

The main benefit of TBD is that there’s one single version of the code at any point in time. The same binary is run in every test environment and then deployed to production. If you use GitFlow, you need to retest everything after each merge, since you now have a new code base. With TBD that is implicit, automatic and versioned.

Another benefit is small merge conflicts. Once you push changes of 5, 10 or 20 lines of code, it’s really easy to handle that conflict. Most of the time you won’t even get a conflict, since the developers will notice they are working on the same code before it becomes a problem.

Yet another benefit is that new features are deployed and enabled in production as they become ready. Once you merge a feature branch to the development branch in GitFlow, that feature must be included in the next release. The alternative is to manually remove it. With TBD, you just toggle that feature off, and you’re good.

A final benefit, is that you can enable a new feature at a given time. For example, a new tax regime should probably be enabled at midnight new years eve. With TBD you can simply add a timer that does that, no need for overtime! This might be a bit too risky, but the point is that the actual production upgrade is simply modifying a boolean in a data store somewhere. All the code is already deployed.

Conclusion

TBD sounds like a rather crazy idea, but it is actually rather simple once you start doing it. The main idea is to keep the code as code with configured feature toggles, instead of hard wiring those toggles as branches in an external system (your code repository). This means testers, business people and early adopters can see the working code early by turning on the new toggles. New features can then be made available to users once the features are done, instead of waiting for the next bi-monthly release (or whatever your current release plan stipulates).

| All in all, Trunk Based Development reduces wait times and increases flow.