It was about four years ago I heard the word “trunk” for the first time in a software development context. I was talking to one of the engineers who managed the release of OpenStack and he was talking about trunk being tested three times a day and companies tapping directly of trunk. I’m not sure I really understood what he was saying as this is miles away from our traditional release management. Four years later I still remember this conversation and believe we all should look at a trunk based development approach. As our enterprises evolve to an increasingly digital world, as our business colleagues come up with new business models and approaches at a faster pace, it is becoming clear that our time consuming release process no longer works.
But what is Trunk based developments?
In a detailed article, Paul Hammant explains trunk based developments. In a nutshell, every time a developer has finished a piece of code adding some functionality to an application, and has tested the functionality works within the simulated environment he uses, he commits that code to trunk. This means that his functionality is now part of the trunk version of the application. Regularly, or event driven (new commits), this application goes through rigorous tests. If trunk does not pass the tests, commits are stopped and the problem is resolved with all hands on deck. The objective is to always have deployable code. This does not mean it is deployed. That is an important nuance.
At regular intervals releases take place. This means that a release engineer takes a copy of trunk, potentially cherry-picking amongst the new functionality and prepares the actual release of that functionality in production. The advantage of such approach is the speed at which functionality can be added to the production environment and the fact errors are found earlier in the process. Teams do not wait for months before integration tests take place.
However, such approach has some prerequisites. Let’s discuss these in more details.
1. Functionality is developed in small modules
Each time a developer commits some functionality that one has to be operational within the wider application. This means it needs to be a nicely finished piece of code performing a function that adds value to the enterprise. As commits are expected daily or every couple days, this means the functionality is divided in small self-sustaining chunks. This leads to using service oriented architecture principles, more loosely coupled functionality, clearly defined APIs etc. It brings with it a different application architecture. In most existing enterprises, many developers work on existing applications and evolve them. But those applications often are big monoliths and do not follow the architectural principles I highlighted. Let’s come back to this later in this article.
2. Many tests are automated.
Testing is at the center of this approach. Unit tests are performed by the developer, using studs for simulating functionality outside his control, databases, integration links etc. As he commits regularly, he will want to run most tests automatically so he remains focused on the software development process. By automating these tests, he is able to test the functionality very quickly and spot problems. He can correct issues immediately.
In a traditional approach, developers work on a given module for long periods of time and then hand the module over to be tested by a separate team. They continue with the next job. It’s only much later that they will be told what results the tests provided and potentially asked to correct the code. They will have to re-invest in understanding what they had done previously so they can address the issue. A lot of time and effort is lost in such approach. By being able, while going for a coffee, to have your code tested, helps you stay focused on one task and bring it to fruition.
3. Test in production like environments
How often have releases derailed because, although applications ran perfectly in test environments, something went wrong in production? The reason? The use of different software components, middleware, configurations etc. So, why not do integration testing in a production like environment. Easier said than done. Operations people keep evolving production environments regularly. Changes in configurations, patches etc. are common. To keep test and production systems in synch, a strict version control is required. Every time a change is made in production, the same change has to be made in the test environments. So, imagine having a test environment that is a scaled down version of your production environment, and have all systems and SIT tests performed on such system? Automate as many tests as possible and run them regularly. That is the idea behind the daily test of trunk. Sure, you will need to be able to run performance tests for which you need a copy of your production environment. Why not provision such environment in a public cloud for the duration of the test? You won’t need the equipment all the time. Some functional tests and in particular user tests have to be done manually. Having trunk and an always deployable application enables testers to run the user tests at any time. They can do this in parallel with the automated tests.
What about existing applications?
In existing enterprises, most developers are not working on new applications. They are rather expanding and improving existing ones. And in the majority of cases, these applications are monoliths. So, the above approach is not usable. So, what do we do?
It’s correct you cannot apply a trunk based development approach to a monolith, but it may be the opportunity to transform your application. As your company evolves to the digital enterprise, you will need the capability to invoke specific functionality when digitizing your business processes. Fundamentally you have two ways of doing this:
- Isolate the existing functionality and making it callable through one or a series of APIs (Re-Interface). All new functionality is developed in a modular (SOA compatible) fashion and uses the APIs to call upon the existing one. If existing functionality needs to be enhanced or corrected, it is actually rewritten as a module and removed from the original monolith. In doing this, over time, you empty the monolith from its content.
- Re-factor the application by dividing the code in loosely coupled modules. Martin Fowler has many hints on Re-factoring that are worth reading. Re-factor breaks your monolith up, changes the architecture and enables you to use the created modules in support of digitally enabled business processes.
The advantage of the second approach is that it gets you to trunk based developments faster as you address the existing functionality immediately. However, when doing that, you do not progress the actual functionality you present to your user. In their book “The DevOps Handbook”, Gene Kim and Co. describe how Amazon, PayPal and others have rewritten their main applications several times to be able to continue evolve their product and service offerings. Most of them have evolved their application implementation by using microservices. Their applications are the pillar of their business. This may not be your case, so ask yourself how far and how quickly you need to move. But keep in mind the need to evolve to a digital enterprise and the importance of your applications in that approach. You may be interested in reading the posts I wrote about how to analyze your portfolio with the digital enterprise in mind: