Three plus years ago as we were starting to contemplate the move toward Continuous Delivery and speeding up our velocity of value delivery to the business, we immediately identified branching as one of the blockers. We were the traditional enterprise Microsoft development shop. Team Foundation Server 2008 with many long-lived branches, typically Main, DEV1 and DEV2 at a minimum plus there were POC, patch and developer-specific branches. It didn’t matter how small the changes were in these branches, the merges were always difficult. We started adding time into our software development lifecycle for merging which could be one to five days depending on how old/big the branch was.
Beyond the additional merge time, these long lived branches made it much more difficult to manage deployments. We needed different deployment pipelines for each branch or a more complex generic pipeline that could choose a branch. Then it was hard to tell what branch had been deployed where.
Ultimately a few of us decided these branches were hurting more than helping us. They were artificial safe guards that made us feel more comfortable with many people working on a single solution but we still had merge conflicts and stomped code. The main issue is this process defers source integration between developers until the end of feature development or a project.
Bring Pain Forward
One of the core tenants of Continuous Delivery is to bring pain forward. If something is painful, most of us will instinctively avoid that pain and defer until absolutely necessary. If we are practicing continuous improvement we actually want to bring the pain forward and do it more often so we can learn from the pain to make it better next time. So instead of long-lived branches that we only merge once a month/quarter, we should merge everyday into a single main branch with all the other developers.
This was not a popular idea at the time. We had been using this process for a decade or longer and most developers were fearful of the change. After much angst, we decided to try it for a month as an experiment and see how it worked out. Release Readiness was also a big part of this same conversation and in the end part of our success.
Three plus years later we have 120 agile team members all working with their teams in a single main branch. They do at times use short-lived feature branches but this is by far the exception. The teams use unit tests, Release Readiness, gated check-ins and continuous integration to avoid breaking the build and blocking their teammates.
It hasn’t always been perfect but overall it has been very positive experience. We have even had a team member go back to try a long lived branch for a big feature, in the end he found how terrible the merge was and vowed to never do it again.
What has your experience been with long-lived branches? Leave your comments below and continue the conversation.
Great experience! How do your team handle if a feature needs to be removed from the release(main branch) at last minute?
We can turn off/on the feature flag anytime we want. We typically remove feature flags 1-2 iterations later when we are sure we don’t need it anymore. Does that answer your question?
How do you manage working on multiple releases on the same branch then? Say a bunch of people are working on v1.0 but other on v2.0, each with their own release cycle?
We use release readiness for this – https://www.dotnetcatch.com/2016/02/16/are-you-release-ready/
Basically, the developers assume anything they checkin will go to production immediately so they must have a way to hide/disable the new code to prevent blocking a release. This is much easier if you are making smaller incremental changes and not large site wide refactorings. We have been using this process for 3+ years with 8-10 teams of 8-10 people and it has worked well for us.
Pingback: Los sistemas tienen vida y debemos respetarla :) | Jersson on the block!