31

I have a client who insisted that we keep our new development separate from the main branches for the entirety of 2016. They had 3-4 other teams working on the application in various capacities. Numerous large changes have been made (switching how dependency injection is done, cleaning up code with ReSharper, etc). It has now fallen on me to merge main into our new dev branch to prepare to push our changes up the chain.

On my initial merge pull, TFS reported ~6500 files with conflict resolution. Some of these will be easy, but some of them will be much more difficult (specifically some of the javascript, api controllers, and services supporting these controllers).

Is there an approach I can take that will make this easier for me?

To clarify, I expressed much concern with this approach multiple times along the way. The client was and is aware of the difficulties with this. Because they chose to short on QA staff (1 tester for 4 devs, no automated testing, little regression testing), they insisted that we keep our branch isolated from the changes in the main branch under the pretense that this would reduce the need for our tester to know about changes being made elsewhere.

One of the bigger issues here is an upgrade to the angular version and some of the other third party softwares --unfortunately we have no come up with a good way to build this solution until all the pieces are put back into place.

Robert Harvey
  • 198,589
  • 55
  • 464
  • 673
user258451
  • 421
  • 3
  • 5
  • 63
    Nope. You have two separate branches that were actively developed for a year. The merge is going to suck. – 17 of 26 Jan 03 '17 at 17:46
  • 2
    What is the extent of the changes that your team made? It may be more efficient to identify those changes and then manually reapply them to the current master codebase. – kdgregory Jan 03 '17 at 17:54
  • 2
    Is it the entirety of 2015 or 2016? If its 2015 its 2 years, which means it will double suck. – David says Reinstate Monica Jan 03 '17 at 22:53
  • 1
    Why does the client care if the work you're doing for them is on a separate branch in your version control system? – Ixrec Jan 04 '17 at 00:00
  • Good luck. I once had to do that on svn. The employer used SVN. I just used git for the entire process, then commited to another merged svn branch. Then we merged the two. Lot of work, but it worked – George Silva Jan 04 '17 at 01:03
  • If you were using Git, I would recommend [git-imerge](https://github.com/mhagger/git-imerge). I've successfully used it for a fairly similar case of long-diverged development. It may even be sensible to do a conversion/import from relevant branches in TFS to Git, use imerge to generate the desired result, and then export that back. – Phil Miller Jan 04 '17 at 03:43
  • @DavidGrinberg mistype, new year problems :) – user258451 Jan 04 '17 at 17:18
  • 17
    Whatever you do, make sure you're billing hourly. – Sean McSomething Jan 04 '17 at 17:18

4 Answers4

37

There would have been a simple way which had kept your new development separate from the main branch without bringing you into this unfortunate situation: any change from the trunk should have been merged into your dev branch on a daily basis. (Was your client really so shortsighted that he could not anticipate that your branch needs to be remerged back into the main line some day?)

Anyway, best approach is IMHO trying to redo what should have happened on first hand:

  • identify the semantics of the changes on the main line for day 1 after the branch was created. Apply them to your current code base as well as you can. If it was a "local change", it should be simple, if it was a "cross cutting refactoring" like renaming a widely used class, apply it in a semantically equivalent manner to your current code base. Hopefully during that year no contradictory cross-cutting changes in the code base were made on "your" branch, otherwise this can become a real brain-teaser
  • test the result (did I mention you need a good test suite for this task)? Fix all bugs revealed by the test
  • now repeat this process for the changes on the main line for day 2, then day 3, and so on.

This might work when the teams strictly obeyed to the classic rules of version control ("only commit compilable, tested states" and "check in early and often").

After 365 repetitions (or 250, if you are lucky and you can bundle the work for weekend changes), you will be almost finished (almost, because you need to add the number of changes which will happen to the main line during the integration period). The final step will be to merge the updated dev branch into the trunk again (so you don't loose the trunk's history). This should be easy, because technically it should be only a replacement of the affected files.

And yes, I am serious, there is probably no shortcut to this. It might turn out that "daily portions" might be sometimes too small, but I would not expect this, I guess it is more likely daily portions can turn out beeing too big. I hope your client pays you really well for this, and that this is so expensive for him that he will learn from his failure.

I should add that you can try this also with switched sides - reintegrating the changes from your branch in small portions to the main line. This might be simpler when on your dev branch there were much fewer changes than on the trunk, or most of the changes happened in new source files which are currently not part of the trunk. One can see this as "porting" a feature from a product A (the dev branch) to a somewhat different product B (current state of the trunk). But if the majority of cross-cutting refactorings were done on the main line, and they affect your new code (the 6500 merge collisions seem to be some evidence for this), it might be easier the way I described it first.

Doc Brown
  • 199,015
  • 33
  • 367
  • 565
  • 9
    If you're continuing development on the trunk during re-integration I suggest you branch the trunk tip first and develop from that. Otherwise you are effectively merging the past and the future simultaneously. But I defer to Doc Brown on any space-time discontinuities, naturally. – radarbob Jan 04 '17 at 00:56
  • 1
    @radarbob: what you suggest makes only sense if the OP integrates from dev to trunk, but not when he decides to merge from trunk to dev, as I described it first. If the OP transfers changes from trunk to his dev branch day-by-day (starting with changes 365 days in the past), it won't matter if development on the trunk goes on. He will just have to continue this tactics until he reaches the present (assumed he can integrate & test the changes of those 3-4 teams on one day in less than one day). – Doc Brown Jan 04 '17 at 06:47
  • Quote: *"I should add that you can try this also with switched sides - reintegrating the changes from your branch in daily bundles to the main line.*" My spidey sense is blowing a fuse. I sense a potential cascade of "harmonic resonance distortion" with conflicting change integrations. – radarbob Jan 04 '17 at 13:52
  • This is good advice. See edit to address a couple of things here. – user258451 Jan 04 '17 at 17:20
  • @radarbob: for example, if in the dev branch the only change in that year is one added module with one public function which is called from exactly one entry point in the existing code base, it would obviously make no sense to integrate the trunk into the dev branch - the sensible approach for this case is to integrate the change from the dev branch into the trunk directly (or into the branch from the trunk tip you suggested). But since the OP mentioned 6500 merge conflicts, I think it is likely not his situation. – Doc Brown Jan 04 '17 at 17:29
  • @user258451: *"they insisted that we keep our branch isolated from the changes in the main branch"* - after your edit, I still don't see any reason why this isolation has to be bidirectional. If your team would have integrated changes from the trunk on a regular basis into the dev branch (but not vice versa, of course), it would not have affected the trunk or any QA of the trunk. – Doc Brown Jan 04 '17 at 17:38
  • @DocBrown it would have affected QA that was testing only our changes-- "why is this button different, there was no user story to update this part of the application". Or am I misunderstanding? – user258451 Jan 04 '17 at 17:44
  • 1
    @user258451: so you had different QA teams for the trunk and the new dev branch in place who did not want to talk to each other? Great Scott :-(( – Doc Brown Jan 04 '17 at 17:49
  • @DocBrown More like a total lack of QA teams – user258451 Jan 04 '17 at 17:55
  • @user258451: now your QA team will have to test a merge induced by one year of changes all at once. I don't think this will become cheaper overall than if they had tested the merge results over the year in smaller portions. – Doc Brown Jan 04 '17 at 17:58
  • @DocBrown A symptom of much deeper issues with how the client chooses to run their operation than specific to IT – user258451 Jan 04 '17 at 18:02
  • @DocBrown, I definitely endorse the advice for one-way, step-wise re-integration. But if I'm "cross forward integrating" 2 active branches without benefit of a true "fixed target" trunk, then I get nervous when I imagine unwinding that (i.e. reverting to some arbitrary revision). – radarbob Jan 04 '17 at 19:03
  • @radarbob: in my first suggestion there *is* a fixed target trunk, I assume the "new dev branch" is not actively evolved any more until the integration work is done. – Doc Brown Jan 04 '17 at 23:31
14

At that stage of merge I would say that automated merging may only over complicate the process. I have had similar issues with branches that have diverged for over a year and the most effective method I have is to do the following:

  • Take a copy of the original unmerged state
  • Diff between the unmerged and latest
  • Break down any common elements
    • For example, do all function name changes, then parameter changes, etc.
    • Ignore white space on diff if standards have changed, otherwise you will waste a lot of time counting spaces
  • Focus on the core functionality first

Eventually compiler warnings and the diffs will be your best friends, keep using the unmerged diff to see exactly what was different and just keep going. There might be various tools you could use to help, but that will be up to you to find which is best.

The key is to keep going.

Edit:

A word of warning, this approach will mean that version control history would become 'corrupted', as you'd lose the evidence of the branch-to-branch merge, as well as the unmerged branch's history.

Thanks to comments from 'Jack Aidley' and '17 of 26'

Erdrik Ironrose
  • 4,806
  • 3
  • 13
  • 25
  • 1
    The major problem with this approach is that it will destroy the record of changes left in the version control system. – Jack Aidley Jan 04 '17 at 12:21
  • Essentially by making all of the same changes a second time during the merge, you would still have a log of the changes, it would just be in the order done in the merge instead of the order they were done during development. Not ideal, but better than nothing. Plus you'd still have the original unmerged state, if that was kept in version control. – Erdrik Ironrose Jan 04 '17 at 13:38
  • You would have a log of the changes, but you would not have historical evidence that the changes were merged from branch to branch. This would be a huge deal in TFS, which offers you only the unmerged changesets to choose from when merging from branch to branch. – 17 of 26 Jan 04 '17 at 13:58
  • @17of26 While I agree that you can restore some record of changes, 17 of 26 is correct that this record will not be complete or accurate. It may be that this approach is sufficiently easier as to make this loss of record an acceptable compromise given the bad situation they're now in. However, I think it's an important drawback to recognise regardless of what they decide. – Jack Aidley Jan 04 '17 at 14:17
  • 1
    @Jack Aidley I definitely agree it's quite the problem, so I've added a bit to my answer to reflect. I hope that's alright? – Erdrik Ironrose Jan 04 '17 at 15:06
  • @JackAidley The problem with keeping the record of changes in the one-year-old-branch is that they lack any context with relation to the HEAD branch. Who cares what the history was in a branch that can't be built into the released product? Better to have the history of the work merged into the released product, and if a person is curious, they can read down the alternate-reality of the never-merged-for-a-year branch. – Edwin Buck Jan 09 '17 at 20:01
8

A few years back we had a client with the same requirements of keeping branches separate. So we did.

We never merged their branch back. They had there own unique version. We charged them extra for changes since essentially we had two major trunks instead of 1 main trunk and branches.

We attempted to merge back to trunk but after 2 weeks we decided to abandon that effort as it was burning hours of time with no tangible benefits.

So, don't merge it back. Going forward merge critical fixes to that client branch as needed and any enhancements would be one offs specifically charged to that client.

Jon Raynor
  • 10,905
  • 29
  • 47
  • This strategy is only feasible if the different lines of development target different clients. Or, if the use cases where the product is involved allow to to use the two different product lines in parallel, in a non-integrated fashion. To my understanding, the OP describes a situation where the new line of development targets the same client as the one who is using the trunk, and it is unclear if parallel usage of two product lines could make some sense for his case. – Doc Brown Jan 05 '17 at 08:53
1

It is not going to be fun, but how painful it will be depends on the nature of changes, and how isolated they are.

I suggest you attempt to converge the branches through refactoring as much as possible before you do an actual merge.

The merge tool is kind of dumb because it only looks at textual differences and does not understand the code in any way. If the main branch have changed the name of a class used throughout the application, and the feature branch uses the old name in some new code, then the merge tool will not understand that the class name should also be changed in the new code. But if you do a refactoring in branch B to rename the class like in branch A it will work in both old and new code and the merge will go smoothly.

Secondly, you should inspect how localized the changes in the development branch is. If the changes in the feature branch are localized to a few areas, you don't have to converge the unaffected code, you can just copy and overwrite from the main branch.

In areas of code where there have been non-trivial changes on both branches, you have to carefully inspect the code and decide how to rewrite.

JacquesB
  • 57,310
  • 21
  • 127
  • 176