44

Just got off a retro call where developers expressed concern around the integration of their stories into the master branch each sprint. The developers all code within their own branch and towards the end of the sprint they all merge into one master branch.

Then, one developer (usually the same one) is left with the task of making sure everything has integrated well with other dev's code (Most of the changes are on the same page. For example, a data display story, data filtering story, and a SLA indicator).

How can we reduce this burden and make it easier for our code to merge together? From my perspective, having the PO or SM prioritize the stories in a more efficient way so we don't have these sort of dependencies in the same sprint may solve some of the issues. How does everyone else tackle this? Or is this just part of the process?

cookiecutter
  • 551
  • 1
  • 4
  • 5
  • 18
    Don't you have a development branch where continuous integration is done? – Kayaman Jun 18 '18 at 13:43
  • 13
    I'm with Kayaman here, the best practice for this is to implement continuous integration. – RandomUs1r Jun 18 '18 at 16:44
  • 27
    [Happy Merge Day!](https://thedailywtf.com/articles/Happy_Merge_Day!) Any time your problem is too similar to something on The Daily WTF, you know you're in trouble. – user3067860 Jun 18 '18 at 17:06
  • Merge early, merge often: { write smallest test code that will fail (red), write smallest production code that will pass (green), refactor, retest, merge } while not finished. – ctrl-alt-delor Jun 18 '18 at 18:14
  • 3
    _First check-in Wins! Never be last!_ :-) – ChuckCottrill Jun 20 '18 at 00:10
  • 2
    The core problem is that your developers are merging towards the end of the sprint. That should not happen ideally. You should pick a features with a mix of storypoints. That way you can finish some features earlier, test them and merge them to develop. Other developers can then easily integrate updated `develop` code into their branch and will thus ensure less integration related bugs. – Abhishek Bedi Jun 23 '18 at 12:10

7 Answers7

89

If you are using Git, each developer would be pulling from the develop branch into their own feature branch so that they ensure they don't go too far from the current baseline. They can do that daily, so that tasks that take more than a couple days stay in sync and merge issues are resolved while they are still small.

When the developer is done with their work, they create a pull request. When approved, that gets merged into the develop branch.

The develop branch should always have working code, and be ready for release at any time. When you actually make a release, you merge develop into master and tag it.

If you have a good Continuous Integration Server, then it will build each branch when changes are checked in--particularly for pull requests. Some build servers integrate with your Git server to auto-approve or disapprove a pull request if the build fails or the automated tests fail. This is another way to find potential integration bugs.

Berin Loritsch
  • 45,784
  • 7
  • 87
  • 160
  • 74
    The important part (which is only implied in your answer) is that the branches should be merged as soon as they are ready, typically with only 1 – 5 commits, and not only at the end of the sprint. One branch per feature/story, not one branch per developer. That requires that stories are truly tiny, i.e. take at most two days. – amon Jun 18 '18 at 14:12
  • @amon, agreed. Added the words "feature branch", but trying to keep this answer fairly small. There's plenty of good articles that go into more depth on this process. – Berin Loritsch Jun 18 '18 at 14:15
  • What if you need to keep your branch isolated from the 'develop' branch for a long period of time? For example - if you're working on something that would break the application the way it is, and won't have it in a working state until much later in your coding process? – Zibbobz Jun 18 '18 at 16:20
  • 5
    Don't stay isolated on your own branch. That's how merge hell starts. Use mainline development, isolate work-in-progress behind feature toggles or other run-time configuration. – Rob Crawford Jun 18 '18 at 16:28
  • 3
    @Zibbobz My team uses explicit "Feature Branches" for those, which are basically treated like the develop branch is, but only for pull requests and commits that relate to that change. Generally, depending on how long it has to remain separate, every few days someone will merge the changes from develop into the feature and resolve any issues. That way the branches are as similar as possible when it comes time to merge. As a note, this is only for really big breaking changes – reffu Jun 18 '18 at 16:28
  • @reffu That sounds like exactly what we'd need. Thank you. – Zibbobz Jun 18 '18 at 16:41
  • @BerinLoritsch *...would be pulling from the develop branch into their own feature branch so that they ensure...* - Maybe a small edit with a `rebase` hint might be good here. – Xatenev Jun 19 '18 at 08:05
  • 10
    "isolate work-in-progress behind feature toggles or other run-time configuration" You've just avoided merge hell by going into config hell instead. "Merge hell" is only a problem for one developer at a time and easily avoided simply by synching up regularly, having tons of ephemeral configuration is hell for all future developers forever. – Cubic Jun 19 '18 at 09:09
  • Minor nitpick: What you are describing is the Git workflow called Git Flow. You can use Git with various other workflows (including the "everyone does whatever he wants" Flow), Git Flow is just the most common. – Tim Pohlmann Jun 19 '18 at 11:24
  • Is every commit in `master` supposed to be tagged by your logic? That's definitely not the practice I've seen in a lot of projects... – user541686 Jun 20 '18 at 03:22
  • 1
    Every time you push a release to production it should be tagged. But I typically don't commit to master. I only do pull requests from the develop stream when we have the right amount of work done. – Berin Loritsch Jun 20 '18 at 11:04
23

I worked in a team where we struggled with the same problem. We found that the less time we had before integrating, the less difficult it became. I know most people teaching continuous integration talk about committing every few minutes - we probably actually committed every hour or so.

We also found that just building wasn't enough. We needed a good test coverage level in order to make sure that we didn't accidentally break each others' code.

Peter Mortensen
  • 1,050
  • 2
  • 12
  • 14
Daniel
  • 2,041
  • 8
  • 12
  • 2
    This is my experience as well. It doesn't really matter how often you commit, but once committed quickly integration/merging the commit saves a lot of effort down the line. I once was on a project where we had three diverging development branches that each had months worths of work. Merging them was Not Fun. I learned a lot from that mistake :) – amon Jun 18 '18 at 14:17
  • 5
    Yes -- this is what "continuous integration" means! You're continuously integrating your changes with the other developers' changes! – Rob Crawford Jun 18 '18 at 16:28
  • @Rob, agreed. My statement wasn't meant to suggest that continuous integration isn't, well, continuous. Just that we didn't quite make the ideal and still saw a lot of benefits in getting close to it. – Daniel Jun 18 '18 at 16:56
12
  • Keep your branches short-lived (it sounds like you're already doing this).
  • Let your test results speak for themselves.
  • Don't wait for the end of the sprint.

You don't even need to subscribe to TDD for this one. All you need are some tests which prove that your developers' features are working correctly. These could include Unit Tests and Integration Tests but will ideally be a couple of automated end-to-end tests of the critical features. Standard regression pack stuff.

Then, once your merge has been completed, you can check the automation test report together and verify that everything has been integrated successfully.

I agree with one of the other answers where the author stated Git PRs would solve this problem by getting each developer to merge their own work.

One other point which I believe is important enough to leave until the last paragraph. I suggest that you run manual tests over your nightly builds, rather than waiting until the end of the sprint. Developers should merge in as soon as the feature is complete so that it can be integrated, deployed, and tested as soon as possible.

Pang
  • 313
  • 4
  • 7
Liath
  • 3,406
  • 1
  • 21
  • 33
6

Don't

Depending on your language and what files you're editing, it may not make sense for each developer to edit them on their own branch. For example, in C# I've found it's best for only one person to edit any UI designer files at a time. These are autogenerated files, and so code is sometimes moved around for no apparent reason - and this wreaks havoc on most merging tools.

This means that some stories may block other stories until the UI work is done. And/Or, a new story is created to just layout the UI, with the other stories implementing functionality. Or, maybe one developer does all the UI work while others implement the functionality of that UI.

On a related note, if you know multiple stories will all be touching the same file(s), you may just want to avoid working on them all at the same time. Don't pull them all into the same sprint, or don't start working on them all until one or more are done.

mmathis
  • 5,398
  • 23
  • 33
  • Honestly, the version control tool in use is more critical to successful branching and merging. Even with C# code and the presumably WinForms or WebForms code you must be working with typically don't change _that much_. If they are, perhaps you need to do some mockups before playing with code. XAML based UIs are just as stable as regular code and intermediate code is not checked in. – Berin Loritsch Jun 18 '18 at 15:08
  • 2
    @BerinLoritsch WinForms designer code can indeed change a lot, even with small visual changes. I've found that the lines of code themselves are the same, but the ordering is vastly different - especially when multiple developers are making edits at the same time. Maybe it is an issue of VCS tool (we've used several, maybe we're just using the wrong ones), but for us it's much much simpler to alter our process slightly. – mmathis Jun 18 '18 at 15:15
  • 2
    @BerinLoritsch I have to second mmathis here at least for win forms (never used web forms). The winforms UI designer loves to randomly reorder all the code in the designer file in response to a trivial change somewhere on the form. Unless you manually undo the reorderings prior to each commit (something that can easily be 10 or 15 minutes on a complex form) the history of the designer file is absolutely useless, and if 2 people are working on the form's UI at once will result in a merge conflict from hell. Locking is generally a terrible option, but with winforms really is the least evil. – Dan Is Fiddling By Firelight Jun 18 '18 at 20:25
  • @DanNeely, That's just one of the reasons our team migrated away from WinForms code. Another reason is that the designer is terribly fragile, and some of our complex forms couldn't be edited visually anyway. We ended up having to make changes directly in the codebehind--probably why I don't recall too much upheaval there. That and our users working with high density displays really pushed us to WPF. A painful process with a high learning curve, but a nice reward at the end of it. Most stories in the backlog were for different parts of the app anyway. – Berin Loritsch Jun 18 '18 at 20:30
  • @BerinLoritsch same here. Win forms paid a big part of my bills for most of a decade at my previous job, but I'll be quite happy to never touch it again in the future. – Dan Is Fiddling By Firelight Jun 18 '18 at 20:32
  • As somebody who's worked with winforms as well, I can confirm that even something as minor as changing the font or color of something can and will reorder 80% of the code for no reason in particular. Treat designer files like a binary blob. Seems like the designer's code generator is just non-deterministic by design and there's nothing that can be done about it. I like WPF but IMO it's not all good news; WinForms has an impressive amount of backwards compatibility and third party support; disappointingly WPF *still* has poor 64 bit support and it has its own quirks / instability issues. – jrh Jun 18 '18 at 21:00
  • I don't know about winforms, but the on project on which I'm currently working has a tool used to modify our product model which will generate complex xml files used extensively throughout the project. The tool tends to re-order the xml any time it is touched, so our solution was to sort the xml before committing it. This resulted in much better diffs. Don't know if that's a solution for the problem you folks are having, but it works great for us. – Ted Delezene Jun 18 '18 at 21:54
  • @TedDelezene That would definitely work for xml files, but for *.designer.* files Visual Studio itself on save reads in the changes you made to the GUI and generates C# code, and there's no way to control the order it writes those controls out to the designer file. I guess in theory VS could sort the controls by name, that would be a nice feature request, but I think at this point Microsoft isn't really interested in updating the designer. I haven't seen any major changes to it in quite a while, even though there's open bugs. – jrh Jun 19 '18 at 02:49
  • If the files are autogenerated why are they under version control anyway and why are they being edited? Sorry, not familiar with WinForms but that sounds horrible. – Cubic Jun 19 '18 at 09:12
  • @Cubic They are not autogenerated by the build process, but by the UI designer portion of Visual Studio. VS provides a drag-and-drop interface to build your UI, and autogenerates the code which constructs that UI. Most people don't edit the autogenerated code directly, but rather by adding or changing elements in the VS designer (which then autogenerates new code). – mmathis Jun 19 '18 at 11:59
2

Another possible approach to avoid late and large merges are feature flags: you protect your changes with a (ideally dynamically) configurable flag that prevents them from becoming active before intended.

This allows you to merge your changes early back into either master or your joint development branch without breaking anything. Other developers can then merge these changes back into their feature branches (or rebase their branches accordingly).

As the other answers have already pointed out this should be combined with a continuous integration solution.

Feature flags have additional benefits (for example, they make it easy to do A/B tests). See this article by Martin Fowler for more information.

Florian Brucker
  • 299
  • 1
  • 6
0

We are following an approach of separate development branch for each feature, and then we are merging the branches to a QA branch for testing in integration testing environment.

Once regression and integration testing is completed, we easily move the features which are ready to go, to the release branch.

If all goes well, we merge release branch back to master branch.

emarshah
  • 101
  • 1
0

To put it simply, committing and merging often reduces the window of opportunity for merge conflicts and will greatly reduce conflicts. The other part is indeed planning by the lead, which can further ensure that work flows smoothly.

The other answers give some great insight regarding best practices for commits and simply by following those you'll probably reduce the vast majority of your merge issues. More merges is almost certainly a necessity, but for a smaller team, your branch-per-person approach probably works well enough. Of course, it doesn't hurt (much) to get into more extensible practices though!

However, no one seems to have addressed one of your most significant questions--what to do when you're all touching the same areas of code. This is where it's useful to have a lead who is familiar with the code base and can recognize dependencies of different tasks. If they don't orchestrate the timing of work and commits, you'll likely end up with merge conflicts and line by line resolution. Organizing the tasks\timing is much more difficult with a larger team, but with a small team its possible to identify these conflicting tasks. The lead could then even shift all the related tasks to the same engineer, to avoid the conflict altogether.

Mars
  • 158
  • 5