8

Long story short

What is the best way to design a shared CD for multiple Git repositories (frontend and backend)?

I'm struggling to find the best design for our CD.

The whole picture (The details can be changed)

  • Frontend - Angular 2 (Javascript), no artifactory.
  • There will be only a single frontend in the project.
  • Backend - ASP.NET (C#), no artifactory.
  • There will be only a single backend in the project.
  • Git repository on Git TFS for each.
  • Server for Jenkins.
  • Multiple machines, Dgroup, for deploying the last version from branch development from both the repositories - a playground for each developer.

My goals:

  1. development branch must be protected by pull request and build must pass. In more details: Jenkins need to confirm that the last version of the branch who activated the pull request os compiled and the tests of his repository are pass. Also the integration tests with the other repository won't fail(The tests of the frontend). Meaning for each pull request in every repository, Jenkins need to "unified" the build for both the repositories. If every test of each repository passes, then the pull request is ready to be approved. Else, the pull request is denied.

  2. After each approved pull request, Dgroupmachines should be updated to have the latest version of the product (services + frontend files).

Flaws:

I can't find a way to achieve my first goal from multiple reasons:

  1. The frontend can't push any new feature from the beginning until the backend will finish the same feature from his side, else some frontend's tests won't pass. I don't know how it will effect the workflow of the frontend and if it is the best practice.

  2. Imagine a situation when the development branch in both the repositories is broken but no1 yet wrote the right test that will find the bugs (For now the build passes). Now the frontend create a pull request which contain a test that find a bug in the backend. So the build don't pass so automatically his pull request is denied.
     
    Now the backend fix the problem and try to pull request. Now his change also fail a test of the frontend. So his pull request also denied.As a result no one can pull request anymore.

My CD design, which I did not describe due to the flaws, prevent me from going forward. What is the best practice to design a CD for multiple Git repositories?

Robert Harvey
  • 198,589
  • 55
  • 464
  • 673
Stav Alfi
  • 297
  • 3
  • 10
  • 4
    uuh, what does CD mean in this context? Certainly not Compact Disk, the optical storage medium? – amon Oct 06 '17 at 21:01
  • 2
    I'm sory, CD means continuous deployment. – Stav Alfi Oct 06 '17 at 21:22
  • 1
    consider also [edit]ing to explain what "F.E" and "B.E" stand for, frontend and backend? – gnat Oct 06 '17 at 21:39
  • From what you describe the development of the front & back end for a given feature is done in parallel - have you considered having a single repository rather than two? – Justin Oct 07 '17 at 21:41
  • @Justin I have considered it many times but the repositories may grow too big for doing so. Also I'm more curious to understand how to implement CD with multiple repositories.. Seems like there is no solution for this kind of popular problem. – Stav Alfi Oct 07 '17 at 21:44
  • Did you look into usig Gitlab CI server rather than Jenkins? – Eternal21 Oct 10 '17 at 19:36
  • Your problem isn't the CD configuration. its in the right level of abstraction for the tests. are your tests unit or integration tests? and which ones you dictate to pass in order to pass the PR and deploy? – Bishoy Oct 10 '17 at 21:51
  • @Bishoy We are running integration tests between frontend and backen. if those tests pass, we will deploy. – Stav Alfi Oct 11 '17 at 14:50
  • @Eternal21 How Gitlab services provide solution for this situation? Can I implement those solutions in Git TFS ? – Stav Alfi Oct 11 '17 at 14:51

1 Answers1

4

IMHO the root cause of your problem is attempting to manage an entire system (frontend + backend) while still allowing each one of the parts to move independently and without the proper tool support for it.

A better (IMHO) approach would be to design your CI/CD approach with the system in mind. Any change to either part of the system (or a combination thereof) should always go through testing the entire system. This way you ensure the focus always remains on the entire system's integrity. I know, this goes against the micros-services trend (which I'm not very fond of, one of the reasons being exactly because they leave room for this kind of problems), but it is fundamentally what IMHO you need.

The complication is that the traditional CI tools, Jenkins included, aren't really designed for such approach. You could try to tweak it, but I imagine it's not an easy task.

Another problem is what I'd call absolute verifications vs just regression (or delta, if you want) verifications: for example you shouldn't reject a change to the frontend if verifications fails testing because the backend portion is not done - that testing didn't pass before, thus it's not a regression. Only after both the frontend and the backend functionalities were completed and the test managed to pass at least once a test failure should be blocking the commit - because it is now a regression. Donno if Jenkins supports such functionality.

You might want to take a look at ApartCI, it was designed with such system-oriented development in mind. Disclaimer: I designed it and I am the founder of the company offering it.

Dan Cornilescu
  • 869
  • 7
  • 14
  • Thanks for your answer. I will love to know why you would aproove a pull request of frontend if the backend didn't develop that feature yet. Why not using Git in this praticular situation by leaving that feature in a specific branch and when the backend developed his part, merge and push. – Stav Alfi Nov 26 '17 at 13:35
  • I am a CI fan, I believe that branches are serious obstacles in developing better quality software faster and with lower costs, with their impact increasing exponentially with the increase of the development team size. – Dan Cornilescu Nov 26 '17 at 15:23
  • You are explicitly in vafor for this. Please describe real life examples which " branches are serious obstacles" in your answer. – Stav Alfi Nov 26 '17 at 15:28
  • I come from a telecom background, I fought in the integration hell trenches. Merging branches can be a nightmare, both the feature being merged and integration branch receiving the merge are destabilized. Restoring the quality levels can take days, weeks, months - you never know ahead of time. And that needs to be done before merging the next branch. I even saw a huge project cancelled because after months of hard work the integration branch couldn't be stabilized - a major software release was never shipped for one platform and was significantly delayed for the others as a result. – Dan Cornilescu Nov 26 '17 at 15:57