0

Is there a standard procedure for producing some jars, having them tested downstream, and then promoting these jars to the actual release? By promotion I really mean taking already publish artefacts which are known to be good, and rereleasing them with minimal effort.

The approaches I can see are, either really produce the jars with the release name, and just don't tell anyone they're out, get them tested downstream and if that goes awry, publish over the old 'release', and if it goes well, announce the release.

Or, publish them with a different name like RC-1, and then subject to everything going well, regenerate the poms, copy the published jars potentially renaming them, and potentially having to mutate metadata inside the jar.

Neither approach seems especially great, but it can't just be me who is dissatisfied with the idea of republishing the jars.

daniel purdew
  • 13
  • 1
  • This sounds like un-tested changes are being merged in to the main/master/trunk branch? If so, I'd suggest looking at this as the root issue; i.e. the fact that you don't have confidence in artefacts being pushed into release sounds like you cannot assure the stability of code in the main branch. You could consider putting a much greater emphasis on testing changes while they are still in feature/development branches (keep those branches up-to-date with main) so that anything which touches your release process is already tested. – Ben Cottrell Aug 26 '22 at 08:20
  • @BenCottrell that’s not really so, changes get merged into main after running unit tests. Then downstream we do some further integration testing. We do trunk based development. For some further context, we produce about 1000 jars across 6 repos, with (acyclic) dependencies between the repos. – daniel purdew Aug 26 '22 at 11:27
  • That's what I mean - if the integration tests are only run after the merge then they're not fully tested at the point in time when they're merged or committed into the trunk. Whatever the branching strategy, I would suggest not cutting releases from a branch or trunk that may potentially contain untested changes. – Ben Cottrell Aug 26 '22 at 11:36
  • @BenCottrell I don’t really understand what you’re proposing. It wouldn’t be practical to do the integration tests pre merge. We have maybe 200 developers, the integration tests take a couple of hours and really because they’re downstream of all these repos, they require published snapshots to exist. In principle we could publish jars versioned by the mr etc. but even then, they would become ‘untested’ as soon as someone else merged into main, because that change in conjunction with the one we formerly tested may be breaking. For us at least no work would ever get done like this. – daniel purdew Aug 26 '22 at 20:13
  • I am entirely ready to be told what my company does is crazy. It is the only company I have worked for. – daniel purdew Aug 26 '22 at 20:15
  • It's fine to create unstable/untested packages (You could publish to a separate registry that keeps a limited history and/or purge later with a cleanup task if needed). It sounds like it'd be worth treating the trunk purely as a main development branch and tag packages published from that branch as unstable/pre-release (with a matching tag in source control) -- when you get a green light from testing I would merge that tag to a release branch, and then tie the release process just to that branch, to avoid the untested packages being treated as release. – Ben Cottrell Aug 27 '22 at 07:29
  • The idea of a separate release branch would be that it only contains merge-commits of already-tested code in a final/releasable state; on the other hand the trunk is fluid, so any CI process triggered from trunk merges/commits likely need linting, unit tests, etc (Perhaps even a source for nightly integration test runs). Knowing that commits in the release branch have always been through all that stuff as well as testing, means it doesn't need to repeat any of that - it might even just be a lightweight release process of creating stable-versioned packages and publishing them. – Ben Cottrell Aug 27 '22 at 07:42
  • @BenCottrell I think I've not made my question very clear, probably explains the downvote! And now we're sort of getting into a long chain of comments. But I think we've ended up talking about something different. That 'lightweight release process' you refer to really is what my question is about. I dislike the idea of republishing things at release time. I want a process by which I take some already known good artefacts and 'promote' them, not needing to republish things. One reason (there are many), is that this lightweight publishing would be a different cicd that only got tested at release – daniel purdew Aug 28 '22 at 17:48

1 Answers1

1

Based on the comments, the simplest suggestion I can think is something similar to the Nexus staging workflow, pointing the existing process as well as all downstream integration tests at a new 'staging' registry to ensure real users won't be exposed to untested artefacts.

On versioning, consider using the real release names and version scheme extended with an auto-generated build number for tracing back to a source tag and a pipeline run.

From a versioning point of view (assuming SemVer or some variation), it's reasonable to publish changes with the same Major/Minor/Patch version by including the generated build number at the end, i.e. "X.Y.Z+${build_number}" (note: + is the separator used by SemVer. ${} is for string interpolation/substitution).

The last step for promoting the package could be as simple as copying it into the production registry; without modifying versions or artefacts (I would argue it's beneficial to be able to trace a public release back to its original pipeline run).

Roughly speaking, suggested changes to the CI Process, for each pipeline run triggered from trunk:

  1. Increment a persistent per-pipeline build counter upon each triggered run after the build runner checks out the changes.
  2. Extend the version core to generate a full version identifier by appending the build counter.
  3. At the end of any successful build, tag the commit with the full version identifier.
  4. Publish as before to the staging registry.

Many CI tools should have the facility to manage the build counter and version scheme straight out-the-box; if yours doesn't it's a fairly simple script to write.

If your package registry doesn't include the ability to promote releases (or if it's a feature you're not paying for) then that should also be a fairly simple script to download-then-upload for each artefact, knowing the build number which traces back to that specific pipeline run.

Ben Cottrell
  • 11,656
  • 4
  • 30
  • 41
  • Thank you, this nexus pro staging workflow actually looks brilliant, exactly what I wanted. Unfortunately I won't be able to convince my company to get this nexus pro, looks a bit expensive, but as you say it won't be /so/ hard to implement this workflow myself. And given that Nexus offers this approach I take that as the green light that it is a sensible workflow. – daniel purdew Aug 29 '22 at 16:31