14

How do you keep released binaries under version control? This allows to track which stuffs are changed between each release. I mean to separate the released binaries from source repository. The released binaries are either built from Continuous Integration software or manually compiled.

linquize
  • 1,150
  • 1
  • 11
  • 20

5 Answers5

21

Two options:

a) Don't. Just make sure you have reproducable deterministic builds, that is, building the same source control revision with the same configuration always produces the exact same binary.

b) Designate a directory somewhere as the authoritative source for published builds. Make uploading the binaries part of the deployment / shipping procedure,and make sure the published-build directory is covered by your backup plan. You don't need any version control here; builds are write-once, if you need to change anything, you make a new build.

Either way, binaries and other build output don't belong under source control, for numerous reasons.

tdammers
  • 52,406
  • 14
  • 106
  • 154
  • 7
    a) is often not possible of course http://blogs.msdn.com/b/ericlippert/archive/2012/05/31/past-performance-is-no-guarantee-of-future-results.aspx – jk. Jun 07 '12 at 07:55
  • @jk: nice link. I guess having the source code and all input files for the build stage fully under source control (and having the right compiler version installed) may be "deterministic enough" in a lot of real-world scenarios (and not enough in others, of course). It is case dependent how important it is to have reproducible binaries bit-by-bit. – Doc Brown Jun 07 '12 at 09:43
  • Link is dead, I think this is the same article. https://ericlippert.com/2012/05/31/past-performance-is-no-guarantee-of-future-results/ – MichaelCG8 Mar 21 '23 at 12:14
10

Use an artifact repository for binaries, not a version control system. A specific version of a released binary is not supposed to change over time, hence version control does not make sense since the file(s) wouldn't change.

See for example Maven repositories as a repository to archive/publish/offer releases and other binaries (e.g. such as documentation)

Community
  • 1
mhaller
  • 339
  • 1
  • 4
  • 1
    a specific version of a released source file isn't supposed to change over time either. SCM is a fine place to put the shipped binary, you just store it alongside the source files used to build it. – gbjbaanb Jun 11 '12 at 13:25
  • 2
    gbjbaanb, SCM stands for "Source Control Management". That should give anyone a hint that these systems are not designed to store binaries but source. If you still want to store binaries, go ahead. I won't. – mhaller Jun 11 '12 at 16:19
  • 4
    SCM stands for "Software Control Management" but nice try. "source code" is often not just text files, but images, documents, diagrams etc. – gbjbaanb May 01 '14 at 07:28
  • Normally, "Software Configuration Management." I have never heard of "Software Control Management." – James McLeod Dec 28 '18 at 23:48
7

Just put them in. There's no problem with that, unless you're using git (which doesn't merge binaries well, so you'll have to manage them yourself) or you're committing them too many times (only commit when it's ready to ship, not everytime you build it).

Most SCMs delta binaries quite well, we used to put a 2Mb resource dll into our SVN and it would delta to a few kb each time.

I hear a lot of arguments that SCMs are for source, not binaries but this is plainly false when you consider most software consists of images, even if they are just icon files. They're binaries, but they are part of the source, so put them in and don't be so dogmatic about it. I also hear that you can just rebuild the binary when needed, often this is the case, but it can be a huge time-wasting effort for older systems that are no longer actively supported. If you have to re-create a system with only older service packs or patches to correspond with the system that was used to build a binary 3 years ago, you'll be glad you added the bin to your SCM back then.

The only time you need to worry about adding builds to your SCM is if you're doing it automatically as part of the build server process - don't do this. You will fill your SCM up with builds that have no benefit to you. Instead only add them when they are released. This way you know exactly what your customer has and you can reproduce any customer-reported issues with the binaries they are using, and not ones that you've rebuilt (using, lets say, latest updates to the compiler or the OS).

gbjbaanb
  • 48,354
  • 6
  • 102
  • 172
  • Downvoting. Released binaries have no place in version control, because they can be generated at any time from the files that *are* in version control. (You need a repeatable build process for this, but you should have one anyway for other reasons.) – Marnen Laibow-Koser May 18 '18 at 00:50
  • 1
    @MarnenLaibow-Koser you obviously haven't worked in the industry for long. Build environments change over time, so whilst you could rebuild an old one, the chances are you'll have to spend days re-creating the env with the right old tools and SDKs. Which I hope you've versioned.... – gbjbaanb May 18 '18 at 16:38
  • Spare me the condescension. I've been working professionally as a software developer since 1998, and using version control since the early 2000s. I make sure that my version control repositories are set up so that they specify the versions of all their dependencies, and therefore I have never had a problem repeating a build as necessary, so I don't need to put builds in my repo. (It does help that I mostly work with interpreted languages, but I do do some work in compiled languages, and this process works there too.) – Marnen Laibow-Koser May 18 '18 at 16:54
  • 1
    I imagine you'd be able to build an old VC6 binary that was compiled on a XP machine using a old SDK that Microsoft no longer sees fit to release. If someone asks for that binary, its easy to just grab it from the same place everything else is stored. The cost of managing that is massive compared to the pain of having to rebuild, and the cost of storing it with the source is insignificant. I have been there (with a VB6 app made with a 3rd party control when the customer reported a bug - getting the binary and running it was a lot easier than rebuilding it, which turned out to be impossible) – gbjbaanb May 18 '18 at 17:18
  • @gjbaanb Of course you should keep old releases around, but the repository that contains the source code is not the right place to put them (given current VCSs, anyway; I can imagine a hypothetical VCS that would associate a binary release with a source commit in an out-of-band way). This is because they're only in sync with *one particular source commit*, whereas typical VCSs assume that anything committed stays relevant till changed. (You could add your r500 build to the VCS in r500 and delete it in r501, I suppose, but that seems silly with current tools.) Or just store a Dockerfile. – Marnen Laibow-Koser May 18 '18 at 17:23
  • 1
    @gjbaanb I will admit also that you're in a different situation from me in another way: all my external dependencies are OSS, so they can't go away just because some company no longer sees fit to release them. – Marnen Laibow-Koser May 18 '18 at 17:24
  • 1
    My point is that SCMs can store everything, so there's no need to *not* store the binary separately from its source. Its convenient and easy and guarantees you know what's what years later. There's no downside to doing so. Any given release itself is only in sync with one particular source commit too. I don't see the problem, except that some people think SCM means source code text only. No, use it to store almost everything (sometimes inc build tools or libs if they're not huge), life becomes much easier. – gbjbaanb May 18 '18 at 17:32
  • 1
    @gjbaanb And my point is that you're wrong about that. :) Sure, VCSs can store everything, but they're not well set up for storing files that only live for one commit (after all, you don't want your r500 build in the repo at r550; it will just be confusing). The fundamental problem with storing build artifacts in the repo is not that they're *binary*, but that they're *derived data* and will go out of sync with their source in the same repo. The same arguments I'm using would apply to generated text files. – Marnen Laibow-Koser May 18 '18 at 17:35
  • How would they go out of sync when they're stored alongside their source? I think you're assuming all builds go in, whereas I'm suggesting only releases go in (although I know a build system that did store every build and pruned old branches regularly). So when you tag your build for release, the binary gets stored, usually as part of the build system. – gbjbaanb May 18 '18 at 17:45
  • Let us [continue this discussion in chat](https://chat.stackexchange.com/rooms/77707/discussion-between-gbjbaanb-and-marnen-laibow-koser). – gbjbaanb May 18 '18 at 17:45
  • "How would they go out of sync when they're stored alongside their source?" If you check in the r500 build, then the moment you make a source change for r501, the source in the repo is out of sync with the r500 binary that is still in the current version of the repo. • "I think you're assuming all builds go in" No, though that would actually work better than what you're talking about. – Marnen Laibow-Koser May 18 '18 at 19:18
5

I don't keep release binaries under version control. Instead I publish them to a well defined location so that other tools and inspect and use them. I do a lot of work in Java, so that means I publish Jars to local Maven repositories. However, I don't use these tools to track what has changed per release. After all, they are binaries and there's not really much to track other than file count.

In order to track changes between releases, I would tag or label releases in my version control system with the version number of the release. But this is really only to track the source files, not the binaries. The binaries are artifacts of the build and do not need to be under version control.

Jeremy
  • 4,791
  • 1
  • 25
  • 40
1

The best solution is to make exclusive use of your CI system for all organizationally significant builds (releases, release candidates etc...).

This systematically ties the released binaries to the repository content without having to actually store the binaries in the repository.

For example, if you are using SVN, use the branch-major organizational scheme; do all day-to-day development in /trunk, and create a /tag for each release once it is ready.

Configure your CI system to build from tags as well as from trunk, and get it to write output to a network directory whose structure mirrors the top level structure of the repo:

  • /builds/trunk/[rev][date][build_id]/
  • /builds/tags/release_0_1_3beta4/[rev][date][build_id]/

The build system will need to treat the /builds/trunk/ directory like a circular buffer, storing the last n builds, deleting old builds as it goes.

The /builds/tags/ directory, on the other hand, is a permanent store. The build artifacts themselves are stored in directories with names generated according to the following scheme:

  • [rev][date][build_id]

where [rev] is the SVN revision ID, [date] is the date in YYYYMMDD format, and [build_id] is a 3-digit unique counter, incrementing from the first build onwards, making each build directory unique.

The process detailed above gives you the following benefits:

  1. Build artifacts are tied systematically to the source that generated them, so you can find the source for a particular build artifact very easily, (and vice versa).

  2. This forms the basis for further release automation. For example, automatic generation of release documents etc...

William Payne
  • 1,161
  • 8
  • 20