37

Let's say that I have a project that depends on 10 libraries, and within my project's trunk I'm free to use any versions of those libraries. So I start with the most recent versions. Then, each of those libraries gets an update once a month (on average). Now, keeping my trunk completely up to date would require updating a library reference every three days.

This is obviously too much. Even though usually version 1.2.3 is a drop-in replacement for version 1.2.2, you never know without testing. Unit tests aren't enough; if it's a DB / file engine, you have to ensure that it works properly with files that were created with older versions, and maybe vice versa. If it has something to do with GUI, you have to visually inspect everything. And so on.

How do you handle this? Some possible approaches:

  • If it ain't broke, don't fix it. Stay with your current version of the library as long as you don't notice anything wrong with it when used in your application, no matter how often the library vendor publishes updates. Small incremental changes are just waste.
  • Update frequently in order to keep change small. Since you'll have to update some day in any case, it's better to update often so that you notice any problems early when they're easy to fix, instead of jumping over several versions and letting potential problems to accumulate.
  • Something in between. Is there a sweet spot?
gnat
  • 21,442
  • 29
  • 112
  • 288
Joonas Pulakka
  • 23,534
  • 9
  • 64
  • 93
  • 1
    +1 : I wonder if like "bug hunts", you could have an iteration of "Update Sprint" in a project. Curious about answers :) – Matthieu Nov 25 '10 at 17:23

12 Answers12

35

I'm shocked - and indeed appalled - at the number of answers here saying "don't update unless you have to". I've done that, and whilst it's easier in the short term, it burns like hell in the long run. More frequent, smaller updates are much, much easier to manage than occasional big ones, and you get the benefit of new features, bug fixes, and so on sooner.

I don't buy this idea that library changes are somehow more difficult to test than code changes. It's just the same - you're making a change to the codebase, and you need to validate it before you commit, and more deeply before you release. But you must already have processes to do this, since you're making code changes!

If you're working in iterations, of two to four weeks length, i would suggest making updating libraries a once per iteration task, to be done as soon as possible after the start, when things are a little more relaxed than just before an iteration deadline, and the project has more capacity to absorb change. Get someone (or a pair if you do pair programming) to sit down, look at which libraries have been updated, and try bringing each one in and running a rebuild and test. Budget half a day to a day for it each iteration, perhaps. If things work, check in the changes (i'm assuming you keep libraries in source control, as we do; i'm not sure how you'd propagate the change in a controlled way if not). This will obviously be a lot easier if you have automated tests than if testing is entirely manual.

Now, the question is what you do if an update breaks things - do you spend time fixing it, or leave it out? I'd suggest leaning towards the latter; if it can be fixed in an hour, do it, but if an update is going to take significant work to integrate, then raise it as its own development task, to be estimated, prioritised, and scheduled just like any other. The chances are that unless it brings in some very crucial fix or improvement, the priority will be low, and you'll never get round to it. But you never know, by the time the next iterationly update day rolls round, the problem might have fixed itself; even if not, at least now you know that there's a roadblock on the update path, and it won't catch you by surprise.

If you're not doing iterations of that length, i would set up some kind of standalone schedule for updates - no longer than monthly. Is there some other project rhythm you could tie it to, like a monthly status review, or an architecture board meeting? Payday? Pizza night? Full moon? Whatever, you need to find something a lot shorter than a traditional release cycle, because trying to update everything in one go every 6-18 months is going to be painful and demoralising.

Needless to say, if you do stabilisation branches before releases, you wouldn't apply this policy to them. There, you'd only update libraries to get critical fixes.

Tom Anderson
  • 3,022
  • 19
  • 24
  • 4
    +1. I worked on a project where the devs applied the "if it ain't broke don't fix it" policy. Then we found an issue with a 3rd party library (relatively minor but it was required for a new feature), which was only fixed in a much later version, which in turn depended on a much later jvm. With the later jvm, we found issues with other 3rd party libraries, that now had to be upgraded in turn. We also had to upgrade our hardware, since Oracle does not have a 32-bit jvm for Solaris anymore. It was a mess and could so easily have been prevented by simply keeping stuff current. – firtydank Jan 18 '16 at 12:32
  • 1
    +1 for "whilst it's easier in the short term, it burns like hell in the long run". I've experienced both approaches and while many small updates may seem like a nuisance, not performing upgrades and then having to upgrade 10 libraries from versions which are 2 years old is often not possible in any reasonable amount of time. You end up with a system which depends on outdated and unmaintained libraries, you can't use some other libraries since they require a newer version of that library which you can't upgrade and at some point you lose the ability to fix some issues at all. – Michał Kosmulski Feb 06 '17 at 14:37
  • @firtydank I'm curious what you guys did to resolve this after-the-fact, did you implement any new policies? What was the functional change to the organization? – buddyp450 Mar 12 '19 at 20:06
10

I evaluate.

  • Firstly I look for bugs that we've raised against that library and see if they've been fixed.
  • Secondly I look for any other bug fixes in the lib that we could benefit from (perhaps something that is a possible corner case).
  • Thirdly I look for improvements in the lib/API and then investigate the impact of changing our code to utilise that and the benfit tradeoff. I've all too often in the past upgrade libs without actually utilising their new features, silly really!

I then weigh all of that up against staying with the existing lib.

Always test - hopefully your unit/integration tests will ensure that no major regressions occur.

Martijn Verburg
  • 22,006
  • 1
  • 49
  • 81
7

The major problem with third party libraries, is that you need to retest YOUR application when you update them, before it can go in production. So unless you have a reported bug, which require updating a library, you don't touch them until you have the time to do a complete quality assurance cycle.

This is usually done when releasing a new version.

I would, however, suggest that you have a test suite in place for continuous build that allow you to update the libraries in the development branch and do so automatically. This will ensure that you discover early when it breaks, so you can file bugreports to the project.

3

Partially, as described in svn vendor branches. The procedure described there is very useful, when you keep using open-source third-party libraries for a long time, and have made changes to adjust it for your needs.

duros
  • 2,464
  • 1
  • 21
  • 25
  • 2
    Please don't just drop links. Links tend to vanish. Kindly consider at least summarizing what you are linking to. If that link broke, how useful would this be .. perhaps a few years from now? – Tim Post Nov 25 '10 at 17:56
  • And up-voted :) – Tim Post Nov 25 '10 at 18:08
2

I would think about updating all the libraries of a project right before or just after a release. This however could get out of hand if you rely on more than 10 or 15 libraries, in which case some kind of update checking mechanism would help out greatly. The advantage of this is that you have a time dedicated to testing your libraries and can fix any issues in one pass. You also don't have to constantly keep track of updates from every single library, you just check on a certain day for any updates.

I would also go against any kind of auto updating feature in even a dev branch. It would be frustrating if in the middle of me working on something the project broke because a library auto updated itself, or I all of a sudden got Depreciation warnings for using an API that just got superseded by something else.

TheLQ
  • 13,478
  • 7
  • 55
  • 87
2

You have to ask, what do you really want out of the update? Most security fixes are actually trivial patches, in the form of fixing:

  • Off by one errors where arbitrary code can be copied to an unused space in a buffer
  • Dangling pointers, or something else that triggers undefined but (rather) deterministic behavior
  • Bugs that permit some kind of DoS
  • Bugs that accidentally make snooping of private data easy
  • Math blunders
  • Maintainers touching stuff they shouldn't (Debian SSL bug, anyone?)

If you look over most CVE's in the last five years, the patches that fix them are typically quite trivial, if you are using open libraries, which I hope you are.

Then you have actual bug fixes, which you probably want, but perhaps you already fixed it yourself. If it isn't broken, don't fix it.

Finally, you have new features .. and perhaps deprecated features. You need to examine those release notes and diffs carefully. Can you use them, even if they break an API that lots of other stuff depends on? If yes, it is time for surgery .. if no, cherry pick what you want and move on.

Some may disagree with me, but I refuse to use a library without source code.

Tim Post
  • 18,757
  • 2
  • 57
  • 101
  • Surely I prefer open source libraries, but I also use some commercial libraries that cost like $100 without source or $10k with source, so... – Joonas Pulakka Nov 26 '10 at 08:57
2

If your software is security-critical, you must update as soon as possible, no excuses. You don't want a little bug in a graphics library to make your whole program vulnerable.

Otherwise, when the lib is mature, it's "If it ain't broke, don't fix it." for me. Sooner or later, I might need a feature of a later version and have no choice but to update, but til then, the effort is hard to justify. On the other hand, when I work with a relatively new lib or framework, like Grails or ExtJS, I stay up-to-date with the latest version because these products don't feel completely mature yet, so updating is likely to save me from running into one of those bugs the later version fixed.

user281377
  • 28,352
  • 5
  • 75
  • 130
2

It varies depending things like on the libraries, what they're used for, how pervasive they are in your code, the cost (in terms of time and money) of performing the upgrade, and so on.

Ideally you would always have the latest all of the time, but if the new version isn't backward-compatible, what then? You may have to shelve that update for a future release until you can handle the change carefully. There could be some subtle change in behaviour (such as "you now need to set property X before calling method Y, or you get a slow memory leak") which is difficult to verify in testing.

On the other hand, the new version could have some serious security fixes, so you have to take that into account as well.

Short version: take it on a case-by-case basis.

JohnL
  • 1,890
  • 13
  • 14
1

I use NuGet to keep my third party libraries up to date.

When a friend, coworker, or blog notifies me that one of my 3rd party DLLs is out of date, NuGet makes it very easy to update it.

Jim G.
  • 8,006
  • 3
  • 35
  • 66
1

This would depend on your release shedules.

But my advice would be to install a set of libraries on all of the developers machines. Consider it a gold standard if you wish to call it something, then start you development for that release.

Only once the release has been deployed and you are in the post release phase, reassess your libraries, their versions and features. If they offer some significant improvements or new functionality then install then prior to the start of the next development cycle.

Only install new versions if there is a major issue or bug that must be fixed before deploying the software.

It will mean you will miss some versions but it should save some headaches and versioning issues allowing you to concentrate on developing your application.

GrumpyMonkey
  • 716
  • 9
  • 20
1

Subversion Externals

What is great in that feature is that you can specify the revision you want.

Please note that updates will be slower if you have many externals.

  • Actually I'm using them, and they're very useful and very slow X-) But they don't solve the problem of *when* I should update to the newer version. – Joonas Pulakka Nov 26 '10 at 09:01
  • They can be very slow yes. I usually update external libraries when: (a major release OR a bug fix that affects me is available) AND I'm in the first half of the iteration. –  Nov 26 '10 at 09:07
1

I'm currently setting up something like that :

  • 1 mercurial repo for each extension of my application
  • a mercurial repo that gather specific versions of several 3rd party libraries
  • a SVN repo for graphic resources/works (but might change to something else)
  • a mercurial repo for my application, that uses the mercurial subrepos feature to use specific version of the 3rd pary repo, and some of the basic extensions

Now when I need to work on an extension that is not "basic" (implicitely included as a subrepo in the application repo), I simply get clone the repo in the extensions folder and let CMake generate the projects and solutions for the whole application.

That way, I can :

  • change the 3rd parties in a clone, check that it works with the app, push it in the 3rd pary repo, then update the subrepo version of the application repo to the new 3rd party repo version
  • work on extensions independently, all togeter or just pick some specific
  • not have to worry about having to link projects together, this is done by Cmake just by having the projects subrepos scanned with the whole application repo.

Don't have a lot of experience with this organization yet, but i think that's pretty useful.

Klaim
  • 14,832
  • 3
  • 49
  • 62