40

Today, I updated ZBateson\MailMimeParser the PHP e-mail parser library from 1.x to 2.x.

Soon enough, my PHP error log started filling up with errors.

Noting where it happened, I found out that it had to do with their ::parse(...) function: https://mail-mime-parser.org/upgrade-2.0

An additional parameter needs to be passed to Message::from() and MailMimeParser::parse() specifying whether the passed resource should be ‘attached’ and closed when the returned IMessage object is destroyed, or kept open and closed manually after the message is parsed and the returned IMessage destroyed.

That is, instead of picking one of those new "modes" by default, the author(s) simply chose to break all existing code.

Frankly, even after re-reading that page multiple times, I have no clue what the new parameter actually does. I have set it to true just to make the errors stop happening, but I'm worried that this is somehow not the right choice.

My point, and question, is: Why do library developers knowingly break existing code like this? Why not at least have it default to either true or false, whichever is the most reasonable?

Before you tell me that I should have read the upgrade instructions before updating, I sometimes do, but when your life consists of nothing but dealing with constant updates of all kinds of software, you eventually get numb to all the changes and stop spending the time and effort to do so. Is it really reasonable that updating a library (in particular) should break existing code?

And this is not some sort of edge-case of the library, either. It's literally the #1 reason for it to exist in the first place, sure to be used by every single user: parsing an e-mail blob!

user16508174
  • 475
  • 4
  • 4
  • 11
    I think this question should be directed at the developers of the library in question. – JacquesB Sep 02 '21 at 18:11
  • 34
    It doesn't have to be answered specific to this library. It's a general enough question, even though it cites a narrow example. – Karl Bielefeldt Sep 02 '21 at 18:18
  • 8
    In this specific case, with a default option you will either have a memory leak, or silently break certain use cases. "true" is the safer option, since it will close the resource automatically. But "true" will not work if the stream is processed by other functions further down the line, without the IMessage-Object. e.g. asynchronous processing, caching and such things. In these cases you have to specify "false" and make sure the resource is closed at the right time to prevent a memory leak. – Falco Sep 03 '21 at 11:38
  • 3
    Simply put: Because library developers make mistakes and want to fix them. – MechMK1 Sep 03 '21 at 12:02
  • 1
    Similar: *[Resource linking fails on lStar](https://stackoverflow.com/questions/69021225)* – Peter Mortensen Sep 03 '21 at 12:42
  • @Falco your comment would make a good basis for an answer. It gives a practical and clear explanation for why the API ought to change. – Clumsy cat Sep 03 '21 at 14:10
  • 59
    Note that the authors of the library followed semantic versioning principles and changed their Major Version number, which generally indicates breaking changes may occur. I always consider that before upgrading any library, no matter how numb I am. – Robert Harvey Sep 03 '21 at 14:23
  • 13
    Having breaking changes is literally the definition of a Major Version Change. – njzk2 Sep 03 '21 at 21:56
  • 2
    At a guess, the library authors considered just making it `true` (or `false`) by default to make your errors go away, but **they** were worried it would not be the right choice too. So they left it with no default to prompt you to read the documentation and figure out which option is right for you, since they have no reasonable way of getting the information they need from you (and every other client of the library) to choose for you. – Ben Sep 04 '21 at 01:20
  • 2
    "from 1.x to 2.x." full point releases are generally made to "break" from previous versions... otherwise, it would be 1.(x+1) as normal. – WernerCD Sep 04 '21 at 05:08
  • 1
    "I have no clue what the new parameter actually does" -- indeed that doc in passive voice omits the main point: who's responsible for deallocating the resource. – Jerry101 Sep 04 '21 at 06:25
  • If you were the author of some popular library, i'm believe you wouldn't actually take the approach you're advocating they should take. I think you'd carefully weigh a "to break or not to break" and vary what you implement – Caius Jard Sep 04 '21 at 12:28
  • 1
    *eventually get numb to all the changes and stop spending the time and effort to do so* - maybe you need a break, or perhaps a checklist to follow so that you can check off the steps and get a slight reward from doing so. – Andrew Morton Sep 04 '21 at 18:42
  • I don't know the specifics of the change you refer to, but I can speculate. One possibility is that in the old product, it wasn't clear which of these two behaviours occurred; perhaps they both occurred under different circumstances, and it was hard for users to work out which behaviour occurred when, and even harder for them to get the behaviour they wanted. The existing behaviour might have been so arbitrary and implementation-dependent that keeping the rules the same in the next release was well-nigh impossible. So they made the user choose. – Michael Kay Sep 05 '21 at 22:02
  • "you eventually get numb to all the changes and stop spending the time and effort to do so" Maybe you should consider a different career if you no longer care. I care about producing code that works. That means tracing down all the little bugs and annoying edge cases so that I never find out about a bug from a user. I care about the users, which means I have to care about breaking changes as well. – CJ Dennis Sep 06 '21 at 01:21

7 Answers7

187

A major version upgrade literally means they intend to break things. You shouldn't upgrade to a new major version unless you're prepared to deal with it. Most build systems have a way to specify you're okay with automatic upgrades to minor versions, but not to major versions.

APIs break for a number of reasons. In this case, I'm guessing it's because what they would want to set the default to would be surprising to some users, either because it's not a typical convention for the language, or because of history with this library. This way, instead of half the users suddenly getting a difficult to explain "file is closed" error whose reason is difficult to find in the release notes, everyone gets a "missing parameter" error that they can easily look up the purpose of.

Remember, not everyone uses the library the same way as you. When you have a diverse user base, you have to make compromises in the API to accommodate everyone. A change that seems unnecessary to you might be just what another user has been waiting for.

Karl Bielefeldt
  • 146,727
  • 38
  • 279
  • 479
  • 60
    Along the lines of what you've said here, the reality is that people make mistakes, including library authors. Once you've published a mistake like this in your API, it's difficult to resolve without breaking clients. A major release is when such things are addressed by responsible library authors. – JimmyJames Sep 02 '21 at 18:59
  • 52
    Btw, "Major is going to break things" (& Minor is adding new things but backward compatible and Patch is just bugfix) is called [Semantic Versioning](https://semver.org/) – Jan 'splite' K. Sep 03 '21 at 07:42
  • 2
    What infuriates me is that they then drop the old library. "Ok, now that GTK2 is out, no need to ship GTK+ any more". Or "Now that Java16 is out, no need to ship Java15 any more" — yes, there bloody well is; our build system won't work with Java16. – Edward Falk Sep 04 '21 at 06:51
  • 6
    @EdwardFalk Especially for major libraries that you depend on, you are responsible for knowing their published release plans. For instance, you should know that Java has LTS versions and non-LTS versions, and non-LTS versions will _not_ be supported after the next release. If you need LTS support, use an LTS version. Personally, I'd hate if more languages/libraries took the Python approach of "support the old version forever because people can't be asked to upgrade, splitting the community as people continue to make new code on the old version". Either upgrade, or accept you don't get updates. – Azuaron Sep 04 '21 at 10:38
  • 2
    @EdwardFalk as long as it's OS, there is no dropping. What was made remains available and usable forever. Of course, people that put out free libraries are not obliged to keep work on some legacy version because you want new features there :) – Džuris Sep 04 '21 at 15:49
  • @Azuaron: There are legitimate reasons to support older versions for at least a very long time - I would question whether Python's approach is supporting the old version *forever*, as in some cases, its that libraries that people use haven't yet upgraded to the newer version, or are experiencing instability in the newer versions. While that's true, there's no incentive to update until those downstream issues are fixed. And hesitancy is involved in case 2.x -> 3.x leads to an immediate jump a few months later to 3.x -> 4.x due to breaking changes required to correct defects. – Alexander The 1st Sep 05 '21 at 10:12
  • @AlexanderThe1st There are, and many libraries do. And regardless of updates, you can always use old versions. But it's still the library's consumers' responsibility to know any published release plans, to upgrade instead of demanding minor updates on ancient versions, and to stick to LTS versions if they want LTS support instead of demanding non-LTS versions get LTS support. And, if you're worried that a library's _major_ release is going to have such giant problems that another _major_ release will be required to fix those problems, you should probably stop using such an unstable library. – Azuaron Sep 05 '21 at 14:31
55

My point, and question, is: Why do library developers knowingly break existing code like this? Why not at least have it default to either true or false, whichever is the most reasonable?

Because sometimes it's better to force someone to explicitly make a newly added choice, as opposed to making it for them and effectively having to guess.

If my usual restaurant tomorrow starts making two versions of the one dish that they had, I want to choose which dish I have from now on. I don't want them to choose for me and then run the risk of me getting a dish that I do not like and did not knowingly order.

And this is not some sort of edge-case of the library, either. It's literally the #1 reason for it to exist in the first place,

This argues in favor of forcing consumers to explicitly make a choice, exactly because this behavior is so essential to the library's purpose.

The fact that two approaches are now implemented suggests that there is merit to either approach, and one is not superior to the other in every possible way. If it was, then only the superior one would have been implemented.

from 1.x to 2.x

Major version updates tend to break stuff. That's why they're major version updates. They are the biggest scope on how a library can be changed.

If this happened in an update from 1.0.1 to 1.0.2, I would agree with you. Breaking existing code should only be done in a major version update.

Is it really reasonable that updating a library (in particular) should break existing code?

If all road infrastructure today still had to support horse-drawn carriages, the development of road infrastructure would be significantly hindered.

If you're never allowed to break anything, you stand in the way of innovation, and this is precisely how things (especially software) die a silent death.

Flater
  • 44,596
  • 8
  • 88
  • 122
  • 2
    Would you like to be asked which knife the chef should use, which shoes he should wear and what he should eat for breakfast on the next day ? More choice isn’t always better, and convenience is sometimes the best option. Imo, OP’s dish was fine and there was no reason to force him to choose again the exact same thing, but with more steps to get there. – Steve Chamaillard Sep 02 '21 at 23:26
  • If OPs dish was fine, why order something new? The chef needs to ask the customer for how this new item should be prepared, and whatever guess the chef makes will probably be wrong. – Ian Jacobs Sep 03 '21 at 00:39
  • 4
    @SteveChamaillard I'm not saying that consumers should be left to choose every single thing. I'm saying that _when_ there is something the user should configure, that it can sometimes make sense to not assume a default option and instead make the user consciously configure it. And that's not even considering we're talking about a major version upgrade here, where breaking changes of this caliber are well within reason. – Flater Sep 03 '21 at 01:36
  • 12
    @SteveChamaillard Secondly, if OP did not want to deal with the new menu with increased options, he should not have requested menu v2.0 and instead should have stuck with the old one. It's counterintuitive to ask for a (I can't stress this enough: **major**) update and then be upset that things have changed. – Flater Sep 03 '21 at 01:41
  • 3
    The chef analogy fits better if the website developer is the chef, and the end users are the customers. The customers don't care which knife the chef uses, but the chef certainly does, and would likely be very annoyed if the knife manufacturer switches out his knife from under him. – BenM Sep 03 '21 at 03:08
  • 2
    *If all road infrastructure today still had to support horse-drawn carriages, the development of road infrastructure would be significantly hindered.* Not a very good analogy. I saw a horse-drawn carriage in my city a few weeks ago. In fact, nothing has been done to the road infrastructure to hinder the use of horse-drawn carriages. (There are some legal restrictions, of course.) – Steve Summit Sep 03 '21 at 03:31
  • A better analogy would be radio and television broadcasts. For most of the history of both, the broadcast systems were compatible, in spite of the introduction of FM stereo radio, and color television. This preserved consumer's investments, but limited certain kinds of innovation. Finally, in the past few years, "HD" broadcast standards have been adopted — at the cost of obsoleting all old receivers. – Steve Summit Sep 03 '21 at 03:32
  • 11
    @SteveSummit _"I saw a horse-drawn carriage in my city a few weeks ago."_ The question wasn't whether it was possible for a carriage to still go on some roads, but whether carriages were actively considered in the design of the road you saw. _"In fact, nothing has been done to the road infrastructure to hinder the use of horse-drawn carriages."_ You're inverting the logic. I'm saying that the hindrance is to the road infrastructure, not to the carriages. e.g. hooves easily destroy "whisper asphalt" which is used nowadays to combat noise pollution. – Flater Sep 03 '21 at 07:31
  • 2
    @Flater Here's the way to rescue your street-and-carriage analogy, while still accommodating user16508174's point. Suppose you go from roads, to roads with streetcar tracks. You can either put down rails and ties that look like ordinary train tracks, *or* you can embed the tracks in the street, as in fact streetcar tracks typically are. Embedding them is harder, but preserves backwards compatibility by letting existing vehicles continue to use the road. In user16508174's case, however, they were actively hindered: their old code stopped working. – Steve Summit Sep 03 '21 at 12:37
  • 1
    The question boils down to who is hindered more: those making improvements to X, or those who use (and depend) on X. You're saying, those who make improvements to X must not be hindered (as they seek their improvements) by overly restrictive demands for backwards compatibility. user16508174 is wishing that the maintainers of X could have tried harder to maintain backwards compatibility. It's a tradeoff. – Steve Summit Sep 03 '21 at 12:40
  • *The question wasn't whether it was possible for a carriage to still go on some roads* But it was. user16508174 found that it was not possible to use his old code with the new library. – Steve Summit Sep 03 '21 at 12:42
  • @SteveSummit: The question was _why_ library developers break existing code. This is a more general question than simply OP observing that their code broke, which is what your last comment is implying . Innovation is the core answer here. That doesn't mean that an update _has_ to break the code (just like how we don't need to make it impossible for carriages on the road), but it does mean that library developers will at certain points (= major version updates) ignore whether they will break backwards compatibility or not. – Flater Sep 03 '21 at 13:29
  • 6
    I think the point is that, probably, the old version was _wrong_ (say, the dish was palatable but toxic in the long run), and there was no backward compatible solution; the best solutions required choice (say, the toxic ingredient was to be replaced by A or B, but diners have strong opinions about both A and B, or some people are allergic to A or B). – Pablo H Sep 03 '21 at 13:46
  • 1
    @PabloH Right. But yet another option would have been: preserve the legacy `parse` method with its memory-leaking behavior, but deprecate it in favor of a new `parse2` method that takes the explicit keep-it-open-or-not flag. (And, yes, this compromise has a cost: that new name "`parse2`" is invariably ugly. So there's another tradeoff: do you tolerate that ugliness, or do you force user16508174 and every other user to rewrite their code?) – Steve Summit Sep 03 '21 at 15:01
  • 2
    @SteveSummit but when it's deprecated, then it is kind of expected for it to be removed sometime in the future, forcing the user to rewrite their code anyway. – Andrew T. Sep 03 '21 at 15:48
  • 2
    @SteveSummit The tradeoff for a consumer not wanting to change their code is (possibly) locking themselves out of the next major version. This is why old versions remain available well after newer versions have been released. You're also glossing over many side effects, such as in your case now needing to support an old and new method at the same time, and all of the software design that this might entail when there are shared resources between them. In a lot of cases, it makes more sense to simply leave the old version behind (but available in that historical version of the library). – Flater Sep 03 '21 at 18:05
  • @SteveSummit Nothing about what you say is impossible to do as a library developer, but it is not the only way nor standard of good library development. Supporting deprecated code well past its prime is a significant burden on a library developer. – Flater Sep 03 '21 at 18:06
  • @Flater Yes, of course I'm glossing over side points. As I've said here and elsewhere, it's a tradeoff. A library (or other resource) developer can work impossibly hard at maintaining perfect and eternal backwards compatibility, or can make gratuitous breaking changes every month, or anywhere in between. What I am attempting to argue here is simply that it is legitimate for the consumer of a resource to wish for a certain amount of stability, that it's wrong to expect consumers to happily swallow infinite churn in the name of progress. – Steve Summit Sep 03 '21 at 19:11
  • 1
    If maintaining a certain amount of backwards compatibility would cost the developer of a resource 10 work units, and if there are 100 consumers each of whom will have to expend 1 work unit to accommodate a breaking change, then in the grand scheme of things, it would be less work for the developer to support the backwards compatibility hack, at least for a couple of years. (But of course if those 100 users aren't paying the resource developer anything, there's no mechanism to force that tradeoff.) – Steve Summit Sep 03 '21 at 19:12
  • 2
    I don’t really get why people feel the need to introduce metaphors like roads or dishes. We’re on a tech site, we can understand the case without making forced real-life analogies. – Sebastiaan van den Broek Sep 04 '21 at 08:08
12

Some good answers already, however, let me add my two cents from some real-world experience.

More than often, though usually acting in good faith, some API designers are pretty ignorant what kind of casding effort such decisions might cause. They have probably a wrong idea about how much client code has to be fixed by such a change and how much organizational and communication effort can be triggered by a single new mandatory attribute. Or, they have an idea, but actually no economical motivation to take care for their libraries user base.

What in a library vendor's own organization may require 10 minutes to fix, because they know their lib well and have access to the full code which relies on it, could require to hire a new developer in another organization, for example.

Over the years, I have seen many of those non-backwards compatible scenarios, and in at least 50% of them, I am sure if the designers had put a little bit more thought into backwards compatibility, they could have saved us a ton of working hours.

Doc Brown
  • 199,015
  • 33
  • 367
  • 565
  • 8
    Assume I am a principal engineer at acme company. I develop the foo project which is very useful to my fellow acme engineers. I open source it as version 1.0. The world loves it. I make some breaking changes that make it even more useful to my employer. Why would I spend even one second thinking about backwards compatibility? I can just release it as 2.0. If you don't want breaking changes then stay on 1.0. If you want the latest and greatest then deal with the breaking changes. If you want the latest and greatest and don't want to deal with breaking changes, then recruit me. – emory Sep 04 '21 at 00:45
  • 6
    The answer to "Why" may be that you want people to keep using your product. The response "if you don't like the changes stay on 1.0" may work for some users, but I'd start looking for a new library if I was forced to stay on an old version because the maintainers kept making changes which weren't backward compatible, but could have been with a bit of thought. In general deliberately forking your user base into version 1 and version 2 users doesn't seem sustainable. – David Waterworth Sep 04 '21 at 04:31
  • 1
    @emory: thanks for the agreement to my answer. – Doc Brown Sep 04 '21 at 06:23
  • 1
    @DavidWaterworth We can partition our user base into: (1) the users who pay the bills; and (2) the free riders. If the bill paying users want the breaking changes then who cares what the free riders think. They are welcome to tag along for the ride, but they do not get control of the steering wheel. – emory Sep 04 '21 at 16:14
  • 2
    @emory: I have seen ignorance about the consequences of missing backwards compatibility not only in open source products, but also in paid products. – Doc Brown Sep 04 '21 at 19:07
  • 1
    @DocBrown you are correct. I was thinking about products like angular and react. Whenever there is a breaking change in one of those products, I do not get angry but philosophical: (1) I am sure that the change makes google and facebook better; and (2) this is part of the reason my company pays me my salary. Maybe someday my company will be acquired by them and the effect on product will be considered before introducing breaking changes - but who knows where I will be then (many acquisitions end with layoffs). So in the meantime I am just thankful for breaking changes. – emory Sep 04 '21 at 20:08
  • 1
    You're basically right that in many cases, library developers have very little understanding of the effect of changes on their users, and have no economic justification for acquiring such an understanding, because their revenue doesn't depend on it. However, I think you underestimate the costs of maintaining a very high level of compatibility (in the way we used to do in mainframe days). It's not so much the financial cost, it's the opportunity cost - it prevents you taking your product forward into the modern age, and means you're losing market share to competitors. – Michael Kay Sep 05 '21 at 21:52
  • @emory if you think that little of your free users why even bother releasing a free product? If it's because you want to convert to your paid product then they're not free-riders are they, they're prospects. And not many companies who ignore or treat prospects with disdain do very well - startups perhaps for a while as they're disrupters but that's about it. – David Waterworth Sep 05 '21 at 22:54
  • 1
    @DavidWaterworth it is the other way around. I am one of the free users and respect the fact that in the grand scheme of things I mean almost nothing to the library developers. When things don't work, I don't get mad. Why would I get mad? If software engineering was easy, why would my employer hire me? – emory Sep 05 '21 at 22:59
  • @emory. Not to mention that you always have the choice of either (a) developing the library yourself, which you generally don't want to do, or (b) not upgrading to every bleeding edge release that comes out. Either way, free is free. – Mad Physicist Sep 06 '21 at 00:25
  • Yes I agree, I don't get mad either. But I think it's wrong to say that you mean almost nothing to library developers, at least not those fo most successful/widely used open source projects. Generally (and it is a generalisation) the most successful open-source projects do care a lot about user feedback. Very few that I use don't try and keep their users happy, the ones that don't care are generally single developer "pet project" types - and I agree in this case yes you either use it as is or find something else. – David Waterworth Sep 06 '21 at 00:26
7

Consider when you'd break things yourself. Off the top of my head, I can think of a few reasons:

  • You're updating the API to:
    • Conform to a language convention
    • Shift to a more applicable programming style (say, have a function return with partial application where it makes sense)
  • You're refactoring some implementation, and that allows for a more effective API.
  • You've updated the implementation to the point that the API is no longer helpful. These updates can be as trivial as renaming a function, which is something everyone's done at one point or another.

All of these are aimed at improving the code quality, and, when it effects you, API quality. Sometimes devs make bad decisions, and sometimes the API changes decrease the quality. But most of the time, these are incremental changes aimed at slowly and incrementally improving code: both internal code providing the API and external code relying on the API.

So what do I do?

Here are the two things to do:

  • Care about updates to the code you rely on. If you read release notes before updating (and potentially breaking), you'll be prepared for the consequences of API changes. This can be a pain, but it's ultimately effort invested in code quality.
  • Automate the above. Lots of build systems (though this is language dependent) have the ability to only update dependencies when they don't break, usually utilizing a machine-readable versioning system like Semantic Version (SemVer). SemVer is really simple: start at 1.0.0 as soon as the API is stable. Increment the third digit (i.e., 1.0.0 -> 1.0.1) on updates that don't change functionality, the second digit on updates that change the API in a backwards compatible way, and the first digit if you break the API. Then, if you rely on code that just had a major update, you'll know that you need to put some time aside to fix it.

That last bullet point is always going to be difficult (and slightly controversial), but, complementary to the first bullet point, can be exceedingly useful.

Essentially: breaking things is usually good: be prepared.

  • 1
    re "Conform to a language convention", I hope people ignore the C2x proposal to recommend that "size" arguments precede the pointers to the objects whose size they're describing. Yeah, it's annoying that writing functions with VLA arguments whose size follows them requires old-style function declaration syntax, but the solution to that would be to fix the rules for ANSI-syntax prototypes to say that array dimension expressions will be evaluated at the start of the function's execution and need not be resolved until then. – supercat Sep 03 '21 at 14:57
2

Ultimately this comes down to the elusive and controversial goal of backwards compatibility.

As a developer of application A, you would like to spend your time doing three things:

  • adding features
  • fixing bugs
  • if you have done your job well, and there are no features to add or bugs to fix, bask in the glow of having written a long-term stable application that continues to be useful to its users even though you don't have to do a thing to maintain it, such that you can move on to bigger and better things.

And if every last bit of the functionality of application A is code you wrote yourself, you might have a hope of achieving this. But to do that you would probably have to reinvent lots of wheels, and that's no good, either. Another fine software engineering principle is reuse, or standing on the shoulders of others. So there is almost certainly at least one resource R that your application A depends on.

So then the question is, as the maintainer(s) of resource R go about their work, probably striving for the same three goals as you — adding features, fixing bugs, trying not to do any extra work — how much are they supposed to worry about you and application A?

You'd like them to worry about you a lot: you'd like perfect backwards compatibility; you'd like them never to make a change that breaks your application. They probably agree that this might be nice, but they probably also claim that it's not realistic: that sometimes, a bugfix or a code refactoring or an evolving need may force them to make a backwards-incompatible change. Or there may be "legacy" features which are on their way through "deprecated" and on the way to "obsolescent", which the maintainers of resource R are finding it's just way too much work to continue to support (which is of course why they're marking those features as "deprecated" or "obsolescent").

The problem is supposed to be ameliorated by a whole separate, third class of software: dependency managers D which are supposed to ease the job of managing dependencies between applications like A and resources like R. Once you've discovered that application A works with R version 1.x but not R version 2.x, and if you don't have the time or energy to rewrite A just now, you can explicitly record this dependency somewhere, and then your users will get a helpful error message telling them that they're screwed after upgrading the shared library for R on their machine.

At the end of the day it's either a tradeoff or a stalemate. The maintainers of R may try, but they're probably not going to manage (they won't have the time or energy) to achieve as high a level of backwards compatibility as the maintainers of A might like. (For any pair {A, R}, of course.) So, once in a while, the maintainers of A are almost inevitably going to be disappointed, to discover that they can't just bask in the glow of having a long-term stable application, because their application has broken, through no fault of their own.

But you have my absolute sympathy, user16508174. Dependency management can be a real nightmare (which of course is why the term dependency hell was coined), and I regularly seethe with impotent rage against it myself. I wish the maintainers of resources R worked harder to maintain backwards compatibility, or better yet, got things right the first time more often so that they didn't get stuck in these binds in the first place. But my wishing for it doesn't make it so, and I have little choice but to resign myself to the occasional (or even pretty regular) disappointment on this score.

The other thing, as you may have noticed from the tone of the answers and comments on your question, is that there may be a certain amount of religious fervor going on here. Not only are you supposed to accept that your application A is going to be broken from time to time by forces over which you have no control, you are not even supposed to complain about it. This is the way of the world. You are supposed to be glad that the maintainers of resource R have the freedom to fix bugs and add features without worrying overmuch about backwards compatibility. You are supposed to celebrate the extra time you get to spend learning how to use dependency managers D and painstakingly recording every last intricate dependency that application A might have. You should not want to bask in the glow of long-term stability; that's a confession of some kind of laziness. You are not supposed to be troubled that, after users upgrade R on their systems to v2.x in order to satisfy the dependencies of some other application B, your application A will be broken, and that those users are about to be pestering you to spend time upgrading A to use Rv2.x whether you wanted to or not. This is, again, the way of the world.

Finally, lest this answer be discounted as mere whining, let me say what I would like: I would like the maintainers of any resource R to work harder at achieving backwards compatibility. I know this is asking a lot; I know you're working hard already, and that there aren't enough hours in the day. But the reason I want this is simple: you are, presumably, maintaining resource R as a service to those who use and depend on it. You are doing your work in order to save them work. Presumably, there are more of them than there is of you. So a relatively small amount of work by you is, presumably, leveraged into a huge time savings on behalf of all your users. And making their lives easier by not forcing backwards-incompatible changes upon them is one way you can achieve this.

Steve Summit
  • 137
  • 4
  • 3
    I think there's some truth in what you say, but the other answers have given good reasons why breaking compatibility can actually be a **service** to users: maintaining the old behaviour doesn't give you your ideal application, it gives you one that **the library author knows is wrong**, but they need your help to fix it. Just as you want library authors to make more effort to maintain compatibility, they want you to make more effort to understand the libraries and why changes need to be made. – IMSoP Sep 04 '21 at 14:06
  • I don't think there's any "religious fervour" about this: it's straight economics. The amount of effort a supplier is prepared to make to reduce costs for their users depends on the economic leverage that the users have over the supplier. In today's climate of open source components or products that sell millions of copies for $50 each, users have very little leverage over the supplier. In the mainframe days when your top ten customers were 90% of your revenue, they had a lot more. – Michael Kay Sep 05 '21 at 21:40
2

One of the few things I remember from my CS undergraduate course - now 50 years ago - is David Wheeler's quote "compatibility means deliberately repeating other people's mistakes". Over time you learn that the original design was wrong. It can be wrong because it creates a security weakness, because it leads to poor performance, because it prevents you adding new features that people need, or because it creates a usability problem and a support hassle. As a library designer, you then have to make the decision whether the costs of breaking compatibility justify the benefits.

One thing I learned when I started doing open source software is that this changes the equation. When the users aren't paying you anything, you don't have the same kind of obligation towards them: you can devote your attention more to future users and less to existing users, and you can avoid the substantial costs of maintaining old interfaces that clutter the code and increase your development and support costs.

Another quote, this one unattributed: "the future is longer than the past". That says that getting it right for future users is more important than reducing the pain of upgrade for existing users.

Michael Kay
  • 185
  • 3
  • 1
    Although I understand the mentality, and that this is the case... They may not have the same *legal* obligation to support existing users/code, but they still have a *moral* "obligation". If I knew in advance that library X is maintained by somebody who sees me as a worthless lowlife whose time and energy can be freely wasted and whose security/application stability doesn't matter just because I'm not paying them money, I would never start using that library or ever support the author in any way. – user16508174 Sep 05 '21 at 21:29
  • @user16508174 you get what you pay for. If you don't want to pay, you can't expect support; OTOH you can keep using the old version as long as you want, or you could write your own implementation and use that. – Jeremy Friesner Sep 05 '21 at 23:46
  • 1
    @user16508174 No, the developer of an open source package has no moral obligation to support the users of that package. I think there's a reasonable moral obligation to make the software do what it says on the tin, but there's no obligation to provide future versions or enhancements, or to make any future versions or enhancements compatible with the original. It's not that the developer sees you as worthless, it's just that by giving you a free lunch today they're not accepting an obligation to give you a free lunch every day for the next five years. – Michael Kay Sep 06 '21 at 11:19
  • Incidentally, commercial vendors also sometimes discontinue products. For example, Microsoft have discontinued development of .NET Framework, and told their users that the future is with .NET Core, which is not compatible. I dare say that some of Microsoft's biggest clients, like airlines and banks, have made their displeasure known. But for individual users, we have no grounds for complaint: it was never part of the contract that development would continue for ever. And if that's true of commercial software, it's certainly true where there is no contract. – Michael Kay Sep 06 '21 at 11:28
1

As someone who's authored dozens of libraries on Nuget and Github, I can tell you that backwards compatability is an important consideration.

Despite some other answers here, I don't believe most libraries authors deliberately set out to break backwards compatibility. Indeed, it is often possible to add many new features and improve existing ones without breaking any existing code that uses the library. And i think you'll find that most library updates do maintain backwards compatibility.

That said, there are times when a better approach is found. And the decision is made that a breaking change provides more advantages than the disadvantages of breaking that backwards compatibility.

I use C# and it allows you to mark elements as obsolete. This generates a compile warning but still compiles, allowing someone more time to refactoring their code.

Jonathan Wood
  • 589
  • 1
  • 3
  • 10
  • I find that many users upgrade in big jumps, they go straight from version N to version N+5. This means that the strategy of marking an interface as deprecated or obsolete in one release and then withdrawing it in the next is less effective than it might be. – Michael Kay Sep 05 '21 at 21:35
  • @MichaelKay: Which is the reason most of my obsolete members still remain in my stuff. No one said you have to remove it in the next version. Obviously, if I *ever* take it out, it could break code. But this provides an approach that can lessen the impact of breaking changes. – Jonathan Wood Sep 05 '21 at 22:03