How to prevent changes to an internal API from breaking other projects?

Question

We have like 20 - 30 independent modules/solutions. Each of these has about 7 - 10 projects with different classes, components, etc. These are all used internal to our company.

Our problem is when we make a change in one module, we then need to make sure to update all other modules which are accessing this code. This is difficult to know because it is in different codebases.

How can we document where all external uses of an API are? Or otherwise prevent small changes from breaking other modules?

Answer 1

The simplest solution, IMO, is to have a decent number of automated tests for each product. When you update a library, you run the test suite for each product. If the tests fail, then you will know which products need to be updated.

Answer 2

I would suggest against trying to document this (at least manually) as if you require people to update it, it'll require a high level of accuracy to work right. But you will never get that level of accuracy because adding this sort of documentation would... not be fun and no one will do it.

A few better options:

• Have a script that generates a usable list of all method calls in all other modules, based on search criteria. Simple, crude, but probably effective.
• Some sort of version system where you don't break backwards compatibility except on major version numbers. So if you change method foo from version 2.1 to to 2.2, all code referencing version 2.X will still work. When you need to break this compatibility upgrade the major version number (so in this case, 3.0) and announce this to all project owners responsible for other modules. Depending on how your release process works this might be simple - or incredibly complicated - to implement.
• Have automated testing or a CI build process where each time you push code a build runs tests of some sort. This will allow you to identify where problems happen. My guess is with what you are describing as the problem you don't have this already in place..
• Some sort of automatic documentation system for all your modules/repositories

You also might consider slowly refactoring the APIs to not be so brittle, but I expect that's outside the scope of what you can reasonably accomplish if you are an individual and have 20+ large scale modules to work with.

Answer 3

First of all, an API with external uses should not change.

As @BryanOakley mentioned, using automated unit-tests is very important and life-saving in such situations. Beyond that, a few suggestions that may (or may not, according to situation) help you

• Many languages (like Java and C#) offers Function/Method Overriding. Languages like Python offers you to pass (unlimited number of) arguments and keyword arguments to a function:

  Java:

  public void disp(char c)
  {
       System.out.println(c);
  }
  
  public void disp(char c, int num)  
  { 
       System.out.println(c + " " + num);
  }
  
  disp("A")
  disp("A", 3)
  

  Python

  def disp(c, *args):
      if args:
          num = args[0]
          print("%s %f" % (c, num))
      else:
          print("%s" % c)
  
  disp("A")
  disp("A", 3)
  

• Many languages offers public, private and protected methods. You can handle the function call in a public function and do the job in private/protected functions.

  In python, there is no public/private definition for methods and functions, but a leading underscore (_) tell that a method is private and should not be used externally. External API calls is handed by a method which is open to outside world and all tasks are done in so called local functions:

  def format_input(a, b, *args, **kwargs):
      # This function is open to anyone. so use *args and **kwargs to get
      # all possible available arguments. Do not change this function
      # definition and function parameters
      a = _evaluate_input(a)
      b  =_evaluate_input(b)
      # c is not used by all APIs and may not documented in all external
      # API docs. So chech if it was sent as a keyword argument. If yes
      # evalaute it too
      if "c" in kwargs:
          c  =_evaluate_input(kwargs.get("c"))
      _check_extra_arguments(args)
      _check_extra_keyward_arguments(kwargs)
  
  def _evaluate_input(value):
      # This is a private method thus should not be called from the
      # outside world. You can change this method and parameter structure 
      # to fit your needs and do not care for outside world since no
      # outer API should use this function directly.
      ...
  
  def _check_extra_arguments(value):
      # We will check if any extra argument is passed and handle them accordingly
      ...
  

As I said, an API definition which is (also) used by external applications should not changed so oftenty. You can look for ways to make your outer functions more flexible so you can change how the API works without breaking the current state.

Answer 4

Our problem is when we make a change in one module, we then need to make sure to update all other modules which are accessing this code. This is difficult to know because it is in different codebases.

I would suggest that this is impossible to know.

You are responsible for the Components and their Interfaces.
You are not responsible for anything and everything that might make use of them.

How can we document where all external uses of an API are? Or otherwise prevent small changes from breaking other modules?

Short answer? Tests.

Write Tests that exercise the published Interfaces. Re-run them whenever you make a change. So long as the tests "Pass", you haven't broken anything. When a test breaks (and it will) either (a) find and fix the problem or (b) if you can justify the change as legitimate, then re-write the test to accommodate it.

Answer 5

I have seen and coded APIs with version numbers in the function paths and/or names.

In this way you can have different API versions available - complete APIs and different version of the functions within an API.

This puts the work of keeping all the API versions in the code for the API - no other applications' code needs to be changed other than the one for which the new API facility was produced.

I think this is especially important when writing APIs that will be used by applications outside of your organisation.

As an example, here is a code sample to send an SMS using the bulksms api:

http://developer.bulksms.com/eapi/code-samples/csharp/send_sms/

from there is the line:

string url = ".../submission/send_sms/2/2.0";

where the 2 and the 2.0 are version numbers of the API.

As this API is intended for use by many Bulk SMS's customers, a change to this API would potentially break many applications and have the support phone ringing off the hook.