Change architecture design to API to reduce coupling

Question

I was reading articles about TDD and I found the following one. https://blog.cleancoder.com/uncle-bob/2017/03/03/TDD-Harms-Architecture.html

In order to answer this question I think someone should read the full article, but in any case I'll quote the part I'm not able to grasp or to picture in my head.

The right side [...] decouples the users from the services by using an API. What’s more, the services implement the API using inheritance, or some other form of polymorphism [...]

...Okay I guess

I want you to make a simple substitution. Look at that diagram, and substitute the word “TEST” for the word “USER” – and then think. Yes. That’s right. Tests need to be designed. Principles of design apply to tests just as much as they apply to regular code. Tests are part of the system; and they must be maintained to the same standards as any other part of the system.

Here is the part I don't understand. I just don't see an API reducing all those dependencies, for me an API just provides endpoints to be called from outside the application... How come that all that coupling is going to automatically dissapear if I change my backend to be connected within an API?

I'm just too blind to see how it would work... I'd love to understand it because I've been using TDD for two years now and I have suffered that pain he talks about whenever I need to refactor my tests just because we perform a clean up in the backend.

Can anybody come up with a snippet showing how it would work or which concept should I focus on to understand it better?

Answer 1

Change breaks things. Hiding change limits how far that breaking goes and makes changing easier.

If you want to look at this in the most abstract way: this is simply indirection.

A famous aphorism of Butler Lampson goes: "All problems in computer science can be solved by another level of indirection" (the "fundamental theorem of software engineering"). This is often deliberately mis-quoted with "abstraction layer" substituted for "level of indirection". An often cited corollary to this is, "...except for the problem of too many layers of indirection."

wikipedia.org - Indirection

Which is simply a way to say, sure you can do this. But it aint free. It comes with a cost.

When you say:

Here is the part I don't understand. I just don't see an API reducing all those dependencies, for me an API just provides endpoints to be called from outside the application... How come that all that coupling is going to automatically disappear if I change my backend to be connected within an API?

You are 100% right. Not a single dependency has been eliminated. The users have exactly the same needs that they had before.

What's different now is the expression of those needs (what you'd look at to learn what they are) is no longer mixed together with service implementation details and scattered over many disparate services. Separating the implementation and the interface isn't simply to make testing or polymorphism easier. It also makes reading code easier. A reviewer of your code now has a tidy place to go to learn exactly what user needs you're offering to satisfy.

And from the user perspective there is less coupling simply because they don't know what is satisfying their needs.

I'm just too blind to see how it would work... I'd love to understand it because I've been using TDD for two years now and I have suffered that pain he talks about whenever I need to refactor my tests just because we perform a clean up in the backend.

TDD causes early pain. Technical debt causes late pain. So long as the early pain is cheaper than the late pain you got a bargain.

But that doesn't mean there isn't some learning to do. TDD can be done in very messed up ways. From locking down things to where change is actually harder with TDD (test against interfaces not implementation details to avoid most of this) to testing requirements that either don't exist anymore or never should have existed in the first place (map tests to requirements if names don't make it obvious).

However, just because you have to refactor tests doesn't mean you did something wrong. Sometimes it means a requirement changed. Sometimes it means you've restructured the code. TDD never promised that tests never change.

In fact, there is a rigorous methodology for refactoring tests. Michael Feathers gave us refactoring against the red bar. Apparently he got it from Elizabeth Keogh. I've talked about it before. It walks you through steps to change a unit test so that the code under test can't sneak away from you.

Which is nice, but again, it isn't free. Change comes at a cost.

Answer 2

I just don't see an API reducing all those dependencies

Because it doesn't. Uncle Bob is wrong there. Let me list some reasons for this:

The very first thing is, Uncle Bob doesn't seem to think in terms of design trade-offs. It seems to be always in absolutes, like the UI must always be separated from the business, database and all other technologies must all be separated from each other, etc. Just take a look at Uncle Bob's code to see how unmaintainable that is.

Architecture is heavily dependent on the requirements. That means you can't make a trade-off "in general" without even looking at what that application is for.

Second, introducing an additional abstraction is not free. Putting a box in a diagram might be easy, but you're bringing a lot of complexity on board. This is also not acknowledged.

Third, it doesn't even decouple anything necessarily. Uncle Bob leaves out all the details about that one additional box. Like, is that API a data-oriented API, like Uncle Bob usually defines them? Because if it is, both sides have to know what that data is and how it works. That would mean that if the "services" change, the "users" have to change anyway, because of the changed semantics and/or protocol. What if the workflow changes, the meaning of some data elements, etc.?

He might mean, that you don't exactly need to know which service you need to call, because you're calling a central thing. That might help you in some situations, but that's a pretty small part of the whole thing.

In general though, I agree with the conclusions. TDD will not get you a good design automatically, it is not a design practice. It is still all up to us.

Answer 3

The decoupling happens because you put interface between the services and users. It is the same rule as for interfaces in java. If you take a java List for example. List is an interface that has some methods. You expect certain behavior from this interface. Let's just say that you have something like this:

public List<String> appendComma(List<String> strings) {
  strings.append(",");
  return strings;
}

Now my question to you is: Is the List an ArrayList or LinkedList?

The answer is simple: You don't care. Which means that you have decoupled implementation from the behavior. If you do TDD this means that you only have to define the behavior of the interface, which means that you can use the same tests for ArrayList and LinkedList or you can make your own implementation of the List if you wanted to, as long as the behavior will be the same.

It is the same if you are talking about some remote APIs. Http API exposes /users/:id - which returns a user with a specified id.

Again my question is: Which service returns the user? Again, You don't care. Implementation of that feature (behavior) is left completely to the provider of that API. It could be implemented with a monolith, as a microservice, users could be stored in SQL database or NoSql database. The point is that you don't need to know that, which decouples your application from the implementation of the API. In fact, the application of the API provider might have been a monolith in the beginning, but he broke it down into microservices as time went on, but you didn't need to change your code.

If I come back to unit tests: You need to write the tests for the interface and not implementation. When you do TDD spend more time to define the interface so it won't be fragile and it won't change often. When that is done, create a class that implements it. After that you can do whatever you want to that implementation as long as it is passing the tests.

Answer 4

I offer you a (hopefully) better illustration:

You are right to be confused, because the original example is somewhat inconsistent, and the explanation that follows it is cursory and depends too much on the context outside of the article.

^{The one point I want to highlight is the fact that the example is supposed to demonstrate decoupling, but it also includes consolidation — an unrelated design change. This is distracting, because the consolidation is very apparent and draws too much attention. The author should've either demonstrated decoupling without introducing unrelated design changes or made it clear why/how that design change was important.}

But, you're also wrong in the assumption that API is necessarily an endpoint. In the context of web development, the API could be the same layer that you plug into a web framework / reverse proxy. Which means that tests may consume it directly, without spinning up a server.

With all that established, I hope the connection with tests is clear now: if the tests are designed to go through the same indirection the user goes, then the tests will benefit from the indirection the same way the user does: rearranging the implementation should have no impact on the tests.

This is another way of saying that high-level test (that go after the public interface) must be preferred to low-level tests (that are concerned with implementation details).