Wrapping third party library is best practice

Question

I came across with following statements while reading the Clean Code book of Robert C. Martin.

Chapter : 7 : Error Handling
_{Page No : 109}
..In fact, wrapping third-party APIs is a best practice. When you wrap a third-party API, you minimize your dependencies upon it: You can choose to move to a different library in the future without much penalty. Wrapping also makes it easier to mock out third-party calls when you are testing your own code.
One ﬁnal advantage of wrapping is that you aren’t tied to a particular vendor’s API design choices. You can deﬁne an API that you feel comfortable with.

I am quite confused about the bold part.

I am totally clueless how can we move to different library so easily if we wrap the third party library and how testing becomes easy with it ?

@Ewan: so what? The unaccepted answers have high votes, and the questioner of that other question abandoned "Programmers" two years ago. — Doc Brown, Sep 24 '15 at 16:52
@Ewan given that that other question is still open, if that is the case you should provide a complete and current answer there. — , Sep 25 '15 at 13:11
this one has a perfectly good answer now and in my view is a better question — Ewan, Sep 26 '15 at 12:17
@Ewan: fine, still no reason not to mark this as a dupe. Note, we are not *deleting* this question, we simply connect them together. — Doc Brown, Sep 29 '15 at 08:28

score 25 · Accepted Answer · answered Sep 24 '15 at 12:23

25

Imagine you have a complicated library that you depend on. Say the library exposes four different calls to its functionality, and your code base uses each of them three times.

If you use the raw vendor API in your code, then if the vendor library changes or you have to replace it, you'll have to change twelve API calls in your code base. If you had an abstraction layer between your code and their code, you'd have to change four places in the abstraction layer and none at all in your code base. You can see that this saves effort - the more effort the more calls the API offers. And most real-world APIs offer more than four entry points.

answered Sep 24 '15 at 12:23

Kilian Foth

107,706
45
295
310

7

It is very unlikely that the 3rd party libs are so much interchangeable. And even if they were, I'd say that it is a lot easier to do a replace and search using your favourite editor, than to create an abstraction in the first place. – barjak Sep 24 '15 at 13:02
8

The decision to wrap or not to wrap depends highly on the library, sometimes it is a good idea, sometimes a very bad idea. This answer, though not wrong, leaves the impression it is always a good idea to wrap 3rd party libs - and that is definitely not a good advice. – Doc Brown Sep 24 '15 at 16:59
Complicated library with four calls? Doesn't sound like it. Complicated library would be a library with like 178 calls? That would be it. Now, mock that. – KulaGGin Jan 22 '21 at 19:45

score 17 · Answer 2 · answered Sep 24 '15 at 12:51

17

In addition to Killian Foth's very good answer, consider some issues with testing code that uses APIs:

Testing is difficult because the API does something "real". Imagine you want to test your code that uses an API making a credit card payment. If each time you run your code it results in a real transaction, that is going to be a big limit on your testing.
Testing is difficult because you can't control the information the API returns.
- Your code is supposed to do something special on Christmas, but the API only returns the current date. Wait until Christmas to test that feature?
- You want to test handling a million email addresses, but these come from an external platform that only has a few test users. Add a million test users manually to that platform?
- Your code uses a phone GPS API to get the current location and then set the language accordingly. Do you have to actually go to different countries to see if this works?

Many more examples could be created, but you get the idea. All of these issues are solved by wrapping the API. Then you can replace the actual API with a test one that returns anything you want for testing purposes.

answered Sep 24 '15 at 12:51

1

I don't know, the spirit of the advice is more "proactively wrap all your 3rd party lib". I agree it is useful to create an abstraction for one specific use case you want to test, but very rarely will it be a whole library. – barjak Sep 24 '15 at 13:17
1

I think Kilian Foth's answer is only mediocre, not "very good". The testing argument, however, is exactly one of the primary criterions to be checked to make the right decision for or against wrapping. – Doc Brown Sep 24 '15 at 17:01
1

@DocBrown, fair point. I do agree that it is not something you should do *every time*. – Sep 25 '15 at 05:30
@barjak Uncle Bob is the proponent that all of your code should be tested if possible. And this allows you to test it. So you only wrap your 3rd party lib before you start using it because you just wrote a test for your class that uses the interface and you already wrote an interface, a mock and you already tested your class. Now you wrap your library in an adapter, so you can pass the adapter in a constructor(because it accepts an interface). – KulaGGin Jan 22 '21 at 19:51

barjak · Answer 3 · 2015-09-24T12:55:33.793

14

You are right to be suspicious. I haven't read this book, but I personally strongly disagree with this advice.

This approach adds both additional code and additional abtraction.

Code can add bugs and overhead.

Abstractions have their limit, and the poor maintainer now has to understand both the library and the abstraction on top to do its job.

Abtraction is hard: either you mimic the underlying library API almost 1:1, and at this point this "abstraction" is not one anymore. Or you create your own API that will certainly mismatch with the design of the library you are using, or with the future library you will want to use in the future. In the codebase I am working on right now, there is such an abstraction over the XML parser we are using. We have a known performance issue, where the abstraction needs to keep a list of all nodes and iterate over it over and over, to satisfy the abstraction's API. It's been more than 10 years that we depend on this specific 3rd party lib, and we never had the need to change it. Removing this awful abstration is on my todo list...

Also, third party libraries being replaced are an exception and not the rule. This advice is asking you to do some work upfront, with all the disadvantages I outlined above, for a case that is unlikely to happen.

About the "mocking calls to 3rd party lib" part, I am unable to think of a case where I needed to mock a library. I may be a bit shortsighted, here.

edited Sep 24 '15 at 12:55

answered Sep 24 '15 at 12:45

barjak

1,720
12
18

how do you unit test? – Ewan Sep 24 '15 at 13:00
2

The 3rd party libs I use fall either in the "frameworks" category (in my case, Qt), which means that my own code grows inside the framework and it wouldn't make sense to mock it. Or it falls in the "utility" category (zlib, xerces, sha1), and are more an implementation detail for the code I am unit testing. Does this answer your question? – barjak Sep 24 '15 at 13:14
3

yes. That makes sense. I come from the other direction, where the 3rd party libs tend to be interfaces for services, database, payment, messaging etc – Ewan Sep 24 '15 at 13:19
I am using a library that interfaces with a point-of-sale device. I need to mock it to run tests without an actual device. – iceman Sep 24 '15 at 13:31
2

`Also, third party libraries being replaced are an exception and not the rule` Sometimes a newer version of a 3rd party library will feature a breaking change. If your 3rd party vendor released version X and no longer supports version X-1, you will want to upgrade. Wrappers isolate any changes you need to make. I'd add that you should use learning tests to be alerted of any breaking changes. http://www.ibooksonline.com/135/f_0127.html – user2023861 Sep 24 '15 at 14:22
5

Creating an abstraction for a third party library is not as difficult or time consuming as you seem to imagine. It could just be an interface and its implementation which delegate the calls to the actual library. The main benefit is that if you change out the library for another one you don't have to touch anything else but the implementation of your wrapper. If you directly use the library you increase the coupling in your code. You would have to go through your whole code base and make modifications everywhere you use the library - that is not what you want. – Xaver Kapeller Sep 24 '15 at 14:27
2

What about breaking changes if you upgrade to a newer version of a library? There are so many reasons why you would need to change something about the library and by introducing a layer of abstraction here you make your life easier - that is the whole point of this paragraph in the book. And mocking a library is very important if you need to test it, but can't use the library in a unit test - for example if the library does not work on the platform you use to run the unit tests or if the library performs network calls? – Xaver Kapeller Sep 24 '15 at 14:30
2

Introducing layers of abstractions like that should be done everywhere where it makes sense - to reduce the coupling in your code and to reduce the potential for change. Changes to the code base whether it be refactoring or migrating to a different technology are the most part of the work when it comes to maintaining a software and following the advice given by Clean Code helps tremendously with that. – Xaver Kapeller Sep 24 '15 at 14:32
My experience is the opposite: replacing third-party libraries is the rule, not the exception. And it's not always because of a breaking change in an API. Sometimes it's a business decision. ("We got a better deal on service X from vendor Y. How soon can you integrate their SDK?") As @XaverKapeller says, wrapping an API is not difficult. And if you consider the features of three or more alternatives when you design your abstraction (the more different, the better!) there's a very good chance it will accommodate any future library without redesign. – Kevin Krumwiede Sep 25 '15 at 09:08
No, that's not from the case that's unlikely to happen. Uncle Bob is talking from a perspective of a TDD. The case is already happening when you're wrapping your 3rd party lib. You already wrote a test, an interface and a mock. Only now you're wrapping 3rd party lib. – KulaGGin Jan 22 '21 at 19:54

score 8 · Answer 4 · answered Sep 24 '15 at 15:20

Disclaimer: I wrap the middleware of my own company...

There are multiple claims here, so I will address them separately.

Wrapping also makes it easier to mock out third-party calls when you are testing your own code.

There are several benefits here, specifically when mocking just counting the number of calls helps. Some will argue that the number of calls is not functional and therefore should not be tested; however I have seen people introducing changes which multiplied the number of calls by a factor of 10x or 100x, and the performance implications are not trivial. This is especially true when...

... some 3rd party libraries come with a lot of dependencies (network connection, complicated configuration file) and only by mocking can you easily inject errors, among other specific situations you wish to test for.

Of course, at this point, wrapping is more down to get rid of an unwanted dependency (connection, configuration file) than anything else.

..In fact, wrapping third-party APIs is a best practice. When you wrap a third-party API, you minimize your dependencies upon it: You can choose to move to a different library in the future without much penalty.

This should go hand in hand with the DRY principle and with abstraction.

When calling a 3rd party library to parse a XML document, then navigate it to extract some pieces of data, the caller:

is not interested in which 3rd party library you use
is probably not interested in the structure of the XML document

as a result, you will wrap this into some code of your own, which has the double benefits of elevating the abstraction level (it's not a XML document, it's a request to buy 4 vanilla ice creams, with caramel on top).

If you repeatedly decode XML documents, quite rapidly you will naturally fuse parts of the decoding code which comes up over and over. This is just DRY.

As a result, I find than wrapping 3rd party components come naturally: whenever I use the same call twice or thrice, I tend to extract the code in a common function, which somehow becomes a wrapper.

(unasked question about boundaries and business model)

A 3rd party API is a foreigner, it does not speak your language.

There is a boundary between your own code and the 3rd party code: you use different objects. For example, you may model your server as a string (IPv4) and integer (port) but the underlying API expects just a string (IPv4:port), or vice versa. As a result, a transformation needs to be applied when talking to this foreign API.

Ideally, you want to maximize the amount of code dealing with your own data:

you understand it
you control it (invariants are checked and meaningful for your app.)
...

this requires pushing the boundaries of foreign code as far as possible.

Also, because each translation carries its own mismatch impedance risk, you want to reduce the number of places in the code where such translation occur.

In both cases, this argues for a thin layer between your application code and 3rd party APIs.

Shame this is all the way at the bottom. I think it's the best answer. Especially your point about "a 3rd party API is a foreigner". It's hard to strike a balance with wrapping APIs, but the benefits are very clear when testing and adapting data within your own system to feed to the 3rd party API. I see wrappers as a necessary middleman/translator, but in some complex systems where I don't want to re-invent the wheel, I'll often expose the 3rd party API with `getDriver()`. It helps explain to the programmer that they are leaving the safe space of local code and venturing out into the wild. — Ryall, Oct 02 '20 at 10:29

score 7 · Answer 5 · answered Sep 24 '15 at 12:52

I feel that often this practice can be used badly.

For example, say you are using the MongoDB client to access your Mongo database. Wrapping the client itself while still exposing all its functionality is pretty hard. You'll have to implement a query object and transform it to the propitiatory one etc. Its a lot of work and if you do change the client later you may find that the new one follows a completely different pattern and your interfaces and wrapper classes are no longer appropriate.

However, you do want to be able to inject mocked objects etc for testing and generally avoid making other stuff dependent on the thrid party library.

This can still be achieved if you wrap at a higher level, say following the repository pattern. By wrapping at this higher level you avoid having to deal with all the details of the 3rd party library and can expose a limited and constant interface which is relevant to your code, rather than the external code you are wrapping.

I took the quote as advocating wrapping at a higher level, rather than trying to wrap the entire functionality of the library. I agree the latter doesn't make sense. — , Sep 24 '15 at 12:55
you may be right. I've seen the latter done a few times though — Ewan, Sep 24 '15 at 12:58

score 3 · Answer 6 · answered Sep 24 '15 at 14:39

There are differing opinions voiced, but I doubt people fundamentally disagree.

The point is that wrapping APIs can be best practice.

If the interface of the API is trivial, then there's no point in wrapping. Same if the interface is stable, doesn't get in the way of testing, and isn't much more complicated than what you need.
If the interface is complicated and you only use a subset, use a wrapper. For example, an advanced logging framework where you always want to log things in the same format and all you ever use in the app is LogError, LogWarn, LogInfo, and LogDebug. Using a wrapper allows using a mock for testing log messages, and also allows to easily change the implementation
If the interface is complicated because it has to be, and you use all of it, you won't be able to improve the interface and the wrapper will be as complicated as the original interface. That means a wrapper won't offer a benefit when moving to a different library.

Additionally, if your application depends so heavily on a specific library/framework that moving away from it would be equivalent to a rewrite (in terms of code size of replicating the functionality of the framework) then it is kinda pointless to wrap. I build primarily google maps applications: because google maps is the defacto standard in that space any competitor would have to replicate a considerable chunk of their API, and if I wanted to move away otherwise *I'd* have to rewrite it. — Jared Smith, Sep 24 '15 at 14:44

Wrapping third party library is best practice

6 Answers6

Linked