7

According to Martin Fowler's article,

https://martinfowler.com/bliki/TestPyramid.html

It is advisable to write more unit tests than integration tests. Does this mean ideally that every unit of work must be written with unit tests including all the edge cases? How about if there seems no point in unit testing, for instance, an authentication module which is heavily dependent on what the database will return and whose only operation is to store the authentication user to its singleton instance?

Is it right to skip unit testing and go straight writing integration tests?

There's only but one reason that I can think of why unit test must still be written for the authentication module and that is to introduce an abstraction layer between the domain layer and persistence layer, so that the authentication is not tightly coupled to the chosen ORM and switching ORM will be easier.

Xegara
  • 181
  • 1
  • 7
  • Possible duplicate of [Evaluating whether to write unit test or integration tests first on blue-sky/prototype projects](https://softwareengineering.stackexchange.com/questions/345780/evaluating-whether-to-write-unit-test-or-integration-tests-first-on-blue-sky-pro) – gnat May 03 '17 at 06:10
  • 3
    see also [Do I need unit test if I already have integration test?](https://softwareengineering.stackexchange.com/questions/204786/do-i-need-unit-test-if-i-already-have-integration-test) – gnat May 03 '17 at 06:10

3 Answers3

9

In my view integration tests are the most important tests, because they are the ones which tell you if your application actually works when its all put together.

However, they have some downsides

  • they require a running system with all dependencies to test against
  • they are slow to run
  • some tests are necessarily destructive and so cant be run against live environments.
  • they tend to be 'top level' tests. they will tell you something is wrong but not what that thing is.

Unit tests don't have these downsides, you can quickly run hundreds of unit tests on a dev machine without deploying the whole app.

So don't skip unit tests just because you have an integration test and don't skip integration tests because you have unit tests.

One trick you can do to save some code is to make your tests work both as unit and integration by allowing them to mock or not mock dependencies depending on an input or setting.

This allows you to run your tests as unit tests quickly to verify the code. Then as integration to verify the setup and deployment.

Edit : example of how to combine integration and unit tests without code duplication.

public class AuthServiceTests
{
    [TestCase("api.myapp.com/auth", "databaseuser", true, Category="Integration")]
    [TestCase("mock", "mockuser", true, Category="Unit")]
    public void IsUserTest(string target, string username, bool expected)
    {
        //SetUp
        if(target != "mock")
        {
            httpClient = new HttpClient(target);
        }
        else
        {
            httpClient = new MockHttpClient();
        }
        var authService = new AuthService(httpClient);

        var actual = authService.IsUser(user); 
//obvs this might fail if the databases changes in the integration version
//or the db isnt setup right, or the connection string in the api is wrong etc

//the mock one will only fail if the code has a bug of some kind, say it doesnt work with long usernames, or emoticons or something

        Assert.AreEqual(expected, actual);
    }
}
Ewan
  • 70,664
  • 5
  • 76
  • 161
  • Are you saying that the notion of not writing possibly duplicating test cases across the unit, integration and e2e test is wrong because these layers of tests are meant to complement each other rather than minding their own business? – Xegara May 03 '17 at 07:35
  • hmmm, a lot of double negatives in that question. I think you need both unit and integration tests. (some of) those tests might test the same function. but you can avoid literal code duplication through data driven tests or other methods – Ewan May 03 '17 at 07:39
  • Let me rephrase my poorly written question: are the different layers of testing meant to complement each other rather than minding their own business such that it is not considered a bad practice to write and test the same function across the different layers of tests? – Xegara May 03 '17 at 07:44
  • ive edited my question to provide an example of what I mean. yes, code duplication should be avoided where possible. I'm not sure i would say layers or compliment, but yes you have to test that a function works both with a mocked setup of known in memory data AND that it works with a real datasource in a deployed envrionment. Although the code may be identical the test is testing for different errors – Ewan May 03 '17 at 08:01
  • So the reason why we should write both unit and integration tests is because we're testing for different errors? – Xegara May 03 '17 at 09:29
  • I think i would put it the other way around. You have many things to test, some are easier to test with integration and some with unit tests. Its possible that there are simple cases where all the things you need to test fall into one category, but unusual. – Ewan May 03 '17 at 10:29
  • for instance If you have a simple repository which works with a particular db. Unit tests where you mock the DB have limited value because the majority of errors are due to generating and running sql. – Ewan May 03 '17 at 10:36
  • Tbh, I am confused as I still cannot see the "why" reason of writing unit, integration and e2e tests if it were not to complement each other since their approach to testing the same modules are different – Xegara May 03 '17 at 10:44
  • I think you are too caught up in what to call the tests, rather than what the thing you need to verify works is. Write test for all the things. what you call them is academic – Ewan May 03 '17 at 10:47
  • I think I can distinguish the difference between the three tests. But what I'm after is the reason why we should write all of the unit test, integration test and e2e test to test one functionality (e.g. authentication module) – Xegara May 03 '17 at 10:55
  • surely I have answered that specific case? your integration tests fail for reasons other than incorrect code. so if you are working on the train, or the environment is down, or your dont know the ip address etc, you cant run them. plus the other listed disadvantages – Ewan May 03 '17 at 10:59
  • Does not that imply that these different types of tests actually "complement" each other in ways that they address certain problems that other types of test fail to address (e.g. units may passed their unit tests but fail their integration tests due to external reasons and integration tests only tell there is a problem but only the unit test can directly pinpoint the exact piece of logic that causes the error)? – Xegara May 03 '17 at 11:08
  • no. in theory you can expose the exact error either way. Indeed there is a strong argument to run any unit test as an integration or end to end test. In practice though this is hard, expensive and time consuming – Ewan May 03 '17 at 11:19
5

In the real world you will almost never get to 100% test coverage and have all corner cases figured out. So you try to cover as much as you can with both unit and integration tests.

The fact that you see no point in writing an unit test because a module is dependant on a third party source, is because your design probably doesn't allow mocking.

If you have a module that has an external dependency (e.g. file system or database), you should be able to emulate that external dependency, and make it return whatever you like. Then, you will have a perfectly valid unit test, that won't rely on the actual presence of the file or record or whatever, and can be run multiple time.

This usually comes with little design cost and improves testability and modularization quite a bit.

For example, in your Authenticator case, your code should recieve a Connection/Database object and query that, so you can pass it your mock database and let it return valid/invalid datas at will.

BgrWorker
  • 1,694
  • 2
  • 9
  • 14
2

How about if there seems no point in unit testing, for instance, an authentication module which is heavily dependent on what the database will return and whose only operation is to store the authentication user to its singleton instance?

This is a false argument. If the sole purpose of the unit is to store and not to validate, the unit test should test just that purpose. Validating what the DB returns is out of scope to this test and should be tested somewhere else. You may be missing a "credential validator".

Part of the test may be however that the unit uses the credential validator before considering itself done.

The formal answer would be that any function that offers nothing to be tested serves no purpose and should not exist.

Martin Maat
  • 18,218
  • 3
  • 30
  • 57
  • By adding a credential validator to the unit as its dependency, do we introduce an abstraction where the credential validator logic may be easily replaced with another implementation (e.g. changing ORM) thereby and effectively decoupling the persistence logic from our domain layer? – Xegara May 03 '17 at 07:05
  • @Xegara I don't know, I was just making up an example that illustrates validation of yet unknown data is probably not the responsibility of the unit you mentioned. Whether it is helpful to introduce an abstract Validator class into your project or not, I cannot tell. Is dirty data a fact of life in your environment, a problem you face a lot that you want to address? Or can/do you validate on entry? Are there different scenarios, can one set of credentials be valid in one and be invalid in another? Those are questions only you can answer. – Martin Maat May 03 '17 at 08:00
  • I'm guessing that your credential validator implements the query needed to validate the given user against the database. If this is the only functionality it has, will you still write a unit test for this class and mock the ORM and create an integration test integrating the two? – Xegara May 03 '17 at 09:32
  • It is my validator now? What a hypothetical validator would do is besides the point. I only brought it up to stress the importance of separation of concerns in the context of testing. – Martin Maat May 03 '17 at 10:02