26

At our company we typically make sure that we write an end-to-end test for our websites/web apps. That means we access a URL, fill in a form, submit the form to another URL and check the results of the page. We do this to test form validation, test that the HTML templates have the correct context variables, etc.

We also use it to indirectly test the underlying logic.

I was told by a co-worker that the reason for this is that we can rip out and change the underlying implementation at any point as long as the end-to-end tests pass.

I'm wondering if this sort of decoupling makes sense or if it's just a way to avoid writing tests for smaller units of code?

  • 6
    `was told by a co-worker that the reason for this is that we can rip out and change the underlying implementation at any point as long as the end-to-end tests pass.` -- That's also true of unit tests. It sounds to me like the end-to-end tests are being used as an excuse for not writing unit tests. – Robert Harvey May 21 '13 at 18:03
  • 13
    That is not true of unit tests. Changing/removing/creating methods or classes require updating all unit tests for those methods or classes. Not so in end to end tests unless the end user functionality is changing. As long as it's a system level refactor (no modification of end user functionality) no changes are required to the end-to-end tests. – dietbuddha May 22 '13 at 07:11
  • 2
    @dietbuddha, I think the general concept is true of unit tests, but at a smaller (unit) scope. – Sam Oct 11 '13 at 09:07
  • In the article [Write tests. Not too many. Mostly integration.](https://kentcdodds.com/blog/write-tests) Kent C. Dodds suggested to use Trophy instead of popular [Testing Pyramid](https://martinfowler.com/articles/practical-test-pyramid.html) – Michael Freidgeim May 24 '20 at 12:10

5 Answers5

42

End-to-end tests are also necessary. How else can you know you hooked up all the units together correctly? On very simple code, it is possible to test all the paths through the code with only end-to-end tests, but as you get more layers, it becomes prohibitively more costly to do so.

For example, say you have three layers, each with five possible paths. To test all the paths through the entire application would require 53 end-to-end tests, but you can test all the paths through each unit with only 5·3 unit tests. If you only do end-to-end testing, a lot of paths end up getting neglected, mostly in error handling and boundary conditions.

Karl Bielefeldt
  • 146,727
  • 38
  • 279
  • 479
  • That's a good answer. I see the value of both but seeing the number of possibilities for fully testing all paths with end to end tests is helpful for explaining why unit testing needs to be done more than it is –  May 21 '13 at 18:00
  • 16
    I see unit tests as being valuable mostly because they localize problems quickly. End-to-end are valuable because they give you confidence that everything works together. – Jason Swett May 21 '13 at 19:14
  • Unit tests are also a great way to define the tasks you want to implement, following test driven development. End to end tests are less useful for this because as mentioned in this answer you need far more of them to cover corner cases in units. – James Apr 08 '20 at 14:50
21

Yes, end-to-end tests (or integration tests) make a lot of sense, but so do unit tests, too. Ideally, you have both of them, since both typically catch different kind of bugs. Thus, having end-to-end tests should never be an excuse for not having unit tests.

Doc Brown
  • 199,015
  • 33
  • 367
  • 565
  • 2
    A quick note on wording; even though not an exact science, many people will say that end-to-end tests and integration tests are different things. See http://stackoverflow.com/questions/4904096/whats-the-difference-between-unit-functional-acceptance-and-integration-test – sbrattla Oct 15 '15 at 11:32
7

After a few more years of coding and working on projects I'll provide an answer to my own question.

Yes, you should write unit tests. End to end tests are harder to write and brittle especially if they're relying on UI components.

If you're using a framework like Django or Rails (or your own custom classes) you should have a form class that will handle validation of the form. You'll also have view classes that display rendered templates and the form and handle GET and POST requests.

In an end to end test you would:

  1. get the url
  2. fill in the form with valid data
  3. post the form to the url
  4. check to make sure the database has been updated or some action has been executed as a result of the valid form

You're testing a lot of code and your coverage will be pretty good but you're only testing the happy path when everything goes right. How do you ensure that the form has the right validation in it? What if that form is used on multiple pages? Do you write yet another end to end test?

Let's try this again with unit tests:

  1. test the view GET method
  2. test the view POST method with a fake/mock form
  3. test the form with valid data
  4. test the form with invalid data
  5. test the form's side-effects

By using unit tests, you're testing smaller pieces of code and the tests are specific and easier to write. When you combine this with TDD (Test Driven Development) you get higher quality code.

The ease of writing unit tests shouldn't be dismissed because when you're on a project that has no automated testing, you have to start somewhere. Starting with unit tests is easier and faster and lets you immediately start testing for bugs rather than only for the happy path.

  • another data point, Google Testing Blog says no to more end2end tests: http://googletesting.blogspot.ca/2015/04/just-say-no-to-more-end-to-end-tests.html –  May 22 '15 at 14:12
5

Actually end-to-end tests, or integration tests are more important than unit tests since they assure that you have a fully working system. Unit tests don't cover the integration part of a system, which is a complicated task especially in large projects.

However as it has been said, you should do unit testing also, since it is more complicated to catch the edge cases in integration testing.

Random42
  • 10,370
  • 10
  • 48
  • 65
1

It verifies system behavior, whereas unit tests verify unit behavior. The benefits and costs for each are different. You could do one or the other or both.

At my work we do Acceptance Test Driven Development without unit tests which sounds similar to what you've described. We started out doing both and over time, eventually eliminated unit testing do to the cost exceeding the benefits for us.

That being said I think the cost/benefit for your problem domain and environment should drive the decisions to engage in any development practice, different automated testing practices included. Rather than faith, I prefer all practices to have empirical evidence within the context of there use to back them up.

dietbuddha
  • 8,677
  • 24
  • 36
  • What I'm worried about is that we'll be coding towards acceptance tests which leads to large classes or methods instead of smaller units. –  May 22 '13 at 16:35
  • @omouse: Just because you aren't writing unit tests is no reason not to write good code. However, if you have programmers who are inexperienced or just have bad habits then the benefits may outweigh the costs. In either case you should do the analysis and reduce it to a simple equation. `For Any Practice: Practice iff Benefit > Cost` – dietbuddha Jun 15 '13 at 03:20