18

I've been watching a few of the Is TDD Dead? talks on youtube, and one of the things that surprised me is Kent Beck seems to acknowledge that there are just some kinds of programs that aren't worth unit testing. For example, right here DHH says that Kent Beck is

... very happy to say "Well, TDD doesn't fit in this case, I'm just going to bail"

It's frustrating to me that Kent Beck seems to acknowledge this, but nobody asks him to elaborate on it or give concrete examples. I'd like to know the situations where Kent Beck thinks TDD is a bad fit. Nobody can read his mind or speak for him, but I'm hoping he's been transparent enough through his books/tweets/whatever for someone to be able to answer.

I'm not necessarily going to take what he says as gospel, but it would be useful to know that the times I've tried TDD and it just felt impossible/useless are situations that he would have bailed on it himself. Or, if it turned out he would have tested that code it'd suggest to me that I was approaching the process very wrong. I also think it would be enlightening to understand why he would bail on such projects.

My opinion on why this is not a duplicate of "When is it appropriate to not unit test?"

After skimming those answers I'm not satisfied. For example, look at UncleBob's answer. He doesn't even acknowledge that such a situation exists. I really think there's value in understanding Kent Beck's position, not just a general, "What's your opinion?" type of question. After all, he's the father of TDD.

Community
  • 1
Daniel Kaplan
  • 6,776
  • 5
  • 32
  • 46
  • 2
    Tests that check that some particular function got called with a particular set of arguments [are more trouble than they're worth](http://programmers.stackexchange.com/a/233954/116461) if you ask me. Or, more generally, anything more complicated than "here's some inputs, is the output right?" I guess. – Doval Jun 11 '14 at 19:11
  • @GlenH7 after skimming those answers I'm not satisfied (but of course, that's just my opinion). For example, look at UncleBob's answer. He doesn't even acknowledge that such a situation exists. I really think there's value in understanding **Kent Beck's** position, not just a general, "What's your opinion?" After all, he's the father of TDD. – Daniel Kaplan Jun 11 '14 at 19:12
  • 2
    @tieTYT have you considered sending him an email and asking that question? –  Jun 11 '14 at 19:23
  • @MichaelT nope, I'll do that now though. Good idea. If he replies I'll post the answer here. – Daniel Kaplan Jun 11 '14 at 19:25
  • 2
    Voting to close as duplicate, since the only other viable question "What did Kent Beck mean?" is only really answerable by Kent Beck (and primarily opinion based for everyone else). – Telastyn Jun 11 '14 at 20:23
  • @Telastyn JoshKelley's answer shows otherwise, but that's your right. Besides, if Kent Beck replies to my email I'll be able to post it as an answer anyway. But I won't be able to do this is the issue is closed. – Daniel Kaplan Jun 11 '14 at 20:27
  • @Telastyn If you're saying, "Kent Beck could just reply to that other question (or that I could do it on his behalf if he replies)" I see your point. – Daniel Kaplan Jun 11 '14 at 20:33
  • 1
    A reason I haven't seen in the answers yet: new technology. If you've never worked with sockets before and you want to TDD an application where you send data across the network, you'll find it hard to come up with a test beforehand since you don't know yet how an implementation works. – Jeroen Vannevel Jun 11 '14 at 23:33

2 Answers2

11

In Extreme Programming Explained, Kent Beck doesn't say much about what specifically to not unit test, but he does give his rationale of what should be tested (p. 116-118 in my edition):

It is impossible to test absolutely everything, without the tests being as complicated and error-prone as the code. It is suicide to test nothing (in this sense of isolated, automatic tests). So, of all the things you can imagine testing, what should you test?

You should test things that might break. If code is so simple that it can't possibly break, and you measure that the code in question doesn't actually break in practice, then you shouldn't write a test for it...

Testing is a bet. The bet pays off when your expectations are violated [when a test that you expect to pass fails, or when a test that you expect to fail passes]... So, if you could, you would only write those tests that pay off. Since you can't know which tests would pay off (if you did, then you would already know and you wouldn't be learning anything), you write tests that might pay off. As you test, you reflect on which kinds of tests tend to pay off and which don't, and you write more of the ones that do pay off, and fewer of the ones that don't.

To summarize his position, he apparently would not recommend testing:

  • Trivially straightforward code
  • Anything that you can't test "without the tests being as complicated and error-prone as the code." (However, he would no doubt strongly recommend various agile, XP, and OO techniques to reduce the complexity so that it can be tested. For example, he writes, "The more experience I get writing tests, the more I discover I can write tests for non-functional requirements - like performance or adherence of code to standards" (p. 45).)
  • Legacy code (code for which to tests exists) that doesn't need to be touched (see p. 126)
  • Anything for which testing doesn't "pay off"

One specific case: Kent Beck has said that he's not a fan of mocks, and he strongly emphasizes that unit tests need to be "independent" (in the sense that tests don't interact with each other; one failing test doesn't cause others to fail). I infer from this that he would not test code that depends heavily on integration or on many objects collaborating together. (Again, though, I'm sure he'd advocate agile, XP, and OO techniques to minimize this.)

Josh Kelley
  • 10,991
  • 7
  • 38
  • 50
  • FWIW, I don't come to your conclusion about many objects collaborating together. I think he advocates independence for test stability and reproducible results. In fact, to me it seems impossible to avoid "many objects collaborating together" unless you were stubbing/mocking. – Daniel Kaplan Jun 11 '14 at 19:53
5

I'm just watching the recorded videos and I think he said something along these lines as criteria for not unit testing:

  • Needs a lot of mocking.
  • Can't figure out a better design.
  • Need to deliver/ship.

In my experience there are at least a few situations where unit testing should be carefully evaluated:

  • The code is trivial. E.g. getters/setters.
  • The code is glue code between multiple external components. This is where you'd have to resort to some extreme mocking to simulate those external components to the extent that your test is meaningless and is a barrier to refactoring. Code that interacts with hardware, databases, network services etc. can be example of this.
  • There is already very good coverage in other tests that hits what is your candidate for unit testing.
  • You can reason about the correctness through some other means.
  • There is no time or business value to write the code + unit tests, e.g. having poor quality code with no tests means your startup will survive to next month because you can demo your product to a VC vs. having nothing and your startup is dead.

Examples of good situations for (unit) testing would be:

  • Shortest path in a graph algorithm (where you would have some sample input graphs where you know the shortest path)
  • AES encrypt/decrypt where you'd use standard test vectors to verify your algorithm is a compliant implementation.
  • h.264 video encoder.

I think they were also pretty clear that what's important is feedback and unit testing is one tool in the tool-box for getting that feedback. They were also very clear that TDD is one particular style of getting feedback and that the style you use also depends on your personality and how you approach problem solving.

Guy Sirton
  • 1,885
  • 13
  • 15
  • 1
    I think the key here is not **unit** testing--if you have to do a lot of mocking it makes a lot more sense to me to do it as an integration test rather than a unit test. – Loren Pechtel Jun 11 '14 at 20:10
  • @LorenPechtel true and I think this is what I'm trying to communicate. Am I not getting that through? It also goes into the question of what exactly is a "unit" and can every design be broken into clean independent units... – Guy Sirton Jun 11 '14 at 20:14