249

I have been learning writing test cases for BDD (Behavior Driven Development) using specflow. If I write comprehensive tests with BDD, is it necessary to write TDD (Test Driven Development) test separately? Is it necessary to write test cases for both TDD and BDD separately, or are they effectively the same thing?

It seems to me that both are same, the only difference being that BDD test cases can be understood by non developers and testers.

arjun
  • 2,695
  • 3
  • 15
  • 10
  • 2
    That's one crucial difference, yes. – Alex Feinman Feb 15 '12 at 17:50
  • 20
    BDD = Behavior Driven Development, for anyone else wondering. TDD = Test Driven Development, for completeness. – zastrowm Apr 15 '13 at 23:45
  • Confusing BDD with TDD is like confusing Macroeconomics with Microeconomics. They are different. BDD = building an understanding of requirements using examples and optionally may be used to drive automated Macro tests. (agilenoir.biz/en/am-i-behavioral-or-not), TDD = writing micro tests to drive writing coding. The Agile Thoughts podcast covers these differences too: agilenoir.biz/en/agilethoughts/test-automation-pyramid-series – Lance Kind Apr 05 '19 at 03:33

8 Answers8

215

The difference between BDD and TDD is that BDD begins with a B and TDD begins with a T. But seriously, the gotcha with TDD is that too many developers focused on the "How" when writing their unit tests, so they ended up with very brittle tests that did nothing more than confirm that the system does what it does.

BDD provides a new vocabulary and thus focus for writing a unit test. Basically it is a feature driven approach to TDD.

Michael Brown
  • 21,684
  • 3
  • 46
  • 83
  • 114
    Exactly. There is *no* difference between BDD and TDD. BDD is TDD done right. TDD done right is BDD. The problem is that doing TDD right is hard, or more precisely *learning* how to do TDD right is hard. The reason is that TDD has absolutely nothing whatsoever to do with testing, but it is hard to understand that important fact when all the terminology is about testing. So, BDD literally is just TDD with all the testing terminology replaced with behavioral examples terminology. It's like that "try not to think of a pink elephant" thing. – Jörg W Mittag Feb 16 '12 at 03:40
  • 6
    I love the "Try not to think of a pink elephant" example because once you tell someone to do that, that's all they can think of LOL. – Michael Brown Feb 28 '13 at 16:13
  • 10
    Agreed. The trouble with TDD/BDD/xDD is that whichever one you come at first makes other approaches seem a little strange, the key point is not to worry about it too much and use whichever approach fits your circumstances best. A very common approach is to use BDD type approaches for the BA's to define your requirements (Given/When/Then in your stories) but the devs themselves don't use a BDD framework. They just convert the reqs. to TDD type approaches as they code and write unit/integration/acceptance tests in a standard xUnit type framework. – Chris Lee Jan 15 '14 at 22:32
  • 1
    there is no difference, one is a subset of another HOWEVER, there is a difference in that your acceptance criteria become living executables that drive your lower-level unit tests...so there's slightly an abstraction in the middle that you have to account for in code, the specifications (specs) that need to call your SUT code after your lower-level tests pass, the spec should pass. So the spec is what's introduced into BDD so you can't say there is no difference concretely. Sure you test drive the code using lower-level unit tests in both BUT there is also that added layer of specs on top – WeDoTDD.com Aug 07 '15 at 06:42
  • @JörgWMittag "The reason is that TDD has absolutely nothing whatsoever to do with testing". How is this true considering that a fundamental part of the TDD process is writing and running tests? Seems to have rather a lot to do with testing. – DBedrenko Nov 23 '15 at 14:05
  • @DanB I'll give answering that a shot - I agree with Jörg's comment. Implicit in his comment is a heavy emphasis on the word 'development' - not development of the tests, but just development in the standard sense (of the application). As an approach to development, the details then become secondary (in this case, the detail that 'writing tests' are used as the approach to this type of development). In other words, Jörg's comments apply to the high-level view, where 'writing tests' is not part of the picture. Only at the level of detail do writing tests enter the picture. – Dan Nissenbaum Feb 23 '17 at 15:38
  • 1
    TDD and BDD are very different. The first is only meant to be used/done by software developers, while the second is meant to actively involve both developers and "customers" (end users, business experts, business analysts, or other project stakeholders who do not read/write programming code) in the creation of tests. And TDD is not only applicable to "unit" tests, but also to integration, functional, or end-to-end tests; as long as the tests are only read and written by developers, it's TDD; otherwise it's BDD. – Rogério May 16 '18 at 21:10
  • This answer is mostly right. The poster should edit out the bias that TDD produces “brittle tests.” Perhaps programmers make the tests brittle but the process doesn’t. The Agile Thoughts podcast covers TDD here http://agilenoir.biz/en/agilethoughts/why-developers-dont-tdd-a-radio-drama/ – Lance Kind Apr 05 '19 at 03:35
  • I said that programmers make the tests brittle. I said there is no difference between the two just that BDD helps put the focus on behavior in their tests to help them write better tests. – Michael Brown Apr 06 '19 at 07:00
52

Behavior Driven Development is an extension/revision of Test Driven Development. Its purpose is to help the folks devising the system (i.e., the developers) identify appropriate tests to write -- that is, tests that reflect the behavior desired by the stakeholders. The effect ends up being the same -- develop the test and then develop the code/system that passes the test. The hope in BDD is that the tests are actually useful in showing that the system meets the requirements.

UPDATE

Units of code (individual methods) may be too granular to represent the behavior represented by the behavioral tests, but you should still test them with unit tests to guarantee they function appropriately. If this is what you mean by "TDD" tests, then yes, you still need them.

Matthew Flynn
  • 13,345
  • 2
  • 38
  • 57
  • How about larger and complicated projects. Is better to split up the BDD and TDD aspects into their own projects and test seperately? – arjun Feb 15 '12 at 17:26
  • The behavior itself can be broken into granular test cases and tested in BDD. I am confused when people talk about splitting BDD and TDD aspects – arjun Feb 15 '12 at 17:32
  • 4
    You're right--the unit tests still test the behavior of the method. I think the confusion is probably because it may be difficult to trace the behavior of the method directly to the behavior that the external stakeholders asked for and/or understand. – Matthew Flynn Feb 15 '12 at 17:36
  • As for splitting the two into different projects--it would depend on whether the presence of the "techie" unit tests would confuse the folks who need to review the tests for the "behavior" mapped to the requirements. If it's useful to separate them, go ahead. Otherwise, I wouldn't bother. – Matthew Flynn Feb 15 '12 at 17:41
  • @MatthewFlynn traceability between behaviour and the code is definitely the issue, to do that you need to understand the design (not something the stakeholder may be capable of) – jk. Feb 15 '12 at 19:08
  • @jk. All the code in question should be covered by the behavioral tests that are mapped to the requirements. The difference is that these may be at some less-granular level than the individual methods that are called do elicit that behavior. I think there is value in unit testing the individual methods, even if they are only a small piece of what's needed to verify the desired behavior. – Matthew Flynn Feb 15 '12 at 19:34
26

BDD utilizes something called a "Ubiquitous Language," a body of knowledge that can be understood by both the developer and the customer. This ubiquitous language is used to shape and develop the requirements and testing needed, at the level of the customer's understanding.

Within the confines of the requirements and testing dictated by BDD, you will use "ordinary" TDD to develop the software. The unit tests so created will serve as a test suite for your implementing code, while the BDD tests will function more or less as acceptance tests for the customer.

Robert Harvey
  • 198,589
  • 55
  • 464
  • 673
22

The differences between TDD and BDD are subtle and mostly boil down to language. BDD tests are often written in the following form:

public void shouldBuyBread() throws Exception {
   //given  
   given(seller.askForBread()).willReturn(new Bread());

   //when
   Goods goods = shop.buyBread();

   //then
   assertThat(goods, containBread());
 }

If you frame the test in terms of behavior it helps scope the responsibility of the class and leads to better design (at least according to BDD'ers). BDD sometimes focuses on executable specifications that your domain experts/customers can understand.

BDD is also more associated with what Martin Fowler calls 'outside-in' or 'mockist' tests, as opposed to state-based verification.

Garrett Hall
  • 2,182
  • 1
  • 15
  • 16
21

In my experience the biggest problem with TDD is the "T". It causes the lay-person (managers, testers, non-TDD devs) to equate it in their minds with the traditional post-development "Testing" phase of a waterfall style. That is something that anyone can get their head around.

The problem that many struggle with is that TDD is for developers, not testers. Done right TDD is not primarily a test strategy or an acceptance test tool, but a technique that drives good software design from the ground up - small, loosely coupled classes, clear, well defined interfaces, and continually cleaned code through on-going refactoring. Refactoring that is performed routinely, frequently, and from a position of confidence.

That you happen to end up with a comprehensive test suite that can form part of your CI / build process is a bonus, not the goal.

BDD compliments this by bridging the gap between business requirements and higher-level acceptance tests. It is the satisfying of the BDD suite that scopes out the development process and which determines when the product as a whole has been adequately delivered.

sim303
  • 311
  • 2
  • 2
  • 1
    It would be nice for them to make it to the second letter of the acronym before quitting. "Driven." It's hard to drive something if you're following it around. – tottinge May 22 '15 at 16:56
12

Since my last reply wasn't very successful, I will try a very straightforward approach.

  • Behaviour Driven Development is a subset of Test Driven Development
  • TDD focuses on each and every unit test for every function, doesn't matter what it does. BDD focuses on software that matters
  • Idiom. TDD settles for tests, BDD enforces story telling format

JavaScript Examples

Unit tests in jasmine (BDD)

describe("A suite", function() {
  it("contains spec with an expectation", function() {
    expect(true).toBe(true);
  });
});

Unit tests in jsUnity (TDD)

function test_lists() { assert.areEqual([1, 2, 3], [1, 2, 3]) }

Here are some Python libraries that help create more BDD like tests with unittest frameworks:

  • Lettuce: Cucumber for python
  • HamCrest
percebus
  • 221
  • 2
  • 5
4

BDD adds one more level of abstraction to the tests. The higher level code (usually in txt) describes what the system tests, the lower level code describes how it tests it. So a BDD framework can use a TDD framework in the lower level code.

This helps a lot by staying DRY. By TDD you can easily end up with "wet" tests containing a lot of code duplication, which make them easy to break by refactoring the code and the tests with it. By BDD you have to modify only the lower abstraction level by refactoring the code, so if the specification does not change, the higher level code won't change.

Btw. this is simple Clean Code, usually it is enough to read the high abstraction level stuff to understand what it does, and dig deeper to the lower abstraction level testing code only if you really need it. This makes (test) code easier to understand in general.

A bad example (because it is too simple):

jasmine TDD style

calculator.add.specs

describe("Calculator: add", function (){

    it("should be able to add 2 numbers together", function (){
        var total = add(1, 2);
        expect(total).toBe(3);
    });

    it("should be able to add 3 numbers together", function (){
        var total = add(1, 2, 3);
        expect(total).toBe(6);
    });
});

jasmine-cucumber BDD style

calculator.specs

feature('Calculator: add')
    .scenario('should be able to add 2 numbers together')
        .when('I enter "1"')
        .and('I add "2"')
        .then('I should get "3"')
    .scenario('should be able to add 3 numbers together')
        .when('I enter "1"')
        .and('I add "2"')
        .and('I add "3"')
        .then('I should get "6"')

calculator.steps

featureSteps('Calculator:')
    .before(function(){
        this.values = [];
        this.total = null;
    })
    .when('I enter "(.*)"', function(value){
        this.values.push(Number(value));
    })
    .when('I add "(.*)"', function(value){
        this.values.push(Number(value));
    })
    .then('I should get "(.*)"', function(expectedTotal){
        this.total = add.apply(null, this.values);
        expect(this.total).toBe(Number(expectedTotal));
    });

implementation

calculator.js

function add(){
    var args = Array.prototype.slice.call(arguments);
    var total = 0;
    for (var i in args)
        total += args[i];
    return total;
}

Now if I rename the add() function to sum() I have to change the TDD code in 2 places, while the BDD code only in a single place in the steps file. Ofc. adding one more abstraction level requires more code...

inf3rno
  • 1,209
  • 10
  • 26
2

Just to keep it confusing, realize that a lot of people now associate BDD with cucumber. They consider any tests you write using gherkin-derived syntax to be a BDD check, and anything written using a unit test framework (junit, unittest.py, etc) to be TDD.

It's wrong. The medium doesn't define the message in this case. Nor does the use of mocks (IMHO).

The main difference was already given: BDD is a feature-driven technique, using ubiquitous language and an outside-in approach. We write specifications (which will later run as tests) for a story or a slice of a story. Smallness and concreteness matter, and it is an "explanation by example" of what is needed/expected from this aspect of the story.

BDD specs are frequently written in a gherkin-like language if the people who need to specify the feature do not all read/write code (frequent case in larger orgs or where a real customer is part of the team). Otherwise, gherkin-ese is not necessary. These tend to be the output of the famous 3 Amigos meeting: business, test, dev all do these together. When using a BDD framework (fit, cucumber, etc) the test are slower and require a bit more infrastructure, so we keep these relatively sparse -- a few per story.

TDD is used when creating code. The purpose is to give an orderly progression of writing the feature with lots of opportunity for refactoring and with a reasonable confidence that you're not breaking anything accidentally while writing the new code. It doesn't need to describe features of the system, only of a method (or a few with a common effect). Nobody but devs need to read the tests written here, but they have to run crazy fast and isolate errors. They run crazy fast because there may be many tests per class member function ('method').

If you do BDD at the outer layer, it is still useful to do TDD in the inner loop because of refactoring and the ability to spot and avoid errors more quickly.

Neither is proof of correctness for a large system. Neither replaces pen and perf testing. Neither ensures thread-safety. The tests are aid in developing, not guarantees of perfect bug-proof-ness.

tottinge
  • 403
  • 2
  • 5