In TDD, is it bad practice to write helper class for unit tests?

Question

Sometimes when writing a unit test using Test Driven Development (TDD) one may find that checking if the result is the correct one is "more complicated than one line of code", probably deserving its own function.

In that case, writing an auxiliary method or even helper/ extension class could be something that comes to mind. For instance, if you find yourself doing the same operations in many test cases, why not improve the readability of that test by refactoring it?

But then the question rises: If I am writing an utility/helper class to improve how I am writing test cases, should I unit test it? Should I be even writing that helper class at all? Is it a bad practice?

---

To exemplify, I will give a reduced example:

Suppose I have the following public API requirements:

• MyList should allow values to be added to it
• Those values need to be retrieved. Order does not matter.

To satisfy that, I am writing the following test:

public class MyListTests
{
    [Test]
    public void Add_AddingValues_ValuesArePresentInList()
    {
        MyList myList = new MyList();
        myList.Add(3);
        myList.Add(1);
        myList.Add(1);
        myList.Add(2);

        int[] expectedValues = new int[] { 1, 1, 2, 3 };
        // Assert that myList.GetValues have the same elements as expectedValues;  
    }
}

However, // Assert that myList.GetValues have the same elements as expectedValues; can be quite a non trivial task by its own if order does not matter. In fact, one could argue that testing if two enumerable have the same set of elements, including number of occurences, should be its own function.

How one should handle this situation? Should a helper class be implemented to better allow this test case? Should this helper class be united tested?

---

As a side note, both implementations should satisfy the test:

Implementation A:

public class MyList
{
    private List<int> values = new List<int>;

    public void Add(int value)
    {
        values.Add(value);
    }

    public void IReadOnlyCollection<int> GetValues()
    {
        return values;
    }
}

Implementation B:

public class MyList
{
    private List<int> values = new List<int>;

    public void Add(int value)
    {
        List<int> newValueAsList = new List<int>();
        newValueAsList.Add(value);

        this.values = newValueAsList.Concat(this.values);
    }

    public void IReadOnlyCollection<int> GetValues()
    {
        return values;
    }
}

This is a question about testing behavior using TDD. Not implementation. The efficiency of the implementation of MyList is not a concern.

Answer 1

Your example is probably too contrived to demonstrate why helper classes may be useful, and why they may require tests on their own. But I am working right here with a lot of automated tests, all driven by NUnit, and we have several helper classes

• to compare the output of specific data formats in a non-brittle way (proprietary formats, or specific CSV or XML formats, standard CAD file formats, and so on)

• to provide certain kinds of test data (especially when it is simpler and more maintainable to generate the data programmatically than to store the data itself in a serialized form)

• for controlling some of the more complex testing steps

To be honest, most of these helper classes play a bigger role in automated tests which are not really unit tests any more. But nevertheless they are extremely useful for us, some of them would probably deserve unit tests on their own (though I guess we currently don't have many unit tests for those classes), and I don't see any "bad practice" in using or introducing such classes into a sensible testing process, quite the opposite.

Answer 2

This is essentially a question about writing clean tests

To attempt to answer the question, I will use as reference the book Clean Code: A Handbook of Agile Software Craftsmanship

The Chapter 9 of the book talks about Unit Tests and addresses most (if not all) of the questions that were asked.

WHAT utility classes are is a Domain-Specific Testing Language

This is defined as such:

(...) Rather than using the APIs that programmers use to manipulate the system, we build up a set of functions and utilities that make use of those APIs and that make the tests more convenient to write and easier to read. These functions and utilities become a specialized API used by the tests. They are a testing language that programmers use to help themselves to write their tests and to help those who must read those tests later on.

This is exactly what is being asked. The author considers that writing Domain-Specific Testing language is not only an option but also a step toward improving test quality.

WHY one should care about writing those utility classes?

(...) tests must change as the production code evolves. The dirtier the tests, the harder they are to change. The more tangled the test code, the more likely it is that you will spend more time cramming new tests into the suite than it takes to write the new production code. As you modify the production code, old tests start to fail, and the mess in the test code makes it hard to get those tests to pass again. So the tests become viewed as an ever-increasing liability.

The author clearly states:

The moral of the story is simple: Test code is just as important as production code. It is not a second-class citizen. It requires thought, design, and care. It must be kept as clean as production code.

With those quotes, I think it is reasonable that the question In TDD, is it bad practice to write helper class for unit tests? should be handled with just as much care as any code that would be written in production.

What is a clean test?

Three things. Readability, readability, and readability. Readability is perhaps even more important in unit tests than it is in production code. What makes tests readable? The same thing that makes all code readable: clarity, simplicity, and density of expression. In a test you want to say a lot with as few expressions as possible.

With that, it is clear that if there is code in the tests that could have its readability improved, it should be. It will make the tests more readable and hence, improve the tests quality. If you are writing the same 10 line code to compare two lists in a specific way, it would be advisable to refactor those 10 lines in a separate method to increase test readability.

Utility classes improve readability.

WHEN should one care about writing those utility classes?

This testing API is not designed up front; rather it evolves from the continued refactoring of test code that has gotten too tainted by obfuscating detail.

You are not going to design it as part of a public API that needs to be checked by unit tests. This testing API will be designed as the tests grows and as part of the refactor of the tests.

As stated by @Doc Brown:

The TDD cycle ("red-green-refactor") is exactly about evolving components by continued refactoring instead of designing them up-front.

This means that writing the utility classes fall under the third step in the TDD cycle. The refactor step.

WHERE should you put your utility classes?

The X Unit Patterns also define the concept of Test Utility Method and it has the following text in its Implementation Details guidelines:

Writing the reusable Test Utility Method is pretty straight-forward. The bigger question is where we would put it. If the Test Utility Method is only needed in Test Methods in a single Testcase Class (page X) then we can put the Test Utility Method onto that class but if we need it from several classes, the solution is a bit more complicated. It all comes down to type visibility. The client classes need to be able to see the Test Utility Method and the Test Utility Method needs to be able to see all the types and classes it depends on. When it doesn't depend on many or when everything it depends on is visible from a single place, the Test Utility Method can be put into a common Testcase Superclass (page X) we define for our project or company. If it depends on types/classes that cannot be seen from a single place that all the clients can see, it may be necessary to put it on a Test Helper in the appropriate test package or subsystem. In larger systems with many groups of domain objects, it is common to have one Test Helper for each group (package) of related domain objects.

---

Should one Unit Test the utility classes?

This question has already been asked in different shapes, one being:

Unit testing utility classes

The most well voted answer clearly states:

(...) YES, write tests for utility methods. NO, don't try to decouple them from other tests. Simply assume that trivial utility functions work correctly, as verified by their own tests. Doing anything more is more effort for no gain whatsoever.

This is essentially saying that the functionality provided by the utility methods will be tested in conjunction with the unit tests that are being written for the target public API. If there is a bug in the utility method you are using in the test, it should appear as a failed test for the public API you are testing.

This could also be seen when asking the question: Should you unit test your unit tests? Of course not. The whole idea of TDD is writing simple unit tests to verify desired behavior of your target code with the premise that:

• The unit test code should be simple enough for it to be easy to write and verify
• If there is a bug in the test case, it should appear as a failing test against your business logic.

By using the same reasoning, the utility method should be simple enough for verification and be trusted. If there is a bug in it, it should show in the tests that are being written anyway.

The test utility method is a simplification of the tests you are writing and should be handled as such.

Additionally consider the discussion in the following question:

In TDD, should I add unit tests to refactored code?

As stated in many of the answers:

Programmer tests should be sensitive to behavior changes and insensitive to structure changes. -- Kent Beck, 2019

The utility method, being defined as a refactor of duplicated code in unit tests, are by definition not sensitive to behavior changes. Hence, it should not be tested.

Refactoring is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior.

Also:

As unit tests are written specifically to test the behavior, it doesn't make sense for you to require additional unit tests after refactoring.

Refactoring can inherently never lead to needing additional unit tests that were not needed before.

---

Summarizing

• In TDD, is it bad practice to write helper class for unit tests?
  No! Not at all! In fact, it is a good practice as it improve readability, the main quality that test code should have. Just keep in mind that before starting writing many helper classes that will only be used in one place, it is advisable to first have the unit tests working before refactoring duplicated code into helper classes.

• Should testing helper classes be unit tested?
  Not directly. Helper classes only should exist as product of refactoring of unit tests for readability. Hence, just as regular unit tests, they should not be unit tested. Their correct behavior will be indirectly tested by their coupling to the unit tests being written for the target public API. If something is wrong, the tests should fail.

Additionally, do not reinvent the wheel

Unit tests are supposed to be simple. Because of that, many times the language or the test framework being used provides a clean way of achieving the desired duplicated piece of code.

Because of that, prefer the already existing methods to implementing your own utility classes. Do not reinvent the wheel. This whole question about implementing and testing or not utility classes can often be entirely avoided.

For instance, the example of comparing if two collections have the same elements out of order could be solved by a method such as CollectionAssert.AreEquivalent(ICollection, ICollection). (thanks for @Greg Burghardt for pointing that in the comments of the question).

It is not a coincidence that the concept of a Domain-Specific Testing Language was defined. Those utility classes are often better justified when the handle the DOMAIN of the application, not general data structures. For that, there are usually standard implementations.

Answer 3

The short answer is, that a test project isn’t all that different from any other project. In your case it seems that the helper method is generic enough to live outside the specific test. Another option could be to nest it inside the test, if it’s a helper method to that specific test.

On a side note I would like to mention that there are test frameworks (like nUnit), which can test with multiple input/output sets, which seems to be what you are looking for.