When, Where, and How to Unit Test

Question

I'm very familiar with xUnit frameworks and I try to implement unit tests on every project I start. Somewhere along the way, I realize that I'm writing the same tests over and over again, and then I run into a really difficult to test method or a test involving remote resources, and then I give up for a while. I end up coming back eagerly whenever I can to test simple things, but as soon as I run into harder things to test, I run for the hills with my tail between my legs.

Let's throw in some examples.

1. Long flow of things to do in a method. (How do I test this? Do I just make sure that the code's being called by stubbing things out? How should I rewrite the code to facilitate easier testing?)

   public void doSomething() {
       Object1 value1 = doSomething1(this.getName());
       if (value1.isError()) {
           sendError(new Object1Error(value1.getError().getMessage()));
           return;
       }
   
       Object2 value2 = doSomething2(value1, this.getName());
       if (value2.isError()) {
           sendError(new Object2Error(value2.getError().getMessage()));
           return;
       }
   
       /* ad infinitum... */
   }
   

2. Overloaded methods and constructors.

   public String toJSON() {
       return toJSON(true);
   }
   
   public String toJSON(boolean prettyPrint) {
       /* do work */
       return result;
   } 
   
   /* test */
   @Test public void testToJSON() { ... };
   
   @Test public void testToJSONBoolean() { /* redundant similar test code... */ }
   

3. Remote resources. How do I test that my upload API is working (ie: right request method, host, request content type, payload, etc.) without actually doing an upload? These uploads go to servers outside of my control.

   public void doUpload() throws IOException {
       HttpClient client = new HttpClient();
       PutMethod putMethod = new PutMethod(...);
       putMethod.setRequestHeader(...);
       putMethod.setRequestEntity(new FileInputStreamRequestEntity(...));
   
       /* etc. */
   
       final int responseCode = client.executeMethod(putMethod);
   }
   

4. Testing server interactions with clients.

   /* before we even get here, other interactions need to take place */
   public void updateClientProfile(UserProfile profile) {
       // 1. validate the input
       // 2. update the user in the db
       // 3. generate an email in HTML
       // 4. send the email
       // 5. serve out a view
   }
   

Testing at first glance seems easy, until I get to these kind of situations. How do I keep my motivation for testing and write my code to be more testable?

It seems that the more things that a method needs to do, the exponentially more difficult and frustrating unit tests are to write. Do I test simply that the code is calling the right services and methods? Do I write unit tests simply to prove to myself that my code calls the methods I told it to call, or to determine that the logic is right? Help!

Answer 1

"Don't blame the mirror for the ugly face"

Problems you observe in tests just honestly reflect the problems with the quality of the source code.

It is very (very) unlikely that continuing blindly throwing unit tests on code that you found to be poorly designed will help you make further progress here. I tried that and I failed. I have been ready to convince self that it's my fault, that it's just me unable of making it work, but I dropped that idea after I saw that colleagues much more talented than me are having exactly the same problems (an advantage of working in strong team is that you can easily gauge when things go wrong because of your own limitations and when it happens because things are wrong).

Your best bet would be to get in touch with professional testers, setup a thorough QA to catch functional regressions and after that, refactor the code into something better - into something that makes writing unit tests productive and fun, as it should be.

Quoting self - thing is, approaches applicable with better codebase just don't cut it:

^{With bad code... unit tests do things opposite to how I use them in good code, like breaking at reasonable changes and failing to catch the real mistakes I make. Which is painful but not surprising - what else would one expect from testing units which have bad design to start with? Instead of helping you improve the design, unit tests often work to preserve bad code - eg in my recent maintenance project I was regularly removing large chunks of code which was only referenced from outdated senseless unit tests. BTW I use that knowledge when writing new code: when I find out that my unit tests tend to get too complicated / fragile, this indicates a need to fix some issue with my own design...}

Answer 2

First things first, When is Unit Testing Inappropriate or Unnecessary?

This is subtly different enough though.

I end up coming back eagerly whenever I can to test simple things, but as soon as I run into harder things to test, I run for the hills with my tail between my legs.

This is the biggest problem. If you get nothing else from the answer: focus on this. This is a problem that will impact your programming, your career... everything. The going will get hard, and learning to deal with it effectively is a vital personal skill.

On to the examples:

1 - a method doing a series of operations

Test the individual operations separately. If they can be mocked, mocking them will help test this method. Breaking this method into doing fewer things will help. If appropriate, having a construct that runs a sequence of delegates/function objects in order unless an error occurs will help because then you can test that construct with arbitrary functions that behave better. As an upside, that construct can be reused if you run into a similar pattern in the future.

For this particular example, I would consider refactoring to use exceptions rather than error codes. It depends on what the actual code is doing and the exception support of the language/platform.

2 - a method overload that trivially calls another

Personally I wouldn't test the trivial overload, just the core function. The trivial overload is obviously correct, and if it changes enough to break things then your other tests should fail when the implied true is not correct.

3 - methods using external resources

There are three things to consider here, since it's not clear what the example entails:

1. If you can, mock out the external resources. As long as your code sends the right stuff to the 'known good' API (and reacts correctly to the expected response), then that's good enough.
2. Make sure that you're not testing the 'known good' API.
3. If it's important enough, then it's worthwhile to make automated tests that aren't unit tests. Have a server that you can control and test if the upload works. These are integration tests or system tests or simply automated functional tests depending on their scope and who you're talking to. These are still useful in scenarios like these where you need to test something, but cannot effectively unit test them.

4 - server processing a request

This is similar to some of the earlier advice: Break out the components, test those individually, mock where possible, use integration tests if appropriate.

A note on mocking

This is a point that I've talked about a few times here, but might not be clear what I mean or how it can help you. Mocking.

Mocking is the use of a test-specific object (a mock object) in place of a real object to isolate the code you're looking to test. There are a number of frameworks to automate this process. Personally, I find these to be oftentimes fragile and overly complex. Sometimes they are vital to getting at things you can't get to. The simplest mock object is one you just make and supply.

When designing your test, any time you call something to get data or send data outward, consider a mock object. They allow you to supply the proper data for the test. They allow you to verify that the outward bound data is what it should be.

They are a bit of overhead though, and can make test maintenance more burdensome. If you find yourself needing them a lot, it can be a sign that your code is too widely coupled. Just be aware of these things and weigh them against the upside when doing test design. For things like database access, they are invaluable.

Answer 3

It seems to me that you may be trying to choose your test cases based on the methods that you've already written. This is putting the cart before the horse somewhat; it tends to lead you towards writing tests around what you ALREADY KNOW that the class does, instead of testing what it is required to do. If there is required behaviour that you have failed to implement correctly (or at all), your unit test probably won't find it for you.

So rather than writing unit tests based on the methods that you have implemented, put your source code to one side for a moment. Instead, write test cases based on the behaviour that your class is required to have. Go back to the documented requirements, if you have them, and consider the different scenarios that can relate to each requirement. Every combination of a requirement and a scenario will then give you one xUnit method. And give your xUnit methods names that reflect the intended behaviour of your class, not the name of the method that implements that behaviour - in other words, a test method name like calculatedPriceEqualsQuantityTimesUnitPrice is so much better than testCalculatePrice.

In terms of your specific issues above -

• Your doSomething method could probably do with one test method for each possible behaviour; that is, for each path through the method.
• The documented requirements of your toJSON(Boolean) method will tell you how many test methods you need, and what scenario each should cover. It's OK to have a common method in your test class, that several of these test methods would call. There's probably no need to test toJSON() separately - its logic is really too simple to go wrong.
• For remote resources and for server interactions, the trick is the same. Bundle the code that interacts with the resource (or the server) into a single class, with no logic of its own (a "wrapper class"). The wrapper class will just pass requests straight through to the resource or the server in question, and return whatever response the resource or the server gives. When you unit test the class that USES the resource, you should supply a mock of the wrapper class (that is, a dummy object whose behaviour you can directly control, and whose interactions get recorded so that you can verify them afterwards). Since the wrapper class has no logic of its own, there's nothing to unit test. However, what you can do is write a "low level integration test", which is similar to a unit test, but includes the resource that the wrapper class is wrapping. You can usually write these in xUnit. The purpose of such a test is to ensure that you are interacting with the resource or the server correctly; but you can usually keep the test cases very simple, because each method of the wrapper class should be very simple.

Answer 4

Whenever you hit a difficult-to-test method, it's highly likely that method does too much. Look at this example you gave:

public void updateClientProfile(UserProfile profile) {
    // 1. validate the input
    // 2. update the user in the db
    // 3. generate an email in HTML
    // 4. send the email
    // 5. serve out a view
}

This is doing more than simply updating the client profile. What this method should actually be called is validateProfileAndUpdateIfValidThenEmailUserAndServeView(). This should be your clue that the method is doing too much.

Now you can have this method, but you need to abstract out the different components.

private ValidationService validator; // inject these however you choose
private EmailerService emailer;
private UserProfileRepository users;

public UserClientView processClientUpdateRequest(UserProfile profile) {
    if (!validator.validateUser(profile)) {
        return new FailureUserClientView(profile);
    }

    users.add(profile);
    emailer.notifyUserOfProfileChange();

    return new SuccessUserClientView(profile);
}

See how much easier this is to test now?

You can, for example, stub out your repository and your emailer with objects that do nothing, mock out the validator you can test all of these methods separately) and simply test the validation logic (i.e. the type returned by the method is based on the response from validateUser).

That's two tests.

Add another two to check that, if the validator responds positively, the DB is called and the emailer notified and you're largely done with this method. You still have the emailer and validation services to test (the repository should be a very thin layer to the database, thus shouldn't benefit from unit testing), but each one individually is much easier to handle than the method you gave yourself.

This is the entire principle behind TDD -- write the tests first and you figure this stuff out much earlier, before you've started developing the method itself. You see very quickly that the name of the method is not descriptive and if you make it descriptive then it indicates a method that does too much.

Note, the only problem you describe that isn't solved by mocking and stubbing services is the overloaded method.

Now this is a very different issue. This is when testing becomes dogmatic and becomes more of a hindrance than a benefit. Give your example:

public String toJSON() {
    return toJSON(true);
}

public String toJSON(boolean prettyPrint) {
    /* do work */
    return result;
}

As long as the latter method is tested, you're fine. Writing tests for the outer method is not going to change your design at all, nor is it going to highlight a regression bug later. Thus it serves no purpose.