Sometimes private functions are simply yet-to-be-extracted internal units of functionality. So why not test them?

Question

Sometimes private functions of a module or class are simply yet-to-be-extracted internal units of functionality, which might deserve their own tests. So why not test them? We will write tests for them later on if/when they're extracted. So why not write the tests now, when they're still part of the same file?

To demonstrate:

First, I wrote module_a. Now I want to write tests for it. I would like to test the 'private' function _private_func. I don't understand why I wouldn't write a test for it, if later I might refactor it to its own internal module anyway, and then write tests for it.

---

Suppose I have a module with the following functions (it could be also be a class):

def public_func(a):
    b = _do_stuff(a)
    return _do_more_stuff(b)

_do_stuff and _do_more_stuff are 'private' functions of the module.

I understand the idea that we should only test the public interface, not the implementation details. However, here's the thing:

_do_stuff and _do_more_stuff contain the majority of the functionality of the module. Each one of them could be a public function of a different, 'internal' module. But they are not yet evolved and large enough to be extracted to separate files.

So testing these functions feels right because they are important units of functionality. If they were in different modules as public functions, we would have tested them. So why not test them when they're not yet (or ever) extracted to a different file?

Answer 1

The need to test is not the same as the need to be public.

Non trivial code needs testing regardless of exposure. Non public behavior doesn't need to exist let alone be tested.

These conflicting views can lead you to want to make every function public or refuse to factor out code into a function unless it will be public.

This is not the answer. Be willing to create private helper functions. Test them through the public interface that uses it as much as possible.

If testing through the public interface doesn't exercise the private function sufficiently the private function is trying to allow to much.

Validation can help narrow what the private function allows. If you can't pass a null going through the public interface you can still throw an exception if one comes through anyway.

Why should you? Why test what you'll never see? Because things change. It may be private now but be public later. The calling code could change. Code that explicitly rejects null makes proper usage and expected state clear.

Of course null could be fine. It's just an example here. But if you expect something, it's useful make that expectation clear.

That might not be the kind of testing you had in mind but hopefully you'll be willing to create private helper functions when appropriate.

The desire to test is good but it shouldn't be the driving force in the design of your public API. Design the public API to be easy to use. It likely won't be if every function is public. The API should be something people can understand how to use without diving into the code. Don't leave such people wondering what this weird helper function is for.

Hiding public helper functions in an internal module is an attempt to respect the need for a clean API while exposing helpers for testing. I won't say this is wrong. You might be taking the first step towards a different architectural layer. But please, master the art of testing private helper functions through the public functions that use them first. That way you won't over use this workaround.

Answer 2

Short answer: No

Longer answer: Yes, but via the public 'API' of your class

The whole idea of private members of a class is that they represent functionality that should be invisible outside the 'unit' of code, however big you want to define that unit to be. In object oriented code that unit often ends up being a class.

You should have your class designed such that it is possible to invoke all the private functionality through various combinations of input state. If you find there isnt a relatively straight forward way to do this, its probably hint that your design needs closer attention.

---

After clarification of the question, this is just a matter of semantics. If the code in question can operate as a separate standalone unit, and is being tested as if it were public code, i cant see any benefit of not moving it into a standalone module. At present, it only serves to confuse future developers (including yourself, in 6 months time), as to why the apparently public code is hidden inside another module.

Answer 3

The whole point of private functions is that they are hidden implementation details that can be changed at will, without changing the public API. For your example code:

def public_func(a):
    b = _do_stuff(a)
    return _do_more_stuff(b)

If you have a series of tests that only use public_func, then if you rewrite it to:

def public_func(a):
    b = _do_all_the_new_stuff(a)
    return _do_all_the_old_stuff(b)

then, as long as the the return result for a particular value of a remains the same, then all your tests will be good. If the return result changes, a test will fail, highlighting the fact that something broke.

This is all a good thing: static public API; well encapsulated inner workings; and robust tests.

However, if you'd written tests for _do_stuff or _do_more_stuff and then made the above change, you'd now have a bunch of broken tests, not because the functionality changed, but because the implementation of that functionality changed. Those tests would need rewriting to work with the new functions, but having got them working, all you'd know is that those tests worked with the new functions. You'd have lost the original tests and so would not know if the behaviour of public_func had changed and so you tests would basically be useless.

This is a bad thing: a changing API; exposed inner workings tightly coupled to tests; and brittle tests that change as soon as changes to the implementation are made.

So no, do not test private functions. Ever.