36

Context:

I am currently working on a small project in Python. I commonly structure my classes with some public methods that are documented but mainly deal with the high level concepts (what a user of the class should know and use), and a bunch of hidden (starting with underscore) methods which are in charge of the complex or low level processing.

I know that tests are essential to give confidence in the code and to ensure that any later modification has not broken the previous behaviour.

Problem:

In order to build the higher level public methods on a trusted base, I generally test the private methods. I find it easier to find whether a code modification has introduced regressions and where. It means that those internal tests can breake on minor revisions and will need to be fixed/replaced

But I also know that unit testing private method is at least a disputed concept or more often considered as bad practice. The reason being: only public behaviour should be tested (ref.)

Question:

I do care about following best practices and would like to understand:

  • why is using unit tests on private/hidden methods bad (what is the risk)?
  • what are the best practices when the public methods can use low level and/or complex processing ?

Precisions:

  • it is not a how to question. Python has not true concept of privacy and hidden methods are simply not listed but can be used when you know their name
  • I have never been taught programming rules and patterns: my last classes are from the 80's... I have mainly learned languages by trial and failure and references on Internet (Stack Exchange being my favourite for years)
d219
  • 140
  • 9
Serge Ballesta
  • 479
  • 1
  • 4
  • 10
  • 2
    Possible duplicate of [Testing private methods as protected](https://softwareengineering.stackexchange.com/questions/292087/testing-private-methods-as-protected) – Greg Burghardt Oct 19 '18 at 15:26
  • 3
    OP, where did you hear or read that testing of private methods was considered "bad"? There are different ways to unit test. See [Unit testing, Black-box testing and White-box testing](https://stackoverflow.com/questions/7258524/unit-testing-black-box-testing-and-white-box-testing/7258612). – John Wu Oct 19 '18 at 20:34
  • @JohnWu: I know the difference between White-box and Black-box testing. But even in White-box testing it looks like the need to test private methods is a hint for a design problem. My question is an attempt to understand what are the best paths when I fall there... – Serge Ballesta Oct 22 '18 at 07:41
  • 2
    Again, where did you hear or read that even in White-box testing the need to test private methods is a hint for a design problem? I would like to understand the reasoning behind that belief before attempting an answer. – John Wu Oct 22 '18 at 07:57
  • @SergeBallesta in other words, put some references to those articles that made you believe that testing private methods is a bad practice. Then explain to us why did you believe them. – Laiv Oct 22 '18 at 12:13
  • @Laiv: almost all the answers to that question say that. And stack exchange is one of my favourite references... – Serge Ballesta Oct 22 '18 at 12:30
  • Just an advice. Critical thinking. Don't blindly believe everything you read, just because it was written in S.E and a bunch of unknown persons agreed with upvote the same answer. Even here, would be good some references to the S.O questions you are referring to. They will bring some "context" to your beliefs. – Laiv Oct 22 '18 at 13:01
  • @Laiv: I'm old enough to know that *rules are to be broken when we come to a special use case*. And I also know that the *most accepted rules* can change over time. But when I find myself following what seems to be often seen as an anti-pattern *with no special reason*, I prefere to have a break and ask about it. I still have questions for my own use case. But at least I now better understand why *iceberg* classes will be harder to maintain because of a higher risk of side effect on a minor (or far) refactoring.... – Serge Ballesta Oct 22 '18 at 13:29
  • ... I also know that it is always possible to write high quality code without following the *most common practices*. It is simply more disturbing for other maintainers/developpers if they want/have to fix, use or add new features to that code. – Serge Ballesta Oct 22 '18 at 13:31
  • @SergeBallesta: NP. – Robert Harvey Oct 22 '18 at 15:26
  • Private methods are not part of the API and therefore not intended to be tested outside the code itself. – Thorbjørn Ravn Andersen Apr 13 '22 at 17:18

6 Answers6

39

A couple of reasons:

  1. Typically when you're tempted to test a class's private method, it's a design smell (iceberg class, not enough reusable public components, etc). There's almost always some "larger" issue at play.

  2. You can test them through the public interface (which is how you want to test them, because that's how the client will call/use them). You can get a false sense of security by seeing the green light on all the passing tests for your private methods. It is much better/safer to test edge cases on your private functions through your public interface.

  3. You risk severe test duplication (tests that look/feel very similar) by testing private methods. This has major consequences when requirements change, as many more tests than necessary will break. It can also put you in a position where it is hard to refactor because of your test suite...which is the ultimate irony, because the test suite is there to help you safely redesign and refactor!

A tip if you're still tempted to test the private parts (don't use it if it bothers you, and YMMV, but it has worked well for me in the past): Sometimes writing unit tests for private functions just to make sure they're working exactly how you think they are can be valuable (especially if you are new to a language). However, after you're sure they work, delete the tests, and always ensure that the public facing tests are solid and will catch if someone makes an egregious change to said private function.

When to test private methods: Since this answer has gotten (somewhat) popular, I feel obligated to mention that a "best practice" is always just that: a "best practice". It doesn't mean you should do it dogmatically or blindly. If you think you should test your private methods and have a legitimate reason (like you're writing characterization tests for a legacy application), then test your private methods. Specific circumstances always trump any general rule or best practice. Just be aware of some of the things that can go wrong (see above).

I have an answer that goes over this in detail on SO which I'll not repeat here: https://stackoverflow.com/questions/105007/should-i-test-private-methods-or-only-public-ones/47401015#47401015

  • 11
    Reason 1: Nebulous. Reason 2: What if your private helper method should not be a part of the public API? Reason 3: Not if you design your class properly. Your last tip: why would I delete a perfectly good test that proves that a method I wrote works? – Robert Harvey Oct 19 '18 at 22:31
  • 6
    @RobertHarvey Reason 2: being indirectly accessible through the public API != being part of the public API. If your private function is not testable through the public API, then perhaps it's a dead code and should be removed? Or your class is indeed an iceberg (reason 1) and should be refactored. – Frax Oct 20 '18 at 18:29
  • 7
    @RobertHarvey if you can’t test a private function through a public API then delete it as it serves no useful purpose. – David Arno Oct 21 '18 at 18:21
  • @Frax, If your private method is not accessible from the public API, it's not API's unit test business to test such method. It's someone else responsibility to do. So, remove the test and refactor the code. These cases don't necessarily point out to dead code, ultimately, unit tests are not meant to provide a high rate of coverage but a high rate of confidence. – Laiv Oct 22 '18 at 12:21
  • 1
    @RobertHarvey 1: Design smells are always somewhat subjective/nebulous, so sure. But I have listed some concrete examples of anti-patterns, and there's more detail in my SO answer. 2: Private methods can't be a part of the public API (by definition: they are private)...so I don't think your question makes much sense. I'm trying to get at the point that if you have something like `bin_search(arr, item)` (public) and `bin_search(arr, item, low, hi)` (private, there are many ways to do bin search), then all you need to test is the public facing one (`bin_search(arr, item)`) – Matt Messersmith Oct 22 '18 at 13:53
  • 3
    @RobertHarvey 3: Firstly, I said *risk*, not *guarantee*. Secondly, claiming "it works if you do it properly" is self-fulfilling. For instance, "You can write an operating system in a single function *if you do it properly*". This isn't false: but it's also not useful. About the tip: You'd delete it because if your implementation changes (i.e. you want to swap out a private impl), then your test suite will get in your way (you'll have a failing test where you shouldn't). – Matt Messersmith Oct 22 '18 at 13:55
  • @Robert I think you know about software development, but these guys know about design patterns :-) I know who I would ask about design patterns, and who I would ask if I need software written. Btw in which languages can you unit test private methods? Swift is the only one I’m aware of. – gnasher729 Oct 22 '18 at 14:16
  • @MattMessersmith: The statement "You can write an operating system in a single function if you do it properly" is meaningless. It's a contradiction, like "plastic glass" or "military intelligence." – Robert Harvey Oct 22 '18 at 15:24
  • 1
    @RobertHarvey A poor example on my part I suppose...but nonetheless the point stands. The Reason 3 comment is still self-fulfilling (and thus shows no evidence for any argument: for or against). – Matt Messersmith Oct 22 '18 at 15:39
  • @gnasher729 I don't think most languages out of the box support unit testing private methods. There's frameworks that can help you do it, though. For instance, `googletest` allows you to write unit tests for private methods in C++ (it does this by making itself a `friend` of the particular class under test using a macro). Java supports reflection (given the JVM has loose security settings), so you can introspect and call private functions in that manner as well. Most scripting languages don't have public/private formally (like Python), but they do "by convention" and so you can call them as-is – Matt Messersmith Oct 22 '18 at 17:28
24

Given that one of the main purposes of unit tests is that you can refactor the internals of your program and then be able to verify that you haven't broken its functionality, it's counterproductive if your unit tests operate at such a fine level of granularity that any change to the program code requires you to rewrite your tests.

Pete
  • 3,181
  • 1
  • 12
  • 18
  • Not sure why your answer has been downvoted. It’s short, to the point and gets the answer 100% correct. – David Arno Oct 19 '18 at 20:05
  • 6
    @DavidArno: Maybe because testing private methods doesn't really have much to do with test granularity. It has everything to do with coupling to implementation details. – Robert Harvey Oct 19 '18 at 22:28
  • 1
    My private methods can be extremely short and single-purpose as well. Why would a test rewrite be necessary? – John Jiang Jan 05 '22 at 18:08
15

Writing unit tests for private methods ties your unit tests to implementation details.

Unit tests should test the behavior of a class at the class's outer surface (it's public API). Unit tests should not have to know anything about the innards of a class. Writing unit tests against a class's implementation details ties your hands when it comes time to refactor. Refactoring is almost certainly going to break those tests, because they're not part of your stable API.

That said, why might you want to write unit tests for your private methods?

There's a natural tension between unit tests and incremental development. Software developers who use a REPL (read-eval-print loop) can attest to how productive it can be to quickly write and test small bits of functionality as you "grow" a class or function. The only good way to do that in environments that are unit test-driven is to write unit tests for private methods, but there's a lot of friction in doing that. Unit tests take time to write, you need an actual method to test against, and your testing framework needs to support the ability to keep the method private so that it doesn't pollute your external API.

Some ecosystems like C# and .NET have ways to create REPL-like environments (tools such as Linqpad do this), but their utility is limited because you don't have access to your project. The immediate window in Visual Studio is inconvenient; it still doesn't have full Intellisense, you have to use fully-qualified names in it, and it triggers a build each time you use it.

Robert Harvey
  • 198,589
  • 55
  • 464
  • 673
  • I've never written them, but I've seen many unit tests that test that `foo()` was called X times with specific arguments. Lots of internal knowledge. – user949300 Oct 19 '18 at 16:24
  • 4
    @user949300 It's a bit hard to debate this without falling into the _no true Scotsman fallacy_, but there are a lot of badly written tests of all kinds. From a unit testing perspective, you should test the public contract of your method without knowing the internal implementation details. Not that asserting that a certain dependency has been called X times is always wrong: there are situations where this makes sense. You just have to make sure that this is an information you actually want to convey in the contract of that unit under test. – Vincent Savard Oct 19 '18 at 16:44
  • @VincentSavard no need to debate, I pretty much agree with you that those "called 3 times" tests are dubious. – user949300 Oct 19 '18 at 19:41
  • Incremental development does not lead to a need to test private methods. There are well established ideas, such as "TDD like you mean it" where the code under test starts life in the test class/module and is moved out, and tests removed/modified when it's "complete" or [Seb Rose's Recycling TDD tests](http://claysnow.co.uk/recycling-tests-in-tdd/) that handle such scenarios. There's few absolutes in software, but never ever test privates is one of them. It's sad that you didn't just stop after your second paragraph as it would have been a good answer if you had. – David Arno Oct 19 '18 at 20:34
  • 5
    @DavidArno: *[shrug]* I've been doing this for awhile now. Unit tests for private methods always worked just fine for me, until Microsoft decided to stop supporting proxy objects in their test framework. Nothing ever exploded as a result. I never ripped a hole in the universe by writing a test for a private method. – Robert Harvey Oct 19 '18 at 22:26
  • Well maybe now is as good a time as any to stop doing it and to try a different approach... – David Arno Oct 19 '18 at 22:28
  • 3
    @DavidArno: Why would I quit using a perfectly good technique that provides me with benefit, just because someone on the internet says its a bad idea without providing any justification? – Robert Harvey Oct 19 '18 at 22:29
  • 3
    The primary benefit I get out of unit tests is to give me a "safety net" that allows me to tinker with my code, and be confident knowing my changes aren't introducing regressions. To that end, testing private helper methods makes it easier to find any such regressions. When I refactor a private helper method and introduce a logic error, I break tests specific to that private method. If my unit tests were more general and only tested the interface of that unit of code, then the problem would be much more obscure to find. – Alexander Oct 19 '18 at 23:16
  • @Alexander But in most cases you should be able to modify these tests to test the same behavior through the public interface. If it means you need too much extra setup, then likely your method would be better off being a separate entity, that could be tested independently of the class it is working for. – Frax Oct 20 '18 at 07:44
  • 2
    @Frax Sure, I could, but by that logic, I should forgo unit tests in favour of system-wide integration tests. After all, "in most cases you should be able to modify these tests to test the same behaviour" – Alexander Oct 20 '18 at 16:12
  • 2
    @Frax If i use some helper function to encapsulate some behaviour, and I want to test its correctness and set up a safety net to catch future regressions, doesn't mean I want this 1-10 line function to go in its own file, or in its own class, etc. That's so much unwarranted boiler plate. That's probably normal in say, the Java world, but to everyone else it looks batshit crazy. – Alexander Oct 20 '18 at 16:21
  • @Alexander You are missing the point. I'm not saying "move the tests to as high level as possible". I'm saying "perhaps you can use the public API and make your test more complete and robust". In my experience the conversion is usually easier than it looks at first (and results in better code and better tests). If it isn't, then it may indicate that your function deserve to exist independently from the class, or at least separately from the class objects. – Frax Oct 20 '18 at 18:08
  • @Alexander And who said anything about adding files and boilerplate? You just add a top level funciton and are done. Minimal boilerplate, and your tests should get a simpler setup (compared to using public API - otherwise just use public API). Java indeed makes it harder to separate small stuff like this, in that case I guess using a public static method is a practical solution. It is different from adding public object method, as it gives no extra access to the objects (I mean, it should be written in such a way that it doesn't). I'm not sure, I'm not very familiar with Java. – Frax Oct 20 '18 at 18:08
  • @Frax Indeed, a top level function or a static method is exactly what I was thinking. What I thought you were saying, and what some people actually believe advocate for, is that private helper functions like this should be extracted into public methods of a new dedicated class, which has its own separate tests. That makes you write stupid stuff like `PrimeChecker.checkIfPrime(i)` – Alexander Oct 20 '18 at 19:28
  • Thank you for your time and nice answer as way as all your comments. Nevertheless, I have chosen to accept another answer, because I now think that I had to test internal methods because of an *iceberg* class design. – Serge Ballesta Oct 22 '18 at 08:15
  • `I now think that I had to test internal methods because of an iceberg class design` this is a poor argument. You are just avoiding one problem implementing another one upon the first. Call it "walkaround", call it "technical debt". If Iceberg classes are a problem for you, solve this problem first, don't cover it with more code. – Laiv Oct 22 '18 at 12:07
  • When I write a unit test, I almost always have to mock all of its dependencies and the interactions with those dependencies, so though I am writing unit tests for public methods only, my test is fully exposed to its implementation details. It is true that I can still only validate the contracts, but when I am already mocking the interactions, why not validate them too? – haridsv Aug 13 '20 at 15:06
7

From my experience I have found that unit testing the internal classes, methods usually means that I have to take the tested functions, classes out. To create another level of abstraction.

This leads to better adherence of the Single Responsiblity Principle.

Robert Andrzejuk
  • 580
  • 2
  • 11
4

I think this is a good question because it exposes a common problem in testing coverage. But a good answer should tell you that the question is not exactly right because, in theory, you should not be able to unit test private methods. That's why they are private.

Maybe a better question would be "What should I do when i want to test private methods?", and the answer is kind of obvious: you should expose them in a way that makes testing possible. Now, this doesn't necessarily mean that you should just make the method public and that is it. Most likely you'll want to do higher abstraction; move to a different library or API so you can do your tests on that library, without exposing that functionality in your main API.

Remember that there is a reason why your methods have different accessibility levels, and you should always think on how your classes are going to be used in the end.

  • https://softwareengineering.stackexchange.com/questions/380287/why-is-unit-testing-private-methods-considered-as-bad-practice/380290?noredirect=1#comment836983_380290 – Robert Harvey Oct 20 '18 at 23:10
  • 1
    I tried to address this in my question by saying that *Python has not true concept of privacy and hidden methods are simply not listed but can be used when you know their name* – Serge Ballesta Oct 22 '18 at 07:51
-1

If testing private methods is not a good practice, then my class should have very simple private methods only (kind of help methods) and public methods (those we can test). Otherwise, and according to the thread of answers, I should probably move some of the complex private methods to an external service and test it by itself which will lead to the exact same number of tests at the end as well as the same impact on what regards eventual future refactoring.

What about if your public methods are complex by nature? Instinctively, I will try to break that public methods into smaller chunks (private or protected methods) and make the implemented business rules testable (usually, I prefer to declare them protected to make them testable) - I know, it is not the same thing as Private but it does the job since I am implementing a service that is exposed via a public interface and therefore, no chances to have classes inherit from it and use, unexpectedly the protected methods.

Finally, if I unit test complex protected/private methods, I ensure that each of them is doing what it intends to do and therefore, the sum of all these methods, in other words, the public methods that use them, will be easiest to debug.

I certainly won't test all private methods. It's much more an exception than I rule - only in case I really need to implement complex business rules in a public method -

So finally.... I read my answer and it looks like the 'It depends'.... because there's no such practice or rule that should be followed blindly. It's all about benefits vs inconvenient balance and this depends on the context and the problem you are trying to solve.