35

I'm pretty new to TDD approach and my first experiments say that writing 1 line of functional code means writing about 2-3 lines of testing code. So, in case I'm going to write 1000 LOC, the whole codebase including tests is going to be something like ~3500 LOC.

Is this considered normal? What is the ratio in code you write?

Andrey Agibalov
  • 1,564
  • 2
  • 15
  • 25
  • 8
    Yeap, TDD comes with a pricey tag ! – Yusubov Jul 15 '12 at 10:21
  • 6
    your 2-3X number is **conservative** at best, it is actually closer to 4-5X for things like C#/Java, Python/Ruby might be closer to 2-3X, with something terse like Erlang being closer to 1:1. Depends on how **dogmatic** you are about TDD, the more dogmatic, the bigger those ratios get! –  Jul 15 '12 at 10:35
  • 1
    You are hitting upon one of the most contentious issues in TDD. When is it overkill? (personally I dread writing test) – thomas-peter Jul 15 '12 at 10:41
  • 6
    @tomwrong: In Kent's book he cites Ward that you should write tests until your fear transforms into boredom. – herby Jul 15 '12 at 10:54
  • 4
    @ElYusubov: I don't agree that it's "pricey" at all. It seems so for those who still count amount of work in LOCs. But the price is not in LOC, it is in money and time-to-market. And there, TDD is not more pricey than any other reasonable development process. – herby Jul 15 '12 at 10:58
  • 5
    Guys, why do you comment instead of posting answers? What you write makes sense. – Andrey Agibalov Jul 15 '12 at 11:00
  • [Production Code Vs Unit Tests Ratio @ c2.com](http://c2.com/cgi/wiki?ProductionCodeVsUnitTestsRatio) – palacsint Jul 15 '12 at 19:59
  • 1
    @herby Well somebody has to design and write the tests. This *is* pricey. If your test-writing isn’t pricey, you’re doing it wrong. – Konrad Rudolph Jul 16 '12 at 06:55
  • 1
    @KonradRudolph: It isn't when compared to the price of finding errors later, and it also saves a lot more in simplicity of the result system. In TDD, you should grow system from test-to-test, using simplest thing that work. In reality, you end up with simpler implementation. So no, it is not pricey, and no, I am not doing things wrong (I know very well it is hard to write good tests). – herby Jul 16 '12 at 08:34
  • @herby Well your initial comment made it sound as if tests didn’t have any substantial cost, rather than as if the cost of tests was offset by their usefulness. In reality, tests *do* have a cost but yes, their cost is justified due to the savings later. But sometimes the cost of extensive tests can be *very* substantial, to the point where you have to change your application architecture fundamentally (by introducing dependency injection specifically for mocking) to allow for effective testing via mock implementations. I think it’s not constructive to pretend that this cost doesn’t exist. – Konrad Rudolph Jul 16 '12 at 08:55
  • @KonradRudolph: I did not pretend anything, I just put it very densely [the price ... is at ... time-to-market], as is often the case with me; but it meant what I said in the reply to you. – herby Jul 16 '12 at 09:01
  • What I hate the most is that TDD means 'cost', 'projects', 'price', 'growth', 'substantial', 'justified'. We all got to get paid but think back to why you are a programmer, does a 10 year boy want to write tests when he grows up? TDD only makes sense if you are robot. – thomas-peter Jul 16 '12 at 11:58
  • I don't have enough rep to make into an answer, but someone studied a random sample of 1000 rails apps, and found there was, on average, [1.03 lines of test code per line of application code](https://semaphoreci.com/blog/2018/04/11/state-of-testing-in-rails.html). – stevec Jan 12 '22 at 13:14

6 Answers6

18

1:3 is normal with TDD.

From my experience, and also from other citations I remember.

herby
  • 2,734
  • 1
  • 18
  • 25
  • 14
    ...what citations? – TehShrike Jul 15 '12 at 12:34
  • ... vaguely remember ... I already don't remember where it was (maybe in the Kent Beck's TDD By Example, maybe in somewhere in c2.com). I remember the meaning, though, that three times more test code than production code is ok. – herby Jul 15 '12 at 12:36
  • Wow, exactly the same in my experience, too. (I am staring at results from cloc right now and I searched google to find any posts on this ratio) – Nikolay Tsenkov Apr 12 '14 at 10:56
14

There are variations based on different coding styles and languages. However, regardless of the language you use, the biggest variation is you.

Robert Martin once said:

“As the tests get more specific, the code gets more generic.”

That made me think. More specific tests mean more test code. More generic production code means less code, so test/code ratios should rise as the code evolves.

But wait, that's not good either. In some particular cases, for example when you define a certain algorithm, you may have only 6-10 lines of code containing a couple of "if"s, a while and maybe 2-3 recursions. I can tell you, that code will have probably more 100 lines of test code.

In a real project, something bigger than just a few algorithms, the test/code ratio should be somewhere between 1:1 and 2:1. If it gets above 2:1, it's a smell that you have tests that should be refactored or deleted (or maybe code that is hard to test). You should always invest the same amount of care and refactoring in your tests as in your production code.

Any way, the best answer to your question maybe is "Cyclomatic Complexity". The higher the cyclomatic complexity of your method, the exponentially more test you have to write for it to cover all the cases.

Patkos Csaba
  • 2,054
  • 12
  • 16
  • The quote you mentioned describes a thing called [Test Contra-Variance](https://blog.cleancoder.com/uncle-bob/2017/10/03/TestContravariance.html). Uncle Bob discusses it in further detail in the blog post I've linked. – byxor Feb 04 '20 at 15:14
3

The ratio is going to vary depending on the size of your methods. The size of your methods will vary by programming style, language and problem domain.

If your methods are short then 3:1 is reasonable. If your methods are long then 3:1 is on the high side.

So, to answer your question, it depends. :-)

Jon Strayer
  • 666
  • 5
  • 7
  • It depends on what you mean by "methods are long". What image it created in my head, is that methods are unnecessary long, doing too much work and having too much responsibilities (often having too many parameters). In that case, such method needs proportionally more combinations to cover by tests, so I don't think the ratio would change much... – herby Jul 15 '12 at 15:50
  • Assume for a moment you can set up a test, call the method to be tested and check the result in three lines of code. If the method you are testing is one line long (as happens in Scala) then your test to production code ration is 3:1. If the method is six lines long then it's 1:2. Six lines isn't all that long. – Jon Strayer Jul 16 '12 at 16:02
2

For software critical application, the usual ratio is one day of testing for each 10 functional LoC.

And this is not counting TDD which is not about test but about specification.

mouviciel
  • 15,473
  • 1
  • 37
  • 64
1

The size of my test code is about half of what the 'actual' code is overall. Doing otherwise is indicative that your tests are too complex and/or your code is too hard to test and/or your code is too dense/complex.

Or you're simply testing too much, and wasting your time on diminishing returns.

Also see "when is unit testing inappropriate or unnecessary?"

Community
  • 1
Telastyn
  • 108,850
  • 29
  • 239
  • 365
-1

My ratio is around 2-1 to 10-1 (code to test). Make sure you testing is about value/behaviour and not implementation.

Erez A. Korn
  • 99
  • 1