37

You can find an endless list of blogs, articles and websites promoting the benefits of unit testing your source code. It's almost guaranteed that the developers who programmed the compilers for Java, C++, C# and other typed languages used unit testing to verify their work.

So why then, despite its popularity, is testing absent from the syntax of these languages?

Microsoft introduced LINQ to C#, so why couldn't they also add testing?

I'm not looking to predict what those language changes would be, but to clarify why they are absent to begin with.

As an example: We know that you can write a for loop without the syntax of the for statement. You could use while or if/goto statements. Someone decided a for statement was more efficient and introduced it into a language.

Why hasn't testing followed the same evolution of programming languages?

Peter Mortensen
  • 1,050
  • 2
  • 12
  • 14
Reactgular
  • 13,040
  • 4
  • 48
  • 81
  • 24
    Is it really better to make a language specification more complex by adding syntax, as opposed to designing a language with simple rules that can easily be extended in a generic way? – KChaloux Jul 03 '14 at 16:58
  • 6
    What do you mean by "at the syntax level"? Do you have any detail/idea about how it would be implemented? – SJuan76 Jul 03 '14 at 16:58
  • @KChaloux: Totally agree with you, even though many mainstream languages are going in the opposite direction. – Giorgio Jul 03 '14 at 17:03
  • 2
    Some languages have Design By Contract (http://en.wikipedia.org/wiki/Design_by_contract) that lets you specify pre- and post-conditions for functions as well as loop invariants. Other languages can gain this feature through frameworks. It doesn't replace unit testing, but can make writing test cases easier, when certain conditions are already checked as a part of your program. Have this as a part of the language is not necessary, as you can write code in this style without any special syntax. – FrustratedWithFormsDesigner Jul 03 '14 at 17:09
  • 21
    Could you give a pseudo-code example of testing as a syntax-supported feature? I'm curious to see what you think it would look like. – FrustratedWithFormsDesigner Jul 03 '14 at 17:11
  • I highly doubt that all three of these languages' common compilers were unit tested, especially C++ - especially not with what we know today as unit tests. – Telastyn Jul 03 '14 at 17:11
  • 12
    @FrustratedWithFormsDesigner [See the D language](http://dlang.org/unittest.html) for an example. – Doval Jul 03 '14 at 17:44
  • 1
    @Doval: Interesting. Maybe you should write a longer answer around this? – FrustratedWithFormsDesigner Jul 03 '14 at 17:48
  • 3
    @FrustratedWithFormsDesigner I could, but for the most part I agree with KChaloux and Robert; rather than adding the feature to the language, it's better to add whatever features are needed to make implementing a good unit testing library easy, and then ship a unit testing package in the standard library. Take a look at Haskell's QuickCheck for example, which is more of *unit test generator* in the sense that you specify the invariants you're testing more than how to test it. E.g. "For all x and y, `f (x, y) == f (y, x)`" and then QuickCheck tries to call `f` with a large number of arguments. – Doval Jul 03 '14 at 17:56
  • 1
    @Doval: The OP seemed to think that *no* language had such features and wondered why that was. You've shown that D has some unit-testing features baked right into the language itself. Answering by explaining why D chose to add such a feature when most lanugages don't would be interesting. I know *I'm* curious about this now... – FrustratedWithFormsDesigner Jul 03 '14 at 18:05
  • 5
    Another language with built-in unit testing features is Rust. – Sebastian Redl Jul 03 '14 at 18:32
  • If I give a language feature example for unit testing, then people will argue my example isn't very good. My question is why hasn't this been added to the language given that unit testing existed long before C# and Java. – Reactgular Jul 03 '14 at 20:43
  • Why it's not a language level feature? Because it doesn't need to be. Why add more syntax than strictly necessary? D's unittest block seems to corresponds to doctest in python rather than traditional unittest. Having special syntax for doctest in python isn't necessary because the language has support for getting docstring during runtime. – Lie Ryan Jul 03 '14 at 23:39
  • You could also use a language with proofs, such as [Coq](https://www.youtube.com/watch?v=CmPw7eo3nQI). – Simon Richter Jul 04 '14 at 10:58
  • Does C's `assert()` count? – goldilocks Jul 04 '14 at 13:35
  • @goldilocks assert is not meant to be for testing but debugging, there's a difference – ratchet freak Jul 04 '14 at 20:28
  • The language [pyret](http://www.pyret.org/) has built in syntactic support for defining unit tested function – Jack Dec 16 '14 at 22:15

6 Answers6

36

As with many things, unit testing is best supported at the library level, not the language level. In particular, C# has numerous Unit Testing libraries available, as well as things that are native to the .NET Framework like Microsoft.VisualStudio.TestTools.UnitTesting.

Each Unit Testing library has a somewhat different testing philosophy and syntax. All things being equal, more choices are better than less. If unit testing were baked into the language, you'd either be locked into the language designer's choices, or you'd be using... a library, and avoiding the language testing features altogether.

Examples

  • Nunit - General purpose, idiomatically-designed unit testing framework that takes full advantage of C#'s language features.

  • Moq - Mocking framework that takes full advantage of lambda expressions and expression trees, without a record/playback metaphor.

There are many other choices. Libraries like Microsoft Fakes can create "shims..." mocks that don't require you to write your classes using interfaces or virtual methods.

Linq isn't a language feature (despite it's name)

Linq is a library feature. We got a lot of new features in the C# language itself for free, like lambda expressions and extension methods, but the actual implementation of Linq is in the .NET Framework.

There's some syntactic sugar that was added to C# to make linq statements cleaner, but that sugar is not required to use linq.

Community
  • 1
Robert Harvey
  • 198,589
  • 55
  • 464
  • 673
  • 1
    I do agree with the point that LINQ isn't a language feature but to make LINQ happen there needed to be some underlying language features to happen -- particularly generics and dynamic types. – Wyatt Barnett Jul 03 '14 at 19:36
  • @WyattBarnett: Yes, and the same is true for easy unit testing - like reflection and attributes. – Doc Brown Jul 03 '14 at 19:41
  • 8
    LINQ needed lambdas and expression trees. Generics already were in C# (and .Net in general, they're present in the CLR), and dynamics have nothing to do with LINQ; you can do LINQ without dynamic. – Arthur van Leeuwen Jul 03 '14 at 19:57
  • Perl's DBIx::Class is an example of LINQ-like functionality implemented more than 10 years after all the features needed to implement it has been in the language. – slebetman Jul 04 '14 at 01:14
  • Can't a language have testing grammar built-in and also support libraries that offer different philosophies. For example; couldn't C# have built-in ways to test internal classes or syntax for mocking objects. – Reactgular Jul 04 '14 at 02:33
  • @ArthurvanLeeuwen I suspect he meant **implicit** types not dynamic types. They were one of the new language features that needed to be added to make LINQ workable, were they not? – Carson63000 Jul 04 '14 at 06:05
  • 2
    @Carson63000 I think you mean anonymous types in combination with the `var` keyword :) – M.Mimpen Jul 04 '14 at 08:12
  • @ArthurvanLeeuwen -- note that there was a time when generics weren't in .NET nor the CLR -- that came with 2.0. 2.0 also saw a lot of proto linq features on your collections, such as List.Find() and List.ForEach() methods that presage much linq function. – Wyatt Barnett Jul 07 '14 at 22:49
21

There's lots of reasons. Eric Lippert has stated many times that the reason feature X isn't in C# is because it's just not in their budget. Language designers don't have an infinite amount of time nor money to implement things, and each new feature has maintenance costs associated with it. Keeping the language as small as possible isn't just easier for the language designers - it's also easier for anyone writing alternative implementations and tools (e.g. IDEs) Additionally, when something is implemented in terms of the language rather than part of it, you get portability for free. If unit testing is implementing as a library, you only need to write it once and it'll work in any conforming implementation of the language.

It's worth noting that D does have syntax-level support for unit testing. I don't know why they decided to throw that in, but it's worth noting that D is meant to be a "high level systems-programming language". The designers wanted it to be viable for the kind of unsafe, low-level code C++ has traditionally been used for, and a mistake in unsafe code is incredibly costly - undefined behavior. So I suppose it made sense to them to expend extra effort on anything that helps you verify that some unsafe code works. For example, you can enforce that only certain trusted modules can perform unsafe operations like unchecked array accesses or pointer arithmetic.

Quick development was also a priority for them, so much so that they made it a design goal that D code compiles fast enough to make it usable as a scripting language. Baking unit tests right into the language so you can run your tests by just passing an extra flag to the compiler helps with that.

However, I think a great unit testing library does much more than just find some methods and run them. Take Haskell's QuickCheck for example, which lets you test things such as "for all x and y, f (x, y) == f (y, x)". QuickCheck is better described as a unit test generator and allows you to test things at a higher level than "for this input, I'm expecting this output". QuickCheck and Linq aren't all that different - they're both domain-specific-languages. So rather than bolting on unit testing support unto a language, why not add the features needed to make DSLs practical? You'll end up with not just unit testing, but a better language as a result.

Doval
  • 15,347
  • 3
  • 43
  • 58
13

Because testing, and particularly test-driven development, is a deeply counter-intuitive phenomenon.

Almost every programmer begins their career believing they are much better at managing complexity than they actually are. The fact that even the greatest programmer cannot write large and complex programs without severe errors unless they use lots of regression tests is seriously disappointing and even shameful to many practitioners. Hence the prevalent disinclination against regular testing even among professionals who should already know better.

I think the fact that religious testing is slowly becoming more mainstream and expected is largely due to the fact that with the explosion in storage capacity and computing power, larger systems than ever are being built - and extremely large systems are particularly prone to complexity collapse that cannot be managed without the safety net of regression tests. As a result, even particularly obstinate and deluded developers now grudgingly admit that they need testing and will always need testing (if this sounds like the confession at an AA meeting, this is quite intentional - needing a safety net is psychologically hard to admit for many individuals).

Most languages popular today date from before this attitude shift, so they have little built-in support for tests: they have assert, but no contracts. I'm fairly sure that if the trend persists, future languages will have more support on the language rather than the library level.

Kilian Foth
  • 107,706
  • 45
  • 295
  • 310
  • 3
    You overstate your case a bit. While unit testing is unquestionably of paramount importance, building programs the *proper* way, the way that minimizes unnecessary complexity, is equally important, if not more so. Languages like Haskell have type systems that improve the reliability and provability of programs written in them, and best practices in coding and design avoid many of the pitfalls of untested code, making unit testing less critical than it otherwise would be. – Robert Harvey Jul 03 '14 at 20:08
  • 1
    @RobertHarve ...and unit testing is a _very_ good way to indirectly push developers towards that. Working on the API while creating tests, rather than while using it with other code, helps to put them in the right mindset to limit or remove leaky abstractions or odd method calls. I don't think KillianFoth is overstating anything. – Izkata Jul 03 '14 at 23:05
  • @Robert: The only thing I beleive he overstates is "religious testing is slowly becoming more mainstream" If slowly is defined in terms of Geographic time frames he is probably not far off..... :) – mattnz Jul 04 '14 at 02:21
  • +1 In my current job, I'm struck by the tidal shift in development attitudes with the newer libraries and technologies available. For the first time in my career, these guys are lecturing *me* to use a more sophisticated development process, instead of me feeling like I'm on top of a hill shouting at the wind. I agree it's just a matter of time before these attitudes find expression in native language syntax. – sea-rob Jul 04 '14 at 02:41
  • 1
    Sorry, one more thought re: Robert Harvey's comment. Right now type systems are a nice check, but really fall amazingly short of defining constraints on types -- they address the interface, but don't constrain correct behavior. (i.e. 1 + 1 == 3 will compile, but may or may not be true, depending on implementation of +) I wonder if the next trend will to see more expressive type systems that combine what we consider to be unit testing now. – sea-rob Jul 04 '14 at 02:44
6

A lot of languages have support for testing. C's asserts are tests that the program can fail. That's where most languages stop, but Eiffel and more recently Ada 2012 have preinvariants (things the arguments to a function must pass) and postinvariants (things the output of a function must pass), with Ada offering the ability to reference the initial arguments in the postinvariant. Ada 2012 also offers type invariants, so whenever a method is called on an Ada class, the type invariant is checked before return.

That's not the type of full-out testing a good testing framework can give you, but it's an important type of testing that languages can best support.

prosfilaes
  • 236
  • 1
  • 5
  • 2
    Having done a little bit of work in Eiffel and being an extensive user of assertions, my experience has been that anything you can do that's contractual in nature helps to ferret out a lot of bugs. – Blrfl Jul 04 '14 at 15:21
2

Some proponents of strongly typed functional languages would argue that these language features reduce or eliminate the need for unit tests.

Two, imho, good example of this for this is from F# for Fun and Profit here and here

Personally, I am still believe in the value of unit tests, but there are some valid points. E.g. if an illegal state is unrepresentable in code, then it is not only unnecessary to write a unit tests for this case, it is impossible.

Pete
  • 8,916
  • 3
  • 41
  • 53
2

I would argue that you have missed the addition of some of the necessary features because they weren't being highlighted as for unit testing.

For example, unit testing in C# is mainly driven by using attributes. The Custom Attributes feature provides a rich extension mechanism that allows frameworks like NUnit to iterate and compete, with things like Theory-based and Parameterized testing.

This brings me to my second major point - we don't know enough about what makes good testability to bake it into a language. The pace of innovation in testing is much faster than other language constructs so we need to have flexible mechanisms in our languages to leave freedom to innovate.

I'm a user but not fanatic about TDD - it is very helpful in some cases especially for improving your design thinking. It is not necessarily useful for larger legacy systems - higher-level testing with good data and automation will probably yield more benefit along with a strong code inspection culture.

Other relevant reading:

Community
  • 1
Andy Dent
  • 211
  • 1
  • 6