6

Is their only similarity the fact that they are not xUnit (or more precisely, not based on enumerating specific test cases), or is it deeper than that?

Property-based testing (using QuickCheck, ScalaCheck, etc) seem well-suited to a functional programming style where side-effects are avoided. On the other hand, Design by Contract (as implemented in Eiffel) is more suited to OOP languages: you can express post-conditions about the effects of methods, not just their return values.

But both of them involve testing assertions that are true in general (rather than assertions that should be true for a specific test case). And both can be tested using randomly generated inputs (with QuickCheck this is the only way, whereas with Eiffel I believe it is an optional feature of the AutoTest tool).

Is there an umbrella term to encompass both approaches? Or am I imagining a relationship that doesn't really exist.

Todd Owen
  • 173
  • 5

2 Answers2

3

The relationship is that the combination of design by contract and the testing methods are attempting to substitute for a correctness proof, which would be the ultimate goal, if it were feasible. Pre-conditions and post-conditions (for return values) are useful even in methods with no side effects, as they limit input domains and output ranges.

Frank Hileman
  • 3,922
  • 16
  • 18
2

Maybe one relationship lies in the concept of Oracle described by Bertrand Meyer, author of Eiffel, in "Seven Principles of Software Testing".

A test run is only useful if you can unambiguously determine whether it passed. The criterion is called a test oracle. If you have a few dozen or perhaps a few hundred tests, you might afford to examine the results individually, but this does not scale up. The task cries for automation.

Principle 4: Applying oracle Determining success or failure of tests must be an automatic process.

This statement of the principle leaves open the form of oracles. Often, oracles are specified separately. In research such as ours, they are built in, as the target software already includes contracts that the tests use as oracles.

Principle 4 (variant): Contracts as oracles Oracles should be part of the program text, as contracts. Determining test success or failure should be an automatic process consisting of monitoring contract satisfaction during execution.

  • Based on this, I guess we could say QuickCheck and Eiffel's Auto Test both use automated test oracles but different oracle forms, the former being external to the production program (properties expressed in separate test suite) and the latter, built into the production program (contracts).

  • From a historical perspective, we can assume that QuickCheck (first released in 1999) and earlier work on property-based testing dating back to the 90's served as inspiration to Eiffel's AutoTest (which appeared in 2008), although I couldn't find any firm evidence of that in Bertrand Meyer's papers or interviews.

guillaume31
  • 8,358
  • 22
  • 33
  • Thanks for drawing this connection. So in a nutshell, design-by-contract and property-based testing both imply an *automated* means of checking of whether a test case meets the spec or not. – Todd Owen Apr 22 '14 at 03:13
  • And even if the inspiration for AutoTest is not known, when the designer of Eiffel begins advocating randomized testing, we might call it a sign of convergence between design-by-contract and property-based testing. – Todd Owen Apr 22 '14 at 03:21