57

I'm a growing programmer who's finally putting unit testing into practice for a library that I'm storing on GitHub.

It occurred to me that I might include the test suites in the repo, but as I look around at other projects, the inclusion of tests seems hit-or-miss.

Is this considered bad form? Is the idea that users are only interested in the working code and that they'll test in their own framework anyway?

Kyralessa
  • 3,703
  • 2
  • 19
  • 20
parisminton
  • 673
  • 1
  • 5
  • 6
  • 13
    Why not? Tests should come along with the project or they are barely useless. –  Jan 05 '12 at 09:43
  • 64
    The fact that some project don't include unit tests in their repository is more likely due to non-existence of these tests in the first place. – brabec Jan 05 '12 at 10:39
  • 4
    They are built separately, but otherwise u-tests are tightly coupled with code. To ensure consistency there must some kind of dependency management. Be it putting them into same repository, subrepository or maintaing version controlled "link" to test repository, etc. – MaR Jan 05 '12 at 11:03
  • I came, I saw, I said "Duh!" – Mahmoud Hossam Jan 05 '12 at 11:29
  • 2
    @stoupa is pretty definitely right: it'd be lovely to assume the best, that there's a wonderful cache of tests hidden away somewhere, but in the real world, programmers are lazy. – Cascabel Jan 05 '12 at 14:06
  • 2
    Note, that I would consider it a good idea to disable test class compilation in your build script. – user606723 Jan 05 '12 at 15:31
  • @stoupa, one of the servers in my company has the same name as you. Do you like Gummi Bears? – svick Jan 05 '12 at 18:35
  • 1
    Once/If you start using CI, where is the CI server going to get the tests from? – Steven Evers Jan 06 '12 at 18:23
  • @user606723 That just means the tests are likely to get out of date and become useless. Better to compile them but not into the build artifact. – Ricky Clarkson May 09 '12 at 02:39

6 Answers6

121

You definitely should put your tests into the repository. Tests are in my opinion part of the code and can help others immensely to understand it (if well written). Besides, they can help others when changing or contributing to your codebase. Good tests can give you the confidence that your changes do not inadvertently break anything.

The test code should be well separated from production code, though. Maven for example achieves this by putting production and test code into different folders. The question "is this file part of production or of the test code" should never arise.

I personally do not write unit tests for used libraries in my own code. I expect them to be working (at least when I use a release version, though bugs obviously can appear). It gets some test coverage in integration tests, but that's not enough.

ftr
  • 1,601
  • 1
  • 14
  • 19
  • 1
    As another example, take a look at [ipython's testing folder](https://github.com/ipython/ipython/tree/master/IPython/testing). If someone checks it out, no matter where they put it, the relative links for the tests are still true. It makes it trivial to test it, which is important for determining if your new dev machine is configured properly. – Spencer Rathbun Jan 05 '12 at 20:27
  • Tests are not only part of the code, but also part of the documentation, and often part of the spec. Tests (which run and pass) are "living documentation". – Michael Easter May 03 '14 at 12:41
54

If you don't include the unit tests in the checked-in source code, then:

  • how is someone who downloads and builds their own copy of that code going to verify that it's working the way it was intended? Compiler and library bugs are rare, and data errors (particularly ones that don't render the source code impossible to compile) are even more rare, but they definitely can crop in, particularly when you cannot dictate the build environment to the degree that an employer can dictate which tools are used.
  • how are you going to get the tests back if something happens to your local copy of the source code?
  • how is a new developer going to verify that their changes doesn't break anything in the existing code base?

Bottom line, I would consider not including any unit tests written in the official source code repository a very bad thing.

user
  • 2,703
  • 18
  • 25
8

Of course you should put unit tests into the repository, for several reasons:

  • it is easy to revert to previous version
  • other people working on the project also gain access to unit tests
  • some people consider unit tests as part of the documentation (TDD and BDD)
Radian
  • 797
  • 4
  • 9
BЈовић
  • 13,981
  • 8
  • 61
  • 81
6

If there is any chance to run them on another computer, definitely include them. They should be built separately, so users don't have to, and they may have extra dependencies, but they should be definitely included.

I'd strongly suspect most projects that don't include tests in the repository simply don't have any.

There may be a reason to have the tests as separate module, so you can easily run new tests against older code. It would be useful for API-stable library or black-box tests via command-line; usefulness for compiled languages where the new tests probably won't compile with older code is limited.

Jan Hudec
  • 18,250
  • 1
  • 39
  • 62
5

Absolutely. The extra bonus points here are in the ability to track that version X of a source file goes with version Y of a test. In other words, you need to be able to go back to a previous version and pull the appropriate tests designed for that version (in case of an API change or somesuch).

anon
  • 1,474
  • 8
  • 8
3

I've just finished reading "Brownfield Application Development in .Net", and this provides some excellent advice of the structure of an application, including source control and where/how/why to include unit tests (particularly in the area of Continuous Integration). If you're a .Net developer, I'd recommend it.

seanfitzg
  • 131
  • 2