4

I have a weird question for community.

One well respected by management developer is strongly against putting data files for JUnit tests in the Maven project directory so that they are committed to the version control system (VCS).

His reasons might be formulated as:

  • Project managers do not have convenient access to VCS
  • 'Common practice' not in open source projects but in your heads only

I find that I have too few reasons to convince others that he is wrong.

My reasons are:

  • Well respected open source projects do it (I doubt that managers will dig into GitHub)
  • A newcomer to the project can by downloading the project expect all tests will be runable without manually feeding data files for tests (another weak point is that the developer's needs are not the concern of corporate management)
  • The project might outlive any manager for lots of reasons, so any new manager should be able to retrieve data files from VCS. How? They should be educated in the use of the VCS.

It is the same story with 'email notification about critical config changes' vs. 'ReadMe.md or ReadMe.txt in project root in VCS'

Do you have references to proven and respected good/bad practice articles which will help me in this debate?

... Or am I really wrong in my belief?

I am aware of disadvantages of using big or complicated data files for unit tests. My case is referring to the Excel spreadsheet processor.

Ola Ström
  • 209
  • 1
  • 2
  • 6
  • 1
    Possible duplicate of [Where should I store test data?](http://programmers.stackexchange.com/questions/264925/where-should-i-store-test-data) – Doc Brown Sep 26 '16 at 07:59
  • 1
    Possible duplicate of [Should test data be checked into version control?](http://programmers.stackexchange.com/questions/257881/should-test-data-be-checked-into-version-control) – Bart van Ingen Schenau Sep 26 '16 at 18:20

1 Answers1

7

The test data belongs to the tests. As such, it should be versioned at the same time. The best reasons I find for this are:

  • The data is kept with the tests for easy access.
  • As you said, tests should work when the initial project is cloned / downloaded.
  • It's simpler to configure to CI / build system to monitor a single repository.
  • Changes to the test data and tests themselves will be grouped together in the same commit / pull request / branch. Reviewing the change history, data changes should be easier to understand because they will be temporally correlated with test changes.
  • Lots of build systems are aware of branches and able to build branches. This is an incredibly powerful / handy feature because it allows you to build the branches before they are merged back into the mainline of the team. If you've split everything needed for your tests across multiple repositories, then your build system will have to be made aware of branches in each. This is a pain.
  • Pull requests / code reviews / merge requests will all need to be coordinated between the various repositories to keep things in sync.

Yeah, but...

I have found one exceptional case when I do split the data out from the main code repository. If the data itself is disproportionately large compared to the repository, then I may consider pushing that into a different repository to make pulling down multiple copies of the source code easier. With git, this is NOT typically a concern. Git's branching is so easy that I rarely keep multiple local copies of git repositories. However, VCS systems like TFS and SVN make branching a real pain and I frequently do keep a separate local copy of the branches that I'm working on. In that case, for a very large dataset, I would consider pushing it out of the repo. But first, I would really consider whether I could just make the test data much smaller and keep it with the code.