What do I call "unit tests" that are specifically developed during tinkering?

Question

When developing a new section of code functionality with modules I'm not familiar, I'll often create a set of unit tests that are for "tinkering" and interactively analyzing the results in the debugger.

These tests are not good "unit tests" because they often will not be reproducible without some setup from another human. (i.e. not good for automated testing and CI).

Are these just throw-away code? I see some minimal value in these tests, just above the "throw-away" threshold, but barely.

What would I call these tests, so that another developer could obviously know their purpose when coming into the project for the first time?

I've considered calling them "sandbox" tests?

The main purpose of this question is to gather what the community at large would call these.

Answer 1

In general, if they're useful you should bundle them up into a command line util (or a Lambda sorta thing if you're cloud heavy, or a console hook if you're making a game, etc.). Because if they're useful enough now, they're likely to be useful enough later and you'll appreciate later taking the little bit of time now to polish them up a little bit.

If you can't see them being that useful, it's throw away code. Throw it away. Usually these things only take a few minutes to cobble together - making someone do that again is usually cheaper than the overhead it puts on your codebase in maintenance, compilation, and cognitive load for people to ignore it.

Answer 2

Are these just throw-away code?

If these tests are not specifically designed for describing/asserting behavior of your application, in my opinion they should not be committed to the application repository. The cognitive overhead of developers figuring out what the tests are for and maintaining them would offset the minimal value they add. The test suite in its entirety should be descriptive of the behavior of the application, distinct from any module testing/learning exercises.

However, that does not mean that you necessarily have to throw them away (though I myself wouldn't keep them for long). If you really want to keep the tests, but want to have them uncommitted, you could add a directory to your application repo and add that directory to your VCS exclusion list (e.g. .gitignore). That gets into your naming question; if I went that route, I would likely call this directory ignored, sandbox, test_debug or something like that, perhaps with a README explaining what the directory is for.

Better still, you could create a separate test repository (possibly a fork of your application repo) that makes clear the fact that these tests are just for getting familiar with the modules. That way you can keep the tests in VCS without having them directly associated with your real application and its test suite.