2

I have written a class that implements some BLE scanning functionality. I have also written some test code that helped me to manually verify my scanner works as intended.

Now I want to somehow preserve that test code for future use. But I find myself stumped - how would I do this? I am currently using a unit test method, but that's really just because it was the easiest way to add some test cases that I can run on demand with a single mouse click. The actual test however isn't suitable for being committed as a unit test, because:

  • it depends on external factors to have a result (it needs one BLE device in range to see that the code actually does something)
  • I can't really assert that result (since the "result" depends on the devices in range of the computer that's running the test, it will be always different)
  • the execution time is a scan period of 40 seconds, not milliseconds

All this pretty strongly suggests "not a unit test", since it goes against any reasonable definition of what constitutes a unit test. But what else is there, actually? Apart from maybe creating a brand new C# project with a console application, is there any feature in the MS Test framework that I can use to provide future me and my colleagues with code they can "just run" to check if the scanner works?

(Note: I'd usually ask SO, but I guess this maybe fall under the category of "asking for tools", so I figured I'm better off asking here)

LWChris
  • 129
  • 2
  • These tests make for an excellent QA's test plan. I mean, if you have QA this sort of test (manual) is the kind they do most of the time. They are not looking for bugs. They are validating functionality. These tests will offer a way for them to validate that whatever the scanner is reading, it's handled by the application. What it's needed is good documentation and a "how-to" guide. – Laiv Dec 02 '22 at 13:55
  • 1
    Excellent question. As always with good questions, one of our community members votes for closing it. Please ignore them. – Doc Brown Dec 02 '22 at 14:34

2 Answers2

2

A unit testing framework is not restricted to "unit tests." They are general purpose frameworks allowing you to write test code. More generally what you describe is a good fit for an integration test. Don't throw the code away. It helped you develop this. It can help you maintain it.

The challenge is setting up an environment where you can execute this test repeatedly. Consider adding instructions some place in the codebase telling people how to set up this integration test environment. Perhaps this is something your team does on an as-needed basis.

Alternately, bring this up with your team and management. Compile a list of the hardware and software requirements to run these integration tests. Communicate the benefits. You might be surprised and get support to stand up a dedicated integration test environment.

The last challenge is making an assertion. You mention that the results depend on the devices in range. Consider making tests specific to a certain device, or generalizing the assertion. For example, say you have two different brands of scanners that produce JPEG images. Depending on the specific codec, you could end up with different binary data. Certainly you could not do a pixel-by-pixel comparison, but you could ensure the image size is within some pre-defined range, and that there exists some standardized meta data in the image. Perhaps calculate the resolution and assert it is within some acceptable range.

I'm not sure what kind of "scanner" you are dealing with, but try to think of ways to assert the output that can be generalized to multiple scanners, types of scanners, or different manufacturers.

You have written valuable tests. Don't throw them away. There is nothing wrong with spinning them off into their own project, providing instructions on how to set up the environment, and then having developers run them as-needed. Communicate the benefits and requirements to the team. They might be interested, and you might be a trail blazer moving their process to include more automation. This is a good thing.

Greg Burghardt
  • 34,276
  • 8
  • 63
  • 114
  • I believe that the OP is talking BlueTooth Low Energy (BLE) physical hardware device. Thus they are effectively testing the radio link as a part of their code. Correctly mocking this would require inserting something into the BT stack. – Peter M Dec 02 '22 at 14:13
  • @PeterM: why mock anything in this test? It sounds like the intent is to make sure the radio link actually works. If this was useful during development, why not keep it? – Greg Burghardt Dec 02 '22 at 14:41
  • When I have a little more time, let me adjust my answer to focus on bluetooth devices. Maybe that will help. I used an image scanner as an allegory, because I wasn't sure what the OP was testing. – Greg Burghardt Dec 02 '22 at 14:47
  • It seemed via your suggestion of generalizing to multiple scanners that you were trying to abstract the scanner out of the system. That is probably my mis-reading of the your answer. – Peter M Dec 02 '22 at 14:48
  • @PeterM, no problem. I need to refocus my answer on bluetooth devices anyhow to make it more congruent with the question. – Greg Burghardt Dec 02 '22 at 14:48
  • I am dealing with a BLE scanner ("scanning" is a term that refers to a "look which Bluetooth LE devices there are in the area" kind of operation, like searching for WiFi networks). In my head, someone (maybe me) modifies, say, the internal scanner sensitivity, and now wants to test how that affects devices detection. IMO it should be just one click to get a test running to see what the change did. It will always be manual tests as-needed, but I want the code to be there ready to use. So yes, I want to preserve the code, I'm just wondering how. – LWChris Dec 02 '22 at 14:52
  • "A unit testing framework is not restricted to unit tests." Currently, I have two options - add the Ignore attribute, but then the test gets a yellow warning triangle, presumably because the test runner says "you might be missing something". Or I can comment out the `TestMethod` attribute, but that makes the method disappear from the runner and you have to un-comment it first to get the "Run Now" button. Ideally, I'd add an attribute that tells the runner "Show that it exists, but only execute it when explicitly asked, for "Run all test" runs, skip and show 'skipped, but that's okay' status". – LWChris Dec 02 '22 at 15:36
  • @LWChris: this was addressed in the last paragraph of my answer: "There is nothing wrong with **spinning [the tests] off into their own project**..." – Greg Burghardt Dec 02 '22 at 17:59
  • Once in their own project, you can simply *not run* the tests in that assembly. – Greg Burghardt Dec 02 '22 at 17:59
  • 1
    Ah! That's probably something worth considering in general, right? Create one (more or less) global test project that just holds these code bits that help you with manual testing. So usually you'd not open that test project, but if you need to test something, open the project and run the relevant bit from the test runner. – LWChris Dec 02 '22 at 18:30
0

Standard unit testing frameworks like "MS Test" or "NUnit/JUnit/xUnit" are optimized for running fully automated tests. Sure, they can be used not only for real unit tests, but also for tests which require a longer execution time or some long-lasting I/O. But when a test requires user interaction, they will come to their limits.

Hence, I consider your idea of "creating a brand new C# project with a console application" as perfectly sensible. To make this code "future proof", I would consider to go a step further and think of a simple user interface for that application. The UI should

  • tell you (or the dev who runs it) the manual steps for preparation of the test (for example "switch BLE scanner on and press a key ...")

  • tell how to verify manually whether the test was successful or not (for example "verify the display of the scanner shows 42")

(I don't know if your scanner has something like a display with numbers, but I guess you get the idea.)

Maybe be a console UI is sufficient, or maybe you decide to implement a simple, lightweight GUI (which should not really be a huge effort with C#). Ultimately, this depends on how complex the test and the manual interaction are. But sometimes the most simple solution is to create a missing tool by oneself.

Doc Brown
  • 199,015
  • 33
  • 367
  • 565