2

Usually, when I write software, I use unit tests for each function to test if it is working without any problems. However, recently I have found myself writing some software that are not really amenable to unit testing. For example, writing code for an operating system, where standard libraries are not available. Another example is writing the constraints of an Integer Linear Programming problem, where no solution can be seen until all the constraints have been added. In these cases, I have struggled to debug my code, since identifying which part of the code is causing problems has been very difficult. Any suggestions how to deal with these cases?

Update: I wanted to write the page-mapping functionality of an operating system. The system used 4-level paging. My code used one function each to go from one level of page table to the next. Although I knew the algorithm was right, the system was not working as expected. However, I was not able to find out which function was creating the problem, since it was impossible to know the exact correct output of each function. Instead, I had to do an endless series of trials and errors (something that took me over a week) to find out the problem.

Thanks.

Arani
  • 131
  • 5
  • 1
    Do you think you could include a detailed example of what kind of function or system you are writing that doesn't seem amenable to unit testing? It's difficult to think about this without an example – David Scholefield Oct 01 '14 at 17:17
  • @DavidScholefield Thanks -- I have added. Please check if it makes more sense now. – Arani Oct 01 '14 at 17:27
  • I'm not seeing how your examples are unit testable. If you need constraints, those are just inputs for the unit test. And you should be fine to test parts of an OS in isolation without deploying it... What am I missing? – Telastyn Oct 01 '14 at 17:29
  • @Telastyn: you surely meant "how your examples are *not* unit testable", I guess? – Doc Brown Oct 01 '14 at 17:36
  • For hardware-dependent tests, look at this posts, not a full duplicate, but maybe helpful: http://programmers.stackexchange.com/questions/66639/how-to-adopt-agile-methodology-for-developing-firmware-embedded-systems-software – Doc Brown Oct 01 '14 at 17:41
  • @docbrown - eh, yes... Sorry about that Arani. – Telastyn Oct 01 '14 at 17:52
  • @Telastyn For constraints, I am able to give the inputs -- but I don't know the output until all the constraints have been added (since solving ILP without all the constraints is useless). So there is no way to understand if the constraint acts as expected. For OS, it is possible to deploy parts in isolation, but not individual functions. A part of an OS is a significant amount of code, and I did not find it easy to debug it. – Arani Oct 01 '14 at 18:14

1 Answers1

4

Hardware-dependend test can be hard to unit-test, indeed, until you have a reliable simulator available. OS / kernel dependent code can be unit tested by providing an abstract interface to the OS or kernel functions which allows to mock these functions out Of course, when your OS functions do not behave the way you expect them to behave, you might provide unit tests for these functions, one after another, to find out how they really work.

But the code for solving an "Integer Linear Programming" problem is typically well testable:

  • the ILP solver can be unit-tested by using lots of small problems with only few constraints
  • each of the constraints is typically a small, standalone boolean function, which is perfect for unit testing

In fact, if you want to test if you implemented the right constraints, and the only way to check this is to run your black-box ILP solver with a constraint vector of size >10000, then you might have a problem with your integration tests, not with your unit tests. But for such big problem spaces, the constraints are typically parametric, and the number of constraints is typically a parameter as well, which offers you the chance to test it with "n constraints" first where n is choosen a very small number.

Doc Brown
  • 199,015
  • 33
  • 367
  • 565