Everything has an interface. When I put my testing hat on, I use a specific world-view to write a test:
- If something exists, it can be measured.
- If it can't be measured, it doesn't matter. If it does matter, I just haven't found a way to measure it yet.
- Requirements prescribe measurable properties, or they are useless.
- A system fulfils a requirement when it transitions from a not-expected state to the expected state prescribed by the requirement.
- A system consists of interacting components, which may be subsystems. A system is correct when all components are correct and the interaction between components is correct.
In your case, your system has three main parts:
- some kind of data or images, which can be initialized from files
- a mechanism to display the data
- a mechanism to modify the data
Incidentally, that sounds very much like the original Model-View-Controller architecture to me. Ideally, these three elements exhibit loose coupling – that is, you define clear boundaries between them with well-defined (and thus well-testable) interfaces.
A complex interaction with the software can be translated into small steps that can be phrased in terms of the elements of the system we are testing. For example:
I load a file with some data. It displays a graph. When I drag a slider in the UI, the graph becomes all wobbly.
This seems to be easy to test manually and difficult to test automated. But let's translate that story into our system:
- The UI provides a mechanism to open a file: the Controller is correct.
- When I open a file, the Controller issues an appropriate command to the Model: the Controller–Model interaction is correct.
- Given a test file, the model parses this into the expected data structure: the Model is correct.
- Given a test data structure, the View renders the expected output: the View is correct. Some test data structures will be normal graphs, others will be wobbly graphs.
- The interaction View–Model is correct
- The UI provides a slider to make the graph wobbly: the Controller is correct.
- When the slider is set to a specific value, the Controller issues the expected command to the Model: the Controller–Model interaction is correct.
- When receiving a test command regarding wobbliness, the Model transforms a test data structure to the expected result data structure.
Grouped by component, we end up with the following properties to test:
- Model:
- parses files
- responds to file-open command
- provides access to data
- responds to make-wobbly command
- View:
- Controller:
- provides file-open workflow
- issues file-open command
- provides make-wobbly workflow
- issues make-wobbly command
- whole system:
- the connection between the components is correct.
If we do not decompose the problem of testing into smaller subtests, testing becomes really difficult, and really fragile. The above story could also be implemented as “when I load a specific file and set the slider to a specific value, a specific image is rendered”. This is fragile since it breaks when any element in the system changes.
- It breaks when I change the controls for wobbliness (e.g. handles on the graph instead of a slider in a control panel).
- It breaks when I change the output format (e.g. the rendered bitmap is different because I changed the default colour of the graph, or because I added anti-aliasing to make the graph look smoother. Note that in both of these cases).
Granular tests also have the really big advantage that they allow me to evolve the system without fear of breaking any feature. Since all required behaviour is measured by a complete test suite, the tests will notify me should anything break. Since they are granular, they will point me to the problem area. E.g. if I accidentally change the interface of any component, only the tests of that interface will fail and not any other test that happens to indirectly use that interface.
If testing is supposed to be easy, this requires a suitable design. For example, it is problematic when I hard-wire components in a system: if I want to test the interaction of a component with other components in a system, I need to replace those other components with test stubs that let me log, verify, and choreograph that interaction. In other words, I need some dependency injection mechanism, and static dependencies should be avoided. When testing an UI, it's a great help when this UI is scriptable.
Of course, most of that is just a fantasy of an ideal world where everything is decoupled and easily testable and flying unicorns spread love and peace ;-) While anything is fundamentally testable, it is often prohibitively difficult to do so, and you have better uses of your time. However, systems can be engineered for testability, and typically even testing-agnostic systems feature internal APIs or contracts that can be tested (if not, I bet your architecture is crap and you have written a big ball of mud). In my experience, even small amounts of (automated) testing effect a noticeable increase of quality.
