What features should be tested via automated UI testing?

Question

We recently had a consultant tell us that if a feature can only be tested via automated UI tests (e.g. Selenium, Coded UI), then there is an underlying architectural issue. While this statement might be a bit extreme, it is along the same lines of the testing pyramid in that UI tests should make up a small portion of your overall automated test suite.

So, what kinds of features should have automated UI testing? Will a system with a cogent architecture still have features that can only be verified through UI tests, or should these tests just serve as "back-up" for a suite of unit and service tests?

Answer 1

Presumably you have some sort of architecture that connects buttons and other widgets to actions -- click on save and the save function should be called, etc.

Assuming you have good test coverage of the actions themselves via unit or integration tests, the goal of automated UI testing is to insure that the widgets are all making proper calls to the underlying actions, and correctly displaying the results of those actions.

In other words, they validate that the UI properly reflects the state of the model, and is properly hooked up to the controller.

The other component is when some business logic is in the UI code. For example, maybe you require certain characters in a password field. You need to test those features somehow, since it may be difficult or impossible to do that via unit tests.

Answer 2

What features should be tested via automated UI testing?

All ui features.

You should test:

• All functions that users can do.
• Functions that users do in all the browsers (and versions of) that they do them in.
• Both happy (doing the right thing at every step) vs sad (user makes errors) paths.
• Valid and invalid Data.
• Large and Small numbers.
• All roles and functions roles specific to role.
• User and Admin interfaces.
• Attempts to break-in.
• Creating vs. Updating data works as intended.
• Delete's work and delete associated database data in other tables.
• Keyboard vs. Mouse style users.
• Different devices or simulators of - pc's, phones, ipads, etc.

Answer 3

There is no such thing as a feature that can only be tested via automated UI tests. Tools like Selenium just mimic the behavior of a human, without boring the human to the point of suicide. Ask yourself (or your consultant) a question: "How would a program test UI aspects that a person cannot?" The next question is: "If so, why would you care what they do?"

On the other hand if, you have such a feature, well, yeah, I'd say you have an underlying architectural issue. :-)