How you're currently looking at responsiveness goes against the separation of concerns principle. Your question presumes that responsiveness is a single responsibility that only belongs to one component.
I'd suggest that responsiveness is an attribute / responsibility that can be applied to multiple components. For the sake of my example, I'm assuming a generic MVC | MVP | MVVM type pattern.
The View most certainly has a hand in the responsiveness of the application. The UI elements and logic that you use all dictate how the View will perform. So the View is responsible for the responsiveness of the UI elements.
The Controller also has a hand in application responsiveness. The types of data structures and how the business logic is crafted will affect performance. So here, the Controller | Presenter | ViewModel also have responsibility for responsiveness. But this responsibility is over different elements than what the View is responsible for.
Finally, the Model is responsible for the data access / service calls. There are obvious performance considerations in how the data is retrieved and presented to the middle layer. But again, this is a different element that must also be responsive.
Responsiveness as a property is not the single responsibility of any one component. All components must be responsible for their own crafting in order to contribute to the overall application. A great UI and Controller can be rendered useless by seemingly interminable data requests.
As far as testing, using a tiered approach still works in your favor with respect to overall effort and responsiveness. If you have 5 devices and wrote individual layers for each device, you would have 15 components to test. Using the MVC* pattern allows you to remove 8 components from testing since you have a common Model and Controller. If those two layers perform their share of the work in being responsive, then you only have to test them once. You can then focus your remaining efforts on the 5 Views.
