Usability Testing: Where are We and Where are We Going?

Abstract

Four years ago a group of usability test specialists held a panel discussion about this relatively new technique (“Usability testing in the real world,” Mills, 1987, SIGCHI Bulletin, 43–46). One of the objectives of that panel was to expose professionals in the usability community to testing methods. That panel assumed that the topic of testing would be new to its audience. Since that time, the number of laboratories that conduct tests has exploded. While no one seems to know for sure, there appear to be in excess of 100 labs in the U.S. alone. Yet, there is no formal organization for people who do testing. This panel session focuses on the current status of testing and its future. The panel will assume that the audience is at least somewhat familiar with the principles of testing.

The primary objective of the panel is to make the case that usability testing is a method that is evolving and changing. It is changing because product development processes are changing and because its strengths and weaknesses are more obvious now. The panel's second objective is to provoke a discussion about usability testing and testing methods. The panel will focus on several themes:

• •

Usability tests are becoming less formal. The typical evolution of a human factors method is that it becomes more structured and formal over time. Usability testers, however, have had to adapt to changes in the product development process. More functions than ever are being implemented in software. More flexible software tools have made it possible to withhold freezing the components of user interface until very late in the design process. Consequently, conducting quick, informal tests is becoming more typical because the formal test to verify usability often comes too late to influence the product design.

•

Usability testing is moving beyond testing the user interface itself. With the introduction of object-oriented programming and contextual design there is more interest in understanding users' cognitive processes. There is also renewed interest in understanding how testing fits with other methods for improving usability.

•

Usability testing can improve managerial and organizational process. Usability tests reveal more than just flaws in product designs. They also can identify problems in the design process. When test results point to poor management practices and groups that are not cooperating, they can lead to changes in the way organizations develop products. Viewed from this perspective, a usability test has benefits beyond the improvements to the product being tested. It can be used to evaluate management practices in addition to competence in applying effective technical practices.