Panel Session: Real-time GOMS: Comparative Modeling of a User-Nintendo Interaction

Abstract

The panelists in this session accepted a challenge. Each agreed to perform a first-pass GOMS/Model Human Processor (MHP) analysis (Card, Moran, & Newell, 1983) based upon a “mystery” videotape that would be sent to them prior to the conference. They also agreed that at the conference they would, in real-time, compare their analyses, discuss why and how each of their analysis differed from the others, and attempt to derive a second-pass, consensual analysis. It was understood that the domain of the videotape would be one to which GOMS/MHP had not yet been applied. (Indeed, they were told that it might be a domain for which many people believed that GOMS/MHP could not be applied.)

The mystery videotape was sent out this summer. As threatened it involved a type of task for which a GOMS analysis had never been done. The videotape showed an expert user interacting with a computer ir a real-time interactive environment, one that involves alternative subgoals, realtime decision making, and real-time recovery from errors. The expert is 9 years old. The task is Super Mario III™.

GOMS Background.

GOMS has come out of the basic research laboratory to become a valuable tool for the practitioner. GOMS analyses of user interactions have been accepted by major corporations as a way to evaluate telephone operator workstations (quantitative prediction), to interpret confusing results from empirical trials (qualitative explanation), and as a means to structure documentation and HELP for major, consumer, software applications (prescriptive guidance).

Despite these successes GOMS (as well as analytic modeling in general) has not had the wide-spread application that it should. There are, at least, two reasons why this is so. First, while GOMS analyses are straight forward (if time-consuming) for those with experience doing them, it is hard for the uninitiated to know where to begin. Most of those who currently use GOMS have acquired their initial skill as an apprentice to a recognized GOMS-master. Second, and related to the first, it seems difficult for the neophyte to imagine applying GOMS to a domain for which a precedent does not exist. Hence we have the has-not-yet-been-done-and-therefore-cannot-be-done phenomena.

For example, at one period everyone knew that GOMS was only good for tasks like text-editing (1983 common knowledge), later on it was spreadsheets (1988 common knowledge), still later help systems and documentation (1989 common knowledge), and now phone company type tasks (1990 common knowledge). But GOMS has not (and therefore cannot?) be used for real-time, interactive environments. Likewise it has not (and cannot?) handle alternative subgoals, realtime decision making, and so on.

Both of these problems were presented as a challenge to the panel members. While the outcome is by no means certain, we challenge the attendees of HFS ′90 to come, watch, and participate.

Panelists Background.

Mike Atwood, the panel chair and discussant, works with the Intelligent Interfaces Group at the NYNEX Science and Technology Center. He is interested in the Cognitive Engineering side of Cognitive Science and sees GOMS and its related developments (such as Cognitive Complexity Theory and Cognitive Walk-Thrus) as the current best hope of the field.

Jay Elkerton is an assistant professor at the University of Michigan's Center for Ergonomics. He is a veteran of many HFS conferences at which he has presented his and his students' work on applying GOMS to the design of on-line computer help and to the design of minimalist documentation. His current, GOMS-related interests include applying GOMS/MHP to explain and design direct-manipulation interfaces.

Wayne Gray is a Member of Technical Staff at the NYNEX Science & Technology Center. At HFS ′89 he presented preliminary data on a project that compared empirical data with GOMS-based predictions. That project is billed as the World's First Real-World validation of GOMS. His current, GOMS-related interests include using GOMS for interface design as both a prescriptive and descriptive tool.

Bonnie John is a Research Scientist of Computer Science and Psychology at Carnegie Mellon University's School of Computer Science. For her dissertation she developed GOMS models of stimulus-response compatibility and expert transcription typing. She showed that quantitative parameters could apply across task domains and that a GOMS model could explain more data and make better predictions than other typing models. This work extended the GOMS keystroke paradigm to include parallel activities and critical path analysis. Her long term goal in this area is to develop GOMS into an off-the-shelve tool that designers and engineers will find easy to use, convenient, and useful.

Judith Olson is a professor in the Computer & Information Systems Department of the University of Michigan. She has applied GOMS to help analyze the differences in cognitive demands made by two different spreadsheet packages and, recently, has written a review of GOMS theory and application which will be appearing in Human-Computer Interaction. Her continuing interest in GOMS is as a vehicle to study basic cognitive processes as well as a framework for HCI applications.