Making Usability Data More Usable

Abstract

Usability tests often include recording of users' behaviors as they interact with a computer; unfortunately, these data are often underused for design due to their complexity and the difficulty in interpreting them. In recent years, a variety of methods have been proposed for analyzing user behavioral sequences. In this paper we propose five criteria for evaluating the quality and effectiveness of any such method – (1) standardization and repeatability of the method; (2) simplification of the data; (3) validity; (4) interpretable visual representation of the data that suggests design implications; and (5) output that is amenable to additional analyses. We describe how the PRONET method – an analytical method that uses the Pathfinder algorithm to produce network representations of behavioral sequences – successfully addresses these five criteria. Finally, we argue that test design should be conceived of in a framework of usability engineering, thereby leading to the creation of more usable test methods.