Testing Usability Goals With Small Samples

Abstract

If a usability goal is high, a single failure during summative testing can drive standard approaches to large sample sizes. With a minimum task performance criterion of 95%, a failure by one subject would require testing at least 19 other subjects to show a 95% success rate for the sample. The described small sample methodology for usability testing provides a necessary utility. It provides a method to support a conclusion that a usability performance goal was met, even if a small subset of subjects fail to meet the goal.