20+ Applications in 8 Weeks: Lessons from Large-Scale “Checkpoint” Usability Testing

Abstract

In 2005, User Centric was asked by a large technology organization to test over 20 different prototype software applications in face-to-face usability testing during a single 8-week period. We share our experiences and strategies for addressing the logistical challenges so that other usability practitioners can learn about approaches that worked. Challenges included low-incidence recruitment, short window for test preparation, domain-specific applications, moderator fatigue, report creation overlapping with testing, and high-volume production of detailed reports. We provide specific recommendations for staging large-scale checkpoint usability tests and identify which approaches worked especially well in a second round of testing.