Statistical Considerations for the Number of Participants in Human Factors Scaling Studies

Abstract

Scaling is widely used in human factors and other psychological research as the primary method to assess mental states. While scaling biases and ways to mitigate such biases are well documented, there remains little discussion of the appropriate number of participants for scaling-based research studies. This paper reviews evidence suggesting that for most scaling research, the discoverability of findings asymptotes when using five participants. This paper includes a discussion of the use of descriptive vs. inferential statistics for scaling research, a review of the parallels between scaling and usability research with regard to the number of participants, and a discussion of statistical power for scaling research.