Can Percentiles Replace Raw Scores in the Statistical Analysis of Test Data?

Abstract

Educational and psychological testing textbooks typically warn of the inappropriateness of performing arithmetic operations and statistical analysis on percentiles instead of raw scores. This seems inconsistent with the well-established finding that transforming scores to ranks and using nonparametric methods often improves the validity and power of significance tests for nonnormal distributions. This study compared Student’s t test performed on raw scores, on the ranks of scores, and on percentiles of these scores obtained from larger populations for normal and various skewed and symmetric nonnormal distributions. Using percentiles instead of raw scores protected the Type I error rate of t tests, like using ranks instead of raw scores, for all distributions studied. Using percentiles markedly increased the power of t tests for skewed distributions, more so than using ranks, and percentiles were nearly as effective as ranks for symmetric distributions. These findings are relevant to experimental designs involving test scores and other measures when both raw scores and percentiles are available.