How Often are Our Statistics Wrong? a Statistics Class Exercise

Abstract

Using this exercise, students recompute the values of statistical tests published in journal articles, and the recomputed values are compared to the published results. My experience with one undergraduate and one graduate class suggests that errors in statistical results published in professional journals may be more common than is generally believed. Recalculations indicated that approximately 13% of the results reported as statistically significant were not. Some possible reasons for the discrepancies between published and recomputed values are discussed, as are students' reactions to the exercise.