Nominated Samples from Public Schools and Statistical Bias

Abstract

The magnitude and direction of statistical bias from nonrandom samples of specific types was investigated. Samples from school district populations were generated by a multistage sampling technique, with a modification that substituted for random selection the nomination of sample units at each stage. Within these stages, superintendents were asked to nominate samples of “representative” schools, principals to nominate samples of “representative” classrooms, and teachers to nominate samples of “representative” students. Statistics for the nominated samples were compared to the corresponding theoretical sampling distributions, determined by the appropriate population parameters, under the assumption of random selection. As the sampling proceeded from samples of schools, to samples of classrooms, to samples of students, the respective sample statistics increasingly deviated from expectation implying greater degrees of statistical bias.