Systematic Errors in Approximations to the Standard Error of Measurement and Reliability

Abstract

Lord's approximation to the standard error of measurement of a test uses only n, the number of items. Millman's is based on n and p, the mean difficulty. Saupe has used Lord's approximation to derive an approximation to the reliability. Through an empirical demonstration involving 200 classroom tests, all three approximations are shown to be biased. The Lord and Millman approximations overestimate s_X √1-KR20, and thus Saupe's under estimates r_XX' for these tests. The unweighted mean of the tests' mean item difficulties was .68, sup porting Lord's original warning that his approxima tion be used cautiously with tests that are either very difficult or very easy. Still, the approximations did correlate very highly with their criteria, sup porting their continued limited use.