Systematic Errors in Approximations to the Standard Error of Measurement and Reliability

Abstract

Lord's approximation to the standard error of measurement of a test uses only n, the number of items. Millman's is based on n and p, the mean difficulty. Saupe has used Lord's approximation to derive an approximation to the reliability. Through an empirical demonstration involving 200 classroom tests, all three approximations are shown to be biased. The Lord and Millman approximations overestimate s_X √1-KR20, and thus Saupe's under estimates r_XX' for these tests. The unweighted mean of the tests' mean item difficulties was .68, sup porting Lord's original warning that his approxima tion be used cautiously with tests that are either very difficult or very easy. Still, the approximations did correlate very highly with their criteria, sup porting their continued limited use.

Get full access to this article

View all access options for this article.

References

Kuder, G.F. , & Richardson, N.W. The theory of estimation of test reliability . Psychometrika, 1937, 2, 151-160.

Lord, F.M. Do tests of the same length have the same standard errors of measurement? Educational and Psychological Measurement, 1957, 17, 510-521.

Lord, F.M. Tests of the same length do have the same standard error of measurement. Educational and Psychological Measurement, 1959, 19, 233-239.

McMorris, R.F. Evidence on the quality of several approximations for commonly used measurement statistics. Journal of Educational Measurement, 1972, 9, 113-122.

Millman, J. Passing scores and test lengths for domain-referenced measures. Review of Educational Research, 1973, 43, 205-216.

Nie, N.H. , Hull, C.H. , Jenkins, J.G. , Steinbrenner, K. , & Bent, D.H. Statistical package for the social sciences (2nd ed.). New York: McGraw-Hill, 1975.

Saupe, J.L. Some useful estimates of the Kuder-Richardson Formula Number 20 reliability coefficient. Educational and Psychological Measurement, 1961, 21, 63-71.

Swineford, F. Note on "Tests of the same length do have the same standard error of measurement ." Educational and Psychological Measurement, 1959, 19, 241-242.