This study examined the frequency of use of various types of reliability coefficients for a systematically drawn sample of 696 tests appearing in the APA-published Directory of Unpublished Experimental Mental Measures. Almost all articles included some type of reliability report for at least one test administration. Coefficient alpha was the over-whelming favorite among types of coefficients. Several measures treated almost universally in psychological-testing textbooks were rarely or never used. Problems encountered in the study included ambiguous designations of types of coefficients, reporting reliability based on a study other than the one cited, inadequate information about subscales, and simply incorrect recording of the information given in an original source.
Get full access to this article
View all access options for this article.
References
1.
Aiken, L. R. (1997). Psychological testing and assessment (9th ed.). Boston: Allyn & Bacon.
2.
Anastasi, A. , & Urbina, S. (1997). Psychological testing (7th ed.). Upper Saddle River, NJ: Prentice Hall.
3.
Cohen, R. J. , & Swerdlik, M. E. (1999). Psychological testing and assessment: An introduction to tests and measurement (4th ed.). Mountain View, CA: Mayfield.
4.
Crocker, L. , & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart & Winston.
5.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.
6.
Eason, S. (1991). Why generalizability theory yields better results than classical test theory: A primer with concrete examples. In B. Thompson (Ed.), Advances in educational research: Substantive findings, methodological developments (Vol. 1, pp. 83-98). Greenwich, CT: JAI.
7.
Educational Testing Service . (1995). The ETS test collection catalog: Vol. 2.Vocational tests and measurement devices (2nd ed.). Phoenix, AZ: Oryx.
8.
Fan, X. (1998). Item response theory and classical test theory: An empirical comparison of their item/person statistics. Educational and Psychological Measurement, 58, 357-381.
9.
Feldt, L. S. , & Brennan, R. L. (1989). Reliability. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 105-146). Washington, DC: American Council on Education/Oryx.
10.
Goldman, B. A. , Mitchel, D. F., & Egelson, P. E. (1997). Directory of unpublished experimental mental measures (Vol. 7). Washington, DC: American Psychological Association.
11.
Gregory, R. J. (1996). Psychological testing: History, principles, and applications (2nd ed.). Boston: Allyn & Bacon.
12.
Jaeger, R. (1991). Forward. In R. J. Shavelson & N. M. Webb (Eds.), Generalizability theory: A primer (pp. ix-xi). Newbury Park, CA: Sage.
13.
Joint Committee on Standards for Educational Evaluation . (1994). The program evaluation standards: How to assess evaluations of educational programs (2nd ed.). Thousand Oaks, CA: Sage.
14.
Kaplan, R. M. , & Saccuzzo, D. P. (1997). Psychological testing: Principles, applications, and issues (4th ed.). Pacific Grove, CA: Brooks/Cole.
15.
Maddox, T. (1997). Tests: A comprehensive reference for assessments in psychology, education, and business (4th ed.). Austin, TX: Pro-Ed.
16.
Murphy, K. R. , & Davidshofer, C. O. (1998). Psychological testing: Principles and applications (4th ed.). Upper Saddle River, NJ: Prentice Hall.
17.
Murphy, L. L. , Impara, J. C., & Plake, B. S. (1999). Tests in print V. Lincoln: University of Nebraska Press.
18.
Nunnally, J. C. , & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.
19.
Reinhardt, B. (1996). Factors affecting coefficient alpha: A mini Monte Carlo study. In B. Thompson (Ed.), Advances in social science methodology (Vol. 4, pp. 3-20). Greenwich, CT: JAI.
20.
Thompson, B. (1991). Review of generalizability theory: A primer by R. J. Shavelson & N. W. Webb. Educational and Psychological Measurement, 51, 1069-1075.
21.
Thompson, B. , & Snyder, P. A. (1998). Statistical significance and reliability analyses in recent JCD research articles. Journal of Counseling and Development, 76, 436-441.
22.
Thompson, B. , & Vacha-Haase, T. (2000). Psychometrics is datametrics: The test is not reliable. Educational and Psychological Measurement, 60, 174-195.
23.
Vacha-Haase, T. , Ness, C., Nilsson, J., & Reetz, D. (1999). Practices regarding reporting of reliability coefficients: A review of three journals. Journal of Experimental Education, 67, 335-341.
24.
Wilkinson, L. , & The APA Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604. (Reprint available through the APA home page: http://www.apa.org/journals/amp/amp548594.html)
25.
Willson, V. L. (1980). Research techniques in AERJ articles: 1969 to 1978. Educational Researcher, 9(6), 5-10.
