An Overview of Coefficient Alpha and a Reliability Matrix for Estimating Adequacy of Internal Consistency Coefficients with Psychological Research Measures

Abstract

The present article addresses issues in reliability assessment that are often neglected in psychological research such as acceptable levels of internal consistency for research purposes, factors affecting the magnitude of coefficient alpha (α), and considerations for interpreting α within the research context. A new reliability matrix anchored in classical test theory is introduced to help researchers judge adequacy of internal consistency coefficients with research measures. Guidelines and cautions in applying the matrix are provided.

Get full access to this article

View all access options for this article.

References

Aiken

L. R.

(2000) Psychological testing and assessment. (10th ed.) Boston, MA: Allyn & Bacon.

American Educational Research Association, American Psychological Association, and the National Council on Measurement in Education. (1999) Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Anastasi

(1988) Psychological testing. (6th ed.) New York: Macmillan.

Baugh

(2002) Correcting effect sizes for score reliability: A reminder that measurement and substantive issues are linked inextricably. Educational and Psychological Measurement, 62, 254–263.

Bolt

D. M.

Rounds

(2000) Advances in psychometric theory and methods. In Brown

S. D.

Lent

R. W.

(Eds.), Handbook of counseling psychology. (3rd ed.) New York: Wiley. Pp. 140–176.

Boyle

G. J.

(1991) Does item homogeneity indicate internal consistency or item redundancy in psychometric scales? Personality and Individual Differences, 12, 291–294.

Boyle

G. J.

Stankov

Cattell

R. B.

(1995) Measurement and statistical models in the study of personality and intelligence. In Saklofske

D. H.

Zeidnre

(Eds.), International handbook of personality and intelligence. New York: Plenum. Pp. 417–446.

Carmines

E. G.

Zeller

R. A.

(1979) Reliability and validity assessment. Thousand Oaks, CA: Sage.

Cattell

R. B.

(1978) The scientific use of factor analysis in behavioral and life sciences. New York: Plenum.

10.

Cattell

R. B.

(1992) Human motivation objectively, experimentally analysed. British Journal of Medical Psychology, 65, 237–243.

11.

Charter

R. A.

(1997) Confidence interval procedures for retest, alternate-form, validity, and alpha coefficients. Perceptual and Motor Skills, 84, 1488–1490.

12.

Charter

R. A.

(1999) Sample size requirements for precise estimates of reliability, generalizability, and validity coefficients. Journal of Clinical and Experimental Neuropsychology, 21, 559–566.

13.

Charter

R. A.

(2001a) Damn the precision, full speed ahead with the clinical interpretation. Journal of Clinical and Experimental Neuropsychology, 23, 692–694.

14.

Charter

R. A.

(2001b) It is time to bury the Spearman-Brown “Prophecy” formula for some common applications. Educational and Psychological Measurement, 61, 690–696.

15.

Charter

R. A.

(2003a) A breakdown of reliability coefficients by test type and reliability method, and the clinical implications of low reliability. The Journal of General Psychology, 130, 290–304.

16.

Charter

R. A.

(2003b) Study samples are too small to produce sufficiently precise reliability coefficients. The Journal of General Psychology, 130, 117–129.

17.

Charter

R. A.

(in press) Statistical approaches to achieving sufficiently high test score reliabilities for research purposes. The Journal of General Psychology.

18.

Charter

R. A.

Feldt

L. S.

(2000) The relationship between two methods of evaluating an examinee's difference scores. Journal of Psychoeducational Assessment, 18, 125–142.

19.

Charter

R. A.

Feldt

L. S.

(2001) Meaning of reliability in terms of correct and incorrect clinical decisions: The art of decision making is still alive. Journal of Clinical and Experimental Neuropsychology, 23, 530–537.

20.

Charter

R. A.

Feldt

L. S.

(2002) The importance of reliability as it relates to true score confidence intervals. Measurement and Evaluation in Counseling and Development, 35, 104–112.

21.

Churchill

G. A.

Jr. Peter

J. P.

(1984) Research design effects on the reliability of rating scales: A meta-analysis. Journal of Marketing Research, 21, 360–375.

22.

Cicchetti

D. V.

(1994) Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290.

23.

Clark

L. A.

Watson

(1995) Constructing validity: Basic issues in objective scale development. Psychological Assessment, 7, 309–319.

24.

Constantine

M. G.

Ponterotto

J. G.

(2006) Evaluating and selecting psychological measures for research purposes. In Leong

F. T. L.

Austin

J. T.

(Eds.), The psychology research handbook: A guide for graduate students and research assistants. (2nd ed.) Thousand Oaks, CA: Sage. Pp. 104–113.

25.

Cortina

J. M.

(1993) What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98–104.

26.

Cronbach

L. J.

(1951) Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334.

27.

Cronbach

L. J.

Shavelson

R. J.

(2004) My current thoughts on coefficient alpha and successor procedures. Educational and Psychological Measurement, 64, 391–418.

28.

Cunningham

G. K.

(1986) Educational and psychological measurement. New York: Macmillan.

29.

Fan

Yin

(2003) Examinee characteristics and score reliability: An empirical investigation. Educational and Psychological Measurement, 63, 357–368.

30.

Feldt

L. S.

(1965) The approximate sampling distribution of Kuder-Richardson reliability coefficient twenty. Psychometrika, 30, 357–370.

31.

Feldt

L. S.

Ankenmann

R. D.

(1999) Determining sample size for a test of the equality of alpha coefficients when the number of part-tests is small. Psychological Methods, 4, 366–377.

32.

Feldt

L. S.

Charter

R. A.

(2003a) Estimating the reliability of a test split into two parts of equal or unequal length. Psychological Methods, 8, 102–109.

33.

Feldt

L. S.

Charter

R. A.

(2003b) Estimation of internal consistency reliability when test parts vary in effective length. Measurement and Evaluation in Counseling and Development, 36, 23–27.

34.

Feldt

L. S.

Woodruff

D. J.

Salih

F. A.

(1987) Statistical inference for coefficient alpha. Applied Psychological Measurement, 11, 93–103.

35.

Gregory

R. J.

(1999) Foundations of intellectual assessment: The WAIS–III and other tests in clinical practice. Boston, MA: Allyn & Bacon.

36.

Gregory

R. J.

(2000) Psychological testing: History, principles, and applications. (3rd ed.) Boston, MA: Allyn & Bacon.

37.

Guilford

J. P.

Fruchter

(1978) Fundamental statistics in psychology in education. (6th ed.) New York: McGraw-Hill.

38.

Haynes

S. N.

Richard

D. C. S.

Kubany

E. S.

(1995) Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7, 238–247.

39.

Helms

J. E.

Henze

K. T.

Sass

T. L.

Mifsud

V. A.

(2006) Treating Cronbach's alpha reliability coefficients as data in counseling research. The Counseling Psychologist, 34, 630–660.

40.

Henson

R. K.

(2001) Understanding internal consistency reliability estimates: A conceptual primer on coefficient alpha. Measurement and Evaluation in Counseling and Development, 34, 177–189.

41.

Henson

R. K.

(2005) Sample heterogeneity and coefficient alpha: Comment on Mji and Alkhateeb's reliability generalization study and a theoretical clarification. Psychological Reports, 97, 275–276.

42.

Heppner

P. P.

Wampold

B. E.

Kivlighan

D. M.

(2008) Research design in counseling. (3rd ed.) Pacific Grove, CA: Brooks/Cole.

43.

Hogan

T. P.

Benjamin

Brezinski

K. L.

(2000) Reliability methods: A note on the frequency of use of various types. Educational and Psychological Measurement, 60, 523–531.

44.

Hopkins

K. D.

Stanley

J. C.

Hopkins

B. R.

(1990) Educational and psychological measurement and evaluation. (7th ed.) Englewood Cliffs, NJ: Prentice Hall.

45.

Kaplan

R. M.

Saccuzzo

D. P.

(1997) Psychological testing: Principles, applications, and issues. (4th ed.) Pacific Grove, CA: Brooks/Cole.

46.

Kerlinger

F. N.

(1973) Foundations of behavioral research. (2nd ed.) New York: Holt, Rinehart, & Winston.

47.

Kline

(1986) A handbook of test construction: Introduction to psychometric design. London: Methuen.

48.

Kline

(2000) Handbook of psychological testing. New York: Routledge.

49.

Kuder

(1991) Comments concerning the appropriate use of formulas for estimating the internal consistency reliability of tests. Educational and Psychological Measurement, 51, 873–874.

50.

Lord

F. M.

Novick

M. R.

(1968) Statistical theories of mental test scores. Reading, MA: Addison-Wesley.

51.

Meyer

G. J.

Finn

S. E.

Eyde

L. D.

Kay

G. G.

Moreland

K. L.

Dies

R. R.

Eisman

E. J.

Kubiszyn

T. W.

Reed

G. M.

(2001) Psychological testing and psychological assessment: A review of evidence and issues. American Psychologist, 56, 128–165.

52.

Nunnally

J. C.

(1978) Psychometric theory. (2nd ed.) New York: McGraw-Hill.

53.

Nunnally

J. C.

Bernstein

I. H.

(1994) Psychometric theory. (3rd ed.) New York: McGraw-Hill.

54.

Onwuegbuzie

A. J.

Daniel

L. G.

(2002) A framework for reporting and interpreting internal consistency reliability estimates. Measurement and Evaluation in Counseling and Development, 35, 89–103.

55.

Pedhazur

E. J.

Schmelkin

L. P.

(1991) Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Erlbaum.

56.

Peter

J. P.

Churchill

G. A.

Jr. (1986) Relationships among research design choices and psychometric properties of rating scales: A meta-analysis. Journal of Marketing Research, 23, 1–10.

57.

Peterson

R. A.

(1994) A meta-analysis of Cronbach's coefficient alpha. Journal of Consumer Research, 21, 381–391.

58.

Ponterotto

J. G.

(1996) Evaluating and selecting research instruments. In Leong

F. T. L.

Austin

J. T.

(Eds.), The psychology research handbook: A guide for graduate students and research assistants. Thousand Oaks, CA: Sage. Pp. 73–84.

59.

Ponterotto

J. G.

Park-Taylor

(2007) Racial and ethnic identity theory, measurement, and research in counseling psychology: Present status and future directions. Journal of Counseling Psychology, 54, 282–294.

60.

Raykov

(1997a) Estimation of composite reliability for congeneric measures. Applied Psychological Measurement, 21, 173–184.

61.

Raykov

(1997b) Scale reliability, Cronbach's coefficient alpha, and violations of essential tau-equivalence with fixed congeneric components. Multivariate Behavioral Research, 32, 329–353.

62.

Raykov

(1998) Coefficient alpha and composite reliability with interrelated nonhomogenous items. Applied Psychological Measurement, 22, 375–385.

63.

Robinson

J. P.

Shaver

P. R.

Wrightsman

L. S.

(Eds.) (1999a) Measures of political attitudes. San Diego, CA: Academic Press.

64.

Robinson

J. P.

Shaver

P. R.

Wrightsman

L. S.

(1999b) Scale selection and evaluation. In Robinson

J. P.

Shaver

P. R.

Wrightsman

L. S.

(Eds.), Measures of political attitudes. San Diego, CA: Academic Press. Pp. 1–36.

65.

Rodriguez

M. C.

Maeda

(2006) Meta-analysis of coefficient alpha. Psychological Methods, 11, 306–322.

66.

Rosenthal

Rosnow

R. L.

(1991) Essentials of behavioral research: Methods and data analysis. (2nd ed.) New York: McGraw-Hill.

67.

Salvia

Ysseldyke

(1988) Assessment in special and remedial education. (4th ed.) Boston, MA: Houghton-Mifflin.

68.

Sawilowsky

S. S.

(2000) Reliability: Rejoinder to Thompson and Vacha-Haase. Educational and Psychological Measurement, 60, 196–200.

69.

Schmidt

F. L.

Hunter

J. E.

(1996) Measurement error in psychological research: Lessons from 26 research scenarios. Psychological Methods, 1, 199–223.

70.

Schmitt

(1996) Uses and abuses of coefficient alpha. Psychological Assessment, 8, 350–353.

71.

Sternberg

R. J.

(Ed.) (1994) Encyclopedia of human intelligence. Vol. 2. New York: Macmillan.

72.

Streiner

D. L.

(2003) Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80, 99–103.

73.

Thompson

(1994) Guidelines for authors. Educational and Psychological Measurement, 54, 837–847. [Editorial].

74.

Thompson

Vacha-Haase

(2000) Psychometrics is not datametrics. Educational and Psychological Measurements, 60, 174–195.

75.

Tracey

T. J. G.

(2000) Issues in the analysis and interpretation of quantitative data: Deinstitutionalization of the null hypothesis test. In Brown

S. D.

Lent

R. W.

(Eds.), Handbook of counseling psychology. (3rd ed.) New York: Wiley. Pp. 177–198.

76.

Walsh

W. B.

Betz

N. E.

(1990) Tests and assessment. (2nd ed.) Englewood Cliffs, NJ: Prentice-Hall.

77.

Weiner

E. A.

Stewart

B. J.

(1984) Assessing individuals: Psychological and educational tests and measurement. Boston, MA: Little, Brown.

78.

Wilkinson

, & American Psychological Association Task Force on Statistical Inference. (1999) Statistical methods in psychology journals: Guidelines and applications. American Psychologist, 54, 594–604.