The present article extends work on Ponterotto and Ruckdeschel's Reliability Matrix for estimating the adequacy of internal consistency measures. Specifically, it uses statistical tests to determine whether a calculated coefficient alpha is equal to or greater than the hypothesized population coefficient alpha identified in the Reliability Matrix. The Feldt, Woodruff, and Salih (1987) confidence interval test and Bonett's (2002) approximate z-test and N formula are applied. Select guidelines on reliability assessment and conducting statistical tests to determine adequacy of coefficient alpha are presented.
Get full access to this article
View all access options for this article.
References
1.
American Educational Research Association, American Psychological Association, and the National Council on Measurement in Education. (1999) Standards for educational and psychological testing. Washington, DC: American Educational Research Assoc.
2.
BonettD. G. (2002) Sample size requirements for testing and estimating alpha coefficients. Journal of Educational and Behavioral Statistics, 27, 335–340.
3.
BoyleG. J. (1991) Does item homogeneity indicate internal consistency or item redundancy in psychometric scales?Personality and Individual Differences, 12, 291–294.
4.
CharterR. A. (1997) Confidence interval procedures for retest, alternate form, validity, and alpha coefficients. Perceptual and Motor Skills, 84, 1488–1490.
5.
CharterR. A. (1999) Sample size requirements for precise estimates of reliability, generalizability and validity coefficients. Journal of Clinical and Experimental Neuropsychology, 21, 559–566.
6.
CharterR. A. (2001) “Damn the precision, full speed ahead with the clinical interpretation.”Journal of Clinical and Experimental Neuropsychology, 23, 692–694.
7.
CharterR. A. (2008) Statistical approaches to achieving sufficiently high test score reliabilities for research purposes. The Journal of General Psychology, 135, 241–251.
8.
CharterR. A.FeldtL. S. (2001) Meaning of reliability in terms of correct and incorrect clinical decisions: the art of decision making is still alive. Journal of Clinical and Experimental Neuropsychology, 23, 530–537.
9.
CortinaJ. M. (1993) What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98–104.
10.
FeldtL. S.BrennanR. L. (1989) Reliability. In LinnR. L. (Ed.), Educational measurement. (3rd ed.) Washington, DC: American Council on Education. Pp. 105–146.
11.
FeldtL. S.CharterR. A. (2003a) Estimating the reliability of a test split into two parts of equal or unequal length. Psychological Methods, 8, 102–109.
12.
FeldtL. S.CharterR. A. (2003b) Estimation of internal consistency reliability when test parts vary in effective length. Measurement and Evaluation in Counseling and Development, 36, 23–27.
13.
FeldtL. S.WoodruffD. J.SalihF. A. (1987) Statistical inference for coefficient alpha. Applied Psychological Measurement, 11, 93–103.
14.
GroggelD. J.WackerlyD. D.RaoP. V. (1988) Nonparametric estimation in oneway random effects models. Communications in Statistics–Computation and Simulation, 17, 887–903.
15.
HaynesS. N.RichardD. C. S.KubanyE. S. (1995) Content validity in psychological assessment: a functional approach to concepts and methods. Psychological Assessment, 7, 238–247.
16.
KuderF. (1991) Comments concerning the appropriate use of formulas for estimating the internal consistency reliability of tests. Educational and Psychological Measurement, 51, 873–874.
17.
LudbrookJ.DudleyH. (1998) Why permutation tests are superior to t and F tests in medical research. American Statistician, 52, 127–132.
18.
NunnallyJ. C. (1978) Psychometric theory. (3rd ed.) New York: McGraw-Hill.
19.
PonterottoJ. G.RuckdeschelD. E. (2007) An overview of coefficient alpha and a reliability matrix for estimating adequacy of internal consistency coefficients with psychological research measures. Perceptual and Motor Skills, 105, 997–1014.
20.
ThompsonB.Vacha-HaaseT. (2000) Psychometrics is datametrics. Educational and Psychological Measurement, 60, 174–195.
21.
TraceyT. J. G. (2000) Issues in the analysis and interpretation of quantitative data: deinstitutionalization of the null hypothesis test. In BrownS. D.LentR. W. (Eds.), Handbook of counseling psychology. (3rd ed.) New York: Wiley. Pp 177–198.
22.
WorthingtonR. L.WhittakerT. A. (2006) Scale development research: a content analysis and recommendations for best practices. The Counseling Psychologist, 34, 806–838.
23.
ZimmermanD. W.ZumboB. D.LalondeC. (1993) Coefficient alpha as an estimate of test reliability under violation of two assumptions. Educational and Psychological Measurement, 53, 33–49.
