The effect of guessing on the point estimate of coefficient alpha has been studied in the literature, but the impact of guessing and its interactions with other test characteristics on the interval estimators for coefficient alpha has not been fully investigated. This study examined the impact of guessing and its interactions with other test characteristics on four confidence interval (CI) procedures for coefficient alpha in terms of coverage rate (CR), length, and the degree of asymmetry of CI estimates. In addition, interval estimates of coefficient alpha when data follow the essentially tau-equivalent condition were investigated as a supplement to the case of dichotomous data with examinee guessing. For dichotomous data with guessing, the results did not reveal salient negative effects of guessing and its interactions with other test characteristics (sample size, test length, coefficient alpha levels) on CR and the degree of asymmetry, but the effect of guessing was salient as a main effect and an interaction effect with sample size on the length of the CI estimates, making longer CI estimates as guessing increases, especially when combined with a small sample size. Other important effects (e.g., CI procedures on CR) are also discussed.
AndrewsD. W. K. (2000). Inconsistency of the bootstrap when a parameter is on the boundary of the parameter space. Econometrica, 68, 399-405.
2.
BarchardK. A.HakstianA. R. (1997). The effects of sampling model on inference with coefficient alpha. Educational and Psychological Measurement, 57, 893-905.
3.
BayK. S. (1973). The effect of non-normality on the sampling distribution and standard error of reliability coefficient estimates under an analysis of variance model. British Journal of Mathematical and Statistical Psychology, 26, 45-57.
4.
BirnbaumA. (1968). Some latent trait models and their use in inferring an examinee’s ability. In LordF. M.NovickM. R. (Eds.), Statistical theories of mental test scores (pp. 397-479). Reading, MA: Addison-Wesley.
CohenJ. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
7.
CronbachL. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297-334.
8.
DavisonA. C.HinkleyD. V. (1997). Bootstrap methods and their applications. Cambridge, England: Cambridge University Press.
9.
DuhachekA.CoughlanA. T.IacobucciD. (2005). Results on the standard error of the coefficient alpha index of reliability. Marketing Science, 24, 294-301.
10.
EfronB. (1987). Better bootstrap confidence intervals. Journal of the American Statistical Association, 82, 171-185.
11.
EfronB.TibshiraniR. J. (1994). An introduction to the Bootstrap. Boca Raton, FL: Chapman & Hall.
12.
FeldtL. S. (1965). The approximate sampling distribution of Kuder-Richardson reliability coefficient twenty. Psychometrika, 30, 357-370.
13.
HakstianA. R.WhalenT. E. (1976). A k-sample significance test for independent alpha coefficients. Psychometrika, 41, 219-231.
KanoY.AzumaY. (2003). Use of SEM programs to precisely measure scale reliability. In YanaiH.OkadaA.ShigemasuK.KanoY.MeulmanJ. J. (Eds.), New developments in psychometrics (pp. 141-148). Tokyo, Japan: Springer-Verlag.
16.
KristofW. (1963). The statistical theory of stepped-up reliability when a test has been divided into several equivalent parts. Psychometrika, 28, 221-238.
17.
LordF. M. (1955). Sampling fluctuations resulting from the sampling of test items. Psychometrika, 20, 1-22.
18.
MacCannR. G. (2004). Reliability as a function of the number of item options derived from the “knowledge or random guessing” model. Psychometrika, 69, 147-157.
19.
Maydeu-OlivaresA.CoffmanD. L.HartmannW. M. (2007). Asymptotically distribution-free (ADF) interval estimation of coefficient alpha. Psychological Methods, 12, 157-176.
20.
McCullaghP.NelderJ. A. (1989). Generalized linear models (2nd ed.). London, England: Chapman & Hall.
21.
McDonaldR. P. (1999). Test theory—A unified treatment. Mahwah, NJ: Lawrence Erlbaum.
PaekI. (2015). An investigation of the impact of guessing on coefficient α and reliability. Applied Psychological Measurement, 39, 264–277.
30.
PandeyT. N.HubertL. (1975). An empirical comparison of several interval estimation procedures for coefficient alpha. Psychometrika, 40, 169-181.
31.
R Development Core Team. (2014). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org/
32.
RaykovT. (1998). A method of obtaining standard errors and confidence intervals of composite reliability for congeneric items. Applied Psychological Measurement, 22, 369-374.
33.
RaykovT. (2002). Analytic estimation of standard error and confidence interval for scale reliability. Multivariate Behavioral Research, 37, 89-103.
34.
RaykovT. (2012). Scale construction and development using structural equation modeling. In HolyleR. H. (Ed.), Handbook of structural equation modeling (pp. 472-492). New York, NY: Guilford Press.
35.
RaykovT.MarcoulidesG. A. (2011). Introduction to psychometric theory. New York, NY: Routledge.
36.
RomanoJ. L.KromreyJ. D.HibbardS. T. (2010). A Monte Carlo study of eight confidence interval methods for coefficient alpha. Educational and Psychological Measurement, 70, 376-393.
37.
van ZylJ. M.NeudeckerH.NelD. G. (2000). On the distribution of the maximum likelihood estimator of Cronbach’s alpha. Psychometrika, 65, 271-280.
38.
WainerH.BradlowE. T.WangX. (2007). Testlet response theory and its applications. New York, NY: Cambridge University Press.
39.
WallerM. I. (1989). Modeling guessing behavior: A comparison of two IRT models. Applied Psychological Measurement, 13, 233-243.
40.
WoodruffD. J.FeldtL. S. (1986). Tests for equality of several alpha coefficients when their sample estimates are dependent. Psychometrika, 51, 393-413.
41.
YuanK.-H.BentlerP. M. (2002). On robustness of the normal-theory based asymptotic distributions of three reliability coefficient estimates. Psychometrika, 67, 251-259.
42.
YuanK.-H.GuarnacciaC. A.HayslipB. (2003). A study of the distribution of sample coefficient alpha with the Hopkins symptom checklist: Bootstrap versus asymptotics. Educational and Psychological Measurement, 63, 5-23.
43.
ZimmermanD. W. (1975). Proability spaces, Hilbert spaces, and the axioms of test theory. Psychometrika, 40, 395-412.
44.
ZimmermanD. W.WilliamsR. H. (2003). A new look at the influence of guessing on the reliability of multiple-choice tests. Applied Psychological Measurement, 27, 357-371.
