In a pioneering research article, Wollack and colleagues suggested the “erasure detection index” (EDI) to detect test tampering. The EDI can be used with or without a continuity correction and is assumed to follow the standard normal distribution under the null hypothesis of no test tampering. When used without a continuity correction, the EDI often has inflated Type I error rates. When used with a continuity correction, the EDI has satisfactory Type I error rates, but smaller power compared with the EDI without a continuity correction. This article suggests three methods for detecting test tampering that do not rely on the assumption of a standard normal distribution under the null hypothesis. It is demonstrated in a detailed simulation study that the performance of each suggested method is slightly better than that of the EDI. The EDI and the suggested methods were applied to a real data set. The suggested methods, although more computation intensive than the EDI, seem to be promising in detecting test tampering.
American Educational Research Association, American Psychological Association, & National Council for Measurement in Education. (2014). Standards for educational and psychological testing. Washington DC: American Educational Research Association.
2.
BayarriM. J.BergerJ. O. (2000). P-values for composite null models. Journal of the American Statistical Association, 95, 1127-1142.
3.
BelovD. I. (2015). Robust detection of examinees with aberrant answer changes. Journal of Educational Measurement, 52, 437-456.
4.
BockR. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29-51.
5.
GelmanA.CarlinJ. B.SternH. S.RubinD. B. (2003). Bayesian data analysis. New York, NY: Chapman & Hall.
6.
GlasC. A. W.MeijerR. R. (2003). A Bayesian approach to person fit analysis in item response theory models. Applied Psychological Measurement, 27, 217–233.
7.
HansonB. A.HarrisD. J.BrennanR. L. (1994). A comparison of several statistical methods for examining allegations of copying (ACT Research Rep. Series no. 87-15). Iowa City, IA: American College Testing.
8.
LordF. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
9.
LordF. M.WingerskyM. S. (1984). Comparison of IRT true-score and equipercentile observed-score “equating”. Applied Psychological Measurement, 8, 453-461.
10.
MaynesD. (2013). Educator cheating and the statistical detection of group-based test security threats. In WollackJ. A.FremerJ. J. (Eds.), Handbook of test security (pp. 173-199). New York, NY: Routledge.
11.
MislevyR. J.BockR. D. (1982). Biweight estimates of latent ability. Educational and Psychological Measurement, 42, 725-737.
12.
PrimoliV.LiassouD.BishopN. S.NhouyvanisvongA. (2011, April). Erasure descriptive statistics and covariates. Paper presented at the annual meeting of the National Council of Measurement in Education, New Orleans, LA.
13.
QinS.CohenA. S. (2013, July). An empirical analysis for detection of possible test tampering. Paper presented at the annual meeting of the Psychometric Society, Arnhem, The Netherlands.
14.
R Core Team. (2015). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.
15.
RaoC. R. (1973). Linear statistical inference and its applications (2nd ed.). New York, NY: Wiley.
16.
SinharayS. (2015). Assessment of person fit for mixed-format tests. Journal of Educational and Behavioral Statistics, 40, 343-365.
17.
SinharayS. (2016). Bayesian model fit and model comparison. In van der LindenW. (Ed.), Handbook of item response theory: Vol. 2. Statistical tools. Boca Raton, FL: Chapman & Hall/CRC Press.
18.
SinharayS., & Duong, M. (2016). Detection of aberrant answer changes: An alternative to the indices based on Kullback-Leibler divergence. Manuscript submitted for publication.
19.
SinharayS.JohnsonM. S.SternH. S. (2006). Posterior predictive assessment of item response theory models. Applied Psychological Measurement, 30, 298-321.
20.
TendeiroJ. N.MeijerR. R. (2014). Detection of invalid test scores: The usefulness of simple nonparametric statistics. Journal of Educational Measurement, 51, 239–259.
21.
TiemannG. C.KingstonN. M. (2014). An exploration of answer changing behavior on a computer-based high-stakes achievement test. In KingstonN. M.ClarkA. K. (Eds.), Test fraud: Statistical detection and methodology (pp. 158-171). New York, NY: Routledge.
22.
ToribioS. G.AlbertJ. H. (2011). Discrepancy measures for item fit analysis in item response theory. Journal of Statistical Computation and Simulation, 81, 1345-1360.
23.
van der LindenW. J.JeonM. (2012). Modeling answer changes on test items. Journal of Educational and Behavioral Statistics, 37, 180-199.
24.
van der LindenW. J.JeonM.FerraraS. (2011). A paradox in the study of the benefits of test-item review. Journal of Educational Measurement, 48, 380-398.
25.
van der LindenW. J.LewisC. (2015). Bayesian checks on cheating on tests. Psychometrika, 80, 689-706.
26.
van der LindenW. J.SotaridonaL. (2006). Detecting answer copying when the regular response process follows a known response model. Journal of Educational and Behavioral Statistics, 31, 283-304.
27.
WarmT. A. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika, 54, 427-450.
28.
WollackJ.EckerlyC. (in press). Detecting test tampering at the group level. In CizekG. J.WollackJ. A. (Eds.), Handbook of detecting cheating on tests. Washington, DC: Routledge.
29.
WollackJ. A. (1997). A nominal response model approach for detecting answer copying. Applied Psychological Measurement, 21, 307–320.
30.
WollackJ. A.CohenA. S.EckerlyC. A. (2015). Detecting test tampering using item response theory. Educational and Psychological Measurement, 75, 931-953.
31.
WollackJ. A.CohenA. S.SerlinR. C. (2001). Defining error rates and power for detecting answer copying. Applied Psychological Measurement, 25, 385-404.
32.
ZopluogluC.DavenportE. C. (2012). The empirical power and Type I error rates of the GBT and ω indices in detecting answer copying on multiple-choice tests. Educational and Psychological Measurement, 72, 975-1000.
