This article presents a new method to detect copying on a standardized multiple-choice exam. The method combines two statistical approaches in successive stages. The first stage uses Kullback-Leibler divergence to identify examinees, called subjects, who have demonstrated inconsistent performance during an exam. For each subject the second stage uses the K-Index to search for a possible source of the responses. Both stages apply a hypothesis test given a significance level. Monte Carlo methods are applied to approximate empirical distributions and then compute critical values providing a low Type I error rate and a good copying-detection rate. The results with both simulated and empirical data demonstrate the effectiveness of this approach.
Armstrong, R.D., & Shi, M. ( 2009). A parametric cumulative sum statistic for person fit . Applied Psychological Measurement, 33, 391-410.
2.
Belov, D.I., Pashley, P.J., Lewis, C., & Armstrong, R.D. (2007). Detecting aberrant responses with Kullback-Leibler distance. In K. Shigemasu , A. Okada, T. Imaizumi, & T. Hoshino (Eds.), New trends in psychometrics (pp. 7-14). Tokyo, Japan: Universal Academy Press.
3.
Burnham, K.P., & Anderson, D.R. (2001). Kullback-Leibler information as a basis for strong inference in ecological studies. Wildlife Research, 28, 111-119.
4.
Chang, H.-H., & Ying, Z. ( 1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213-229.
5.
Chernoff, H., & Lehmann, E.L. ( 1954). The use of maximum likelihood estimates in χ2 tests for goodness-of-fit. Annals of Mathematical Statistics, 25, 579-586.
6.
Cover, T.M., & Thomas, J.A. ( 1991). Elements of information theory. New York, NY: John Wiley.
7.
Eggen, T.J.H.M. ( 1999). Item selection in adaptive testing with the sequential probability ratio test. Applied Psychological Measurement , 23, 249-261.
8.
Frary, R.B. ( 1993). Statistical detection of multiple-choice answer copying: Review and commentary. Applied Measurement in Education, 6, 153-165.
9.
Guttman, L. ( 1944). General theory and methods for matric factoring. Psychometrika, 9, 1-16.
10.
Harnisch, D.L., & Linn, R.L. ( 1981). Analysis of item response patterns: Questionable test data and dissimilar curriculum practices. Journal of Educational Measurement, 18, 133-146.
11.
Holland, P.W. ( 1996). Assessing unusual agreement between the incorrect answers of two examinees using the K-Index: Statistical theory and empirical support . Princeton, NJ: Educational Testing Service.
12.
Karabatsos, G. ( 2003). Comparing the aberrant response detection performance of thirty-six person-fit statistics. Applied Measurement in Education, 16, 277-298.
13.
Kullback, S., & Leibler, R.A. ( 1951). On information and sufficiency. Annals of Mathematical Statistics, 22, 79-86.
14.
Lehmann, E.L. ( 1986). Testing statistical hypotheses (2nd ed.). New York, NY: John Wiley .
15.
Levine, M.V., & Rubin, D.B. ( 1979). Measuring the appropriateness of multiple-choice test scores. Journal of Educational Statistics, 4, 269-290.
16.
Lewis, C., & Thayer, D.T. ( 1998). The power of the K-Index (or PMIR) to detect copying . Princeton, NJ: Educational Testing Service.
17.
Lord, F. ( 1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
18.
Massey, F.J., Jr. ( 1951). The Kolmogorov-Smirnov test for goodness of fit. Journal of the American Statistical Society, 46, 68-78.
19.
Meijer, R.R. ( 1996). Person-fit research: An introduction. Applied Measurement in Education, 9, 3-8.
20.
Meijer, R.R., & Sijtsma, K. ( 2001). Methodology review: Evaluating person fit. Applied Psychological Measurement, 25, 107-135.
21.
Sotaridona, L.S., & Meijer, R.R. ( 2002). Statistical properties of the K-Index for detecting answer copying. Journal of Educational Measurement, 39, 115-132.
22.
Sotaridona, L.S., van der Linden, W.J., & Meijer, R.R. ( 2006). Detecting answer copying using the Kappa statistic. Applied Psychological Measurement, 30, 412-431.
23.
Swartz, T. ( 1992). Goodness-of-fit tests using Kullback-Leibler information . Communications in Statistics: Simulation and Computation , 21, 711-729.
24.
van der Ark, L.A., Emons, W.H.M., & Sijtsma, K. ( 2008). Detecting answer copying using alternate test forms and seat locations in small-scale examinations. Journal of Educational Measurement, 45, 99-117.
25.
van der Flier, H. ( 1982). Deviant response patterns and comparability of test scores . Journal of Cross-Cultural Psychology, 13, 267-298.
26.
van Krimpen-Stoop, E.M.L.A., & Meijer, R.R. ( 2000). Detecting person misfit in adaptive testing using statistical process control techniques. In W. J. van der Linden & C. A. W. Glas (Eds.), New developments in computerized adaptive testing: Theory and practice (pp. 201-219). Boston, MA: Kluwer-Nijhoff.
27.
van Krimpen-Stoop, E.M.L.A., & Meijer, R.R. ( 2001). CUSUM-based person-fit statistics for adaptive testing . Journal of Educational and Behavioral Statistics, 26, 2, 199-218.
28.
Weissman, A. ( 2007). Mutual information item selection in adaptive classification testing. Educational and Psychological Measurement, 67, 41-58.
29.
Woolf, B. ( 1972). The log likelihood ratio test (the G-Test). Annals of Human Genetics, 35, 397-409.
