Romero et al. (2015; see also Wollack, 1997) developed the ω statistic as a method for detecting unusually similar answers between pairs of examinees. For each pair, the ω statistic considers whether the observed number of similar answers is significantly larger than the expected number of similar answers. However, one limitation of ω is that it does not account for the particular items on which similar answers are observed. Therefore, in this study, we propose a weighted version of the ω statistic that takes this information into account. We compare the performance of the new and existing statistics using detailed simulations in which several factors are manipulated. Results show that while both the new and existing statistics are able to control the Type I error rate, the new statistic is more powerful, on average.
American Educational Research Association; American Psychological Association; National Council on Measurement in Education. (2014). Standards for educational and psychological testing (5th ed.). American Educational Research Association.
2.
BelovD. I.WollackJ. A. (2021). Graph theory approach to detect examinees involved in test collusion. Applied Psychological Measurement, 45(4), 253–267. https://doi.org/10.1177/01466216211013902
3.
BockR. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37(1), 29–51. https://doi.org/10.1007/BF02291411
4.
ChalmersR. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1–29. https://doi.org/10.18637/jss.v048.i06
5.
DrasgowF.LevineM. V.WilliamsE. A. (1985). Appropriateness measurement with polychotomous item response models and standardized indices. British Journal of Mathematical and Statistical Psychology, 38(1), 67–86. https://doi.org/10.1111/j.2044-8317.1985.tb00817.x
GorneyK.DengJ. (2024). aberrance: An R package for detecting aberrant behavior in test data. Applied Psychological Measurement, 48(4–5), 230–231. https://doi.org/10.1177/01466216241261707
8.
GorneyK.WollackJ. A. (2024). Using response times in answer similarity analysis. Journal of Educational and Behavioral Statistics. Advance online publication. https://doi.org/10.3102/10769986241248770
9.
MaynesD. (2014). Detection of non-independent test taking by similarity analysis. In KingstonN. M.ClarkA. K. (Eds.), Test fraud: Statistical detection and methodology (pp. 53–80). Routledge.
10.
RomeroM.RiascosA.JaraD. (2015). On the optimality of answer-copying indices: Theory and practice. Journal of Educational and Behavioral Statistics, 40(5), 435–453. https://doi.org/10.3102/1076998615595628
van der LindenW. J.SotaridonaL. (2006). Detecting answer copying when the regular response process follows a known response model. Journal of Educational and Behavioral Statistics, 31(3), 283–304. https://doi.org/10.3102/10769986031003283
13.
WollackJ. A. (1996). Detection of answer copying using item response theory. University of Wisconsin-Madison. ProQuest Dissertations & Theses Global.
14.
WollackJ. A. (1997). A nominal response model approach for detecting answer copying. Applied Psychological Measurement, 21(4), 307–320. https://doi.org/10.1177/01466216970214002
15.
WollackJ. A.MaynesD. D. (2017). Detection of test collusion using cluster analysis. In CizekG. J.WollackJ. A. (Eds.), Handbook of quantitative methods for detecting cheating on tests (pp. 124–150). Routledge. https://doi.org/10.4324/9781315743097-6
16.
XuY.CuiY.WangX.HuangM.LuoF. (2023). Confidence screening detector: A new method for detecting test collusion. Applied Psychological Measurement, 47(3), 237–252. https://doi.org/10.1177/01466216231165299
17.
ZopluogluC. (2017). Similarity, answer copying, and aberrance: Understanding the status quo. In CizekG. J.WollackJ. A. (Eds.), Handbook of quantitative methods for detecting cheating on tests (pp. 25–46). Routledge. https://doi.org/10.4324/9781315743097-2
18.
ZopluogluC.DavenportE. C. (2012). The empirical power and Type I error rates of the GBT and ω indices in detecting answer copying on multiple-choice tests. Educational and Psychological Measurement, 72(6), 975–1000. https://doi.org/10.1177/0013164412442941
