We present a novel application of a generalized item response tree model to investigate test takers’ answer change behavior. The model allows us to simultaneously model the observed patterns of the initial and final responses after an answer change as a function of a set of latent traits and item parameters. The proposed application is illustrated with large-scale mathematics test items. We also describe how the estimated results can be used to study the benefits of answer change and to further detect potential academic cheating.
CrockerL.BensonJ. (1980). Does answer changing affect test quality?Measurement and Evaluation in Guidance, 12, 223–239.
6.
De BoeckP.PartchevI. (2012). IRTrees: Tree-based item response models of the GLMM family. Journal of Statistical Software, 48, 1–28.
7.
DebeerD.JanssenR.De BoeckP. (2014). Modeling responses and omissions within a tree-based IRT framework. Manuscript submitted for publication.
8.
DiTrapaniJ.JeonM.De BoeckP.PartchevI. (2016). Attempting to differentiate fast and slow intelligence: Using generalized item response trees to examine the role of speed on intelligence tests. Intelligence, 56, 82–92.
9.
FooteR.BelinkyC. (1972). It pays to switch? Consequences of changing answers on multiple choice examinations. Psychological Reports, 31, 667–673.
10.
FriedmanS. J.CookG. L. (1995). Is an examinee’s cognitive style related to the impact of answer changing on multiple-choice tests?Journal of Experimental Education, 63, 199–213.
11.
JeonM.De BoeckP. (2015). A generalized item response tree model for psychological assessments. Behavior Research Methods, 48, 1070–1085.
12.
JeonM.RijmenF.Rabe-HeskethS. (2014). Flexible item response theory modeling with the R package flirt. Applied Psychological Measurement, 38, 404.
13.
LordF. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
14.
MathewsC. O. (1929). Erroneous first impressions on objective tests. Journal of Educational Measurement, 20, 280–286.
15.
McMorrisR. F.DeMersL. P.SchwarzS. P. (1987). Attitudes, behaviors, and reasons for changing responses following answer-changing instruction. Journal of Educational Measurement, 24, 131–143.
16.
McMorrisR. F.SchwarzS. P.RichichiR. V.FischerM.BuczekN. M.ChevalierC. L.MelandK. A. (1991). Why do young students change answers on tests?(ERIC Document Reproduction Service, ED 342 803). Washington, D.C.: ERIC Clearinghouse. Retrieved fromhttp://www.eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED342803
17.
MiliaL. D. (2007). Benefiting from multiple-choice exams: The positive impact of answer switching. Educational Psychology, 27, 607–615.
18.
MillmanJ.BishopC. H.EbelR. (1975). An analysis of testwiseness. Educational and Psychological Measurement, 12, 251–254.
19.
PapanastasiouE.ReckaseM. (2008). Item review as a non-traditional method of item analysis. Paper presented at the Annual conference of the Psychometric Society, Durham, NH.
20.
PartchevI.De BoeckP. (2012). Can fast and slow intelligence be differentiated?Intelligence, 40, 23–32.
21.
PlieningerH.MeiserT. (2014). Validity of multiprocess IRT models for separating content and response styles. Educational and Psychological Measurement, 74, 875–899.
22.
ReckaseM. D. (2009). Multidimensional item response theory. New York, NY: Springer-Verlag.
23.
SegallD. O. (1996). Multidimensional adaptive testing. Psychometrika, 61, 331–354.
24.
van der LindenW. J.JeonM. (2012). Modeling answer changes on test items. Journal of Educational and Behavioral Statistics, 37, 180–199.
25.
van der LindenW. J.JeonM.FerreraS. (2011). A paradox in the study of the benefits of test-item review. Journal of Educational Measurement, 48, 380–398.
26.
VispoelW. (1998). Reviewing and changing answers on computer-adaptive and self-adaptive vocabulary tests. Journal of Educational Measurement, 35, 329–346.
