Interactive problem-solving assessments allow examinees to learn from trial-and-error, potentially changing their response patterns when revisiting problem states due to learning or fatigue effects. This study introduces the Sequential Response Model With Growth parameters, which captures how examinees’ cognitive processes evolve through accumulated experience during task interaction via growth parameters. An empirical application to Complex problem-solving assessment reveals significant positive effects in intermediate states (mastering task mechanisms) and negative effects at the initial state (accumulated frustration). Simulation results confirmed robust parameter recovery when growth effects were present, while maintaining comparable performance when such effects were absent. These findings suggest that incorporating operational history enhances our understanding of dynamic problem-solving processes and provides insights for assessment design and educational practice.
AgrestiA. (2002). Categorical data analysis. John Wiley & Sons.
2.
BennettR. E.PerskyH.WeissA. R.JenkinsF. (2007). Problem solving in technology-rich environments: A report from the NAEP technology based assessment project (NCES 2007–466). U.S. Department of Education; National Center for Education Statistics.
3.
BrooksS. P.GelmanA. (1998). General methods for monitoring convergence of iterative simulations. Journal of Computational and Graphical Statistics, 7(4), 434–455. https://doi.org/10.1080/10618600.1998.10474787
4.
BuchnerA. (1995). Basic topics and approaches to the study of complex problem solving. In FrenschP. A.FunkeJ. (Eds.), Complex problem solving: The European perspective (pp. 27–63). Lawrence Erlbaum Associates.
5.
CareE.GriffinP.ScoularC.AwwalN.ZoanettiN. (2015). Collaborative problem solving tasks. In GriffinP.CareE. (Eds.), Assessment and teaching of 21st century skills: Methods and approach (pp. 85–104). Springer.
6.
ChenY. (2020). A continuous-time dynamic choice measurement model for problem-solving process data. Psychometrika, 85(4), 1052–1075. https://doi.org/10.1007/s11336-020-09734-1
7.
FuY.ZhanP.ChenQ.JiaoH. (2024). Joint modeling of action sequences and action time in computer-based interactive tasks. Behavior Research Methods, 56(5), 4293–4310. https://doi.org/10.3758/s13428-023-02178-2
GelmanA.CarlinJ. B.SternH. S.DunsonD. B.VehtariA.RubinD. B. (2013). Bayesian data analysis (3rd ed.). CRC Press. https://doi.org/10.1201/9780429258411
11.
GelmanA.MengX. L.SternH. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statistica Sinica, 6(4), 733–760.
12.
GelmanA.RubinD. B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7(4), 457–472. https://doi.org/10.1214/ss/1177011136
13.
GreiffS.FunkeJ. (2009). Measuring complex problem solving—The MicroDYN approach. In ScheuermannF. (Ed.), The transition to computer-based assessment—Lessons learned from large-scale surveys and implications for testing (pp. 157–163). Office for Official Publications of the European Communities. https://doi.org/10.11588/heidok.00015502
14.
GreiffS.WüstenbergS.FunkeJ. (2012). Dynamic problem solving: A new measurement perspective. Applied Psychological Measurement, 36(3), 189–213. https://doi.org/10.1177/0146621612439620
15.
GuttmanI. (1967). The use of the concept of a future observation in goodness-of-fit problems. Journal of the Royal Statistical Society: Series B (Methodological), 29(1), 83–100. https://doi.org/10.1111/j.2517-6161.1967.tb00676.x
16.
HanY.JiF.WangPLiuH. (2025). Assessing multiple abilities through process data in computer-based assessments: The multidimensional sequential response model (MSRM). Behavior Research Methods, 57, Article 152. https://doi.org/10.3758/s13428-025-02658-7
17.
HanY.LiuH.JiF. (2022). A sequential response model for analyzing process data on technology-based problem-solving tasks. Multivariate Behavioral Research, 57(6), 960–977. https://doi.org/10.1080/00273171.2021.1932403
18.
HanY.WilsonM. (2022). Analyzing student response processes to evaluate success on a technology-based problem-solving task. Applied Measurement in Education, 35(1), 33–45. https://doi.org/10.1080/08957347.2022.2034821
19.
HastingsW. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57(1), 97–109. https://doi.org/10.2307/2334940
LiuH.LiuY.LiM. (2018). Analysis of process data of PISA 2012 computer-based problem solving: Application of the modified multilevel mixture IRT model. Frontiers in Psychology, 9, Article 1372. https://doi.org/10.3389/fpsyg.2018.01372
26.
MayerR. E.WittrockM. C. (2006). Problem solving. In AlexanderP. A.WinneP. H. (Eds.), Handbook of educational psychology (2nd ed., pp. 287–304). Lawrence Erlbaum Associates.
27.
NokesT. J.SchunnC. D.ChiM. T. H. (2010). Problem solving and human expertise. In PetersonP.BakerE.McGawB. (Eds.), International encyclopedia of education (3rd ed., Vol. 5, pp. 265–272). Elsevier. https://doi.org/10.1016/B978-0-08-044894-7.00486-3
28.
NovickL. R.BassokM. (2005). Problem solving. In HolyoakK. J.MorrisonR. G. (Eds.), The Cambridge handbook of thinking and reasoning (pp. 321–349). Cambridge University Press.
29.
OECD. (2013). PISA 2012 assessment and analytical framework: Mathematics, reading, science, problem solving and financial literacy. https://doi.org/10.1787/9789264190511-en
30.
OECD. (2014). PISA 2012 results: Creative problem solving: Students’ skills in tackling real-life problems (Vol. 5). http://dx.doi.org/10.1787/9789264208070-en
RamalingamD.McCraeB.PhilpotR. (2017). The PISA 2012 assessment of problem solving. In CsapóB.FunkeJ. (Eds.), The nature of problem solving: Using research to inspire 21st century learning (pp. 75–91). OECD Publishing.
RubinD. B. (1984). Bayesianly justifiable and relevant frequency calculations for the applied statistician. The Annals of Statistics, 12(4), 1151–1172. https://doi.org/10.1214/aos/1176346785
35.
ShuZ.BergnerY.ZhuM.HaoJ.von DavierA. A. (2017). An item response theory analysis of problem-solving processes in scenario-based tasks. Psychological Test and Assessment Modeling, 59(1), 109–131.
SpiegelhalterD. J.BestN. G.CarlinB. P.Van Der LindeA. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64(4), 583–639. https://doi.org/10.1111/1467-9868.00353
38.
WangP.LiuH. (2024). Polytomous effectiveness indicators in complex problem-solving tasks and their applications in developing measurement model. Psychometrika, 89, 877–902. https://doi.org/10.1007/s11336-024-09963-8
39.
XiaoY.LiuH. (2024). A state response measurement model for problem-solving process data. Behavior Research Methods, 56(1), 258–277. https://doi.org/10.3758/s13428-022-02042-9
40.
YuanJ.XiaoY.LiuH. (2019). Assessment of collaborative problem solving based on process stream data: A new paradigm for extracting indicators and modeling dyad data. Frontiers in Psychology, 10, Article 369. https://doi.org/10.3389/fpsyg.2019.00369
41.
ZhanP.QiaoX. (2022). Diagnostic classification analysis of problem-solving competence using process data: An item expansion method. Psychometrika, 87(4), 1529–1547. https://doi.org/10.1007/s11336-022-09855-9
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
