Interactions Between Termination Criteria and Ability Estimators in Computerized Adaptive Testing

Abstract

Computerized adaptive testing (CAT) aims to optimize measurement by tailoring item administration to individual examinees. The efficiency and precision of a CAT heavily depend on the choice of ability ( $\theta$ ) estimator and the termination criterion (stopping rule). Prior research suggests these components interact, but comprehensive evaluations across varying item bank characteristics remain limited. This simulation study investigated the interactive effects of four $\theta$ estimators (maximum likelihood [MLE], weighted likelihood [WLE], maximum a posteriori [MAP], and expected a posteriori [EAP]) and four termination criteria (fixed-length, standard error of measurement [SEM], minimum information [MI], and change-in-estimate [Δ $\theta$ ]) on measurement bias, precision (RMSE), and test length. These combinations were evaluated across low- (100-item) and high-information (500-item) item banks with both flat and peaked information distributions using the three-parameter logistic model. The results demonstrated that the optimal CAT configuration is contingent on item bank size and shape. Across all conditions, WLE emerged as the most robust estimator, effectively neutralizing the boundary estimation issues of MLE and the shrinkage bias characteristic of Bayesian estimators. In high-information banks, the SEM and fixed-length rules yielded the lowest conditional RMSE and bias regardless of bank shape. However, in low-information peaked banks, the strict SEM rule frequently failed to reach precision targets at the θ extremes, resulting in inefficient, maximum-length tests. Under these sparse conditions, the Δθ rule paired with WLE provided a superior balance of accuracy and efficiency by halting administration when precision gains stagnated. Conversely, the MI rule consistently exhibited the highest bias and RMSE. These findings underscore that optimal CAT design is not a one-size-fits-all solution. For high-quality banks, WLE paired with an SEM or fixed-length rule is recommended. For lower-quality banks, practitioners should adopt a $\Delta \theta$ rule or a hybrid SEM approach to prevent inefficient test elongation.