Sage Journals: Discover world-class research

Abstract

The article presents a new computerized adaptive testing (CAT) procedure for use with batteries of unidimensional tests. At each step of testing, the estimate of a certain ability is updated on the basis of the response to the latest administered item and the current estimates of all other abilities measured by the battery. The information deriving from these abilities is incorporated into an empirical prior that is updated each time that new estimates of the abilities are computed. In two simulation studies, the performance of the proposed procedure is compared with that of a standard procedure for CAT with batteries of unidimensional tests. The proposed procedure yields more accurate ability estimates in fixed-length CATs, and a reduction of test length in variable-length CATs. These gains in accuracy and efficiency increase with the correlation between the abilities measured by the batteries.

Keywords

item response theory computerized adaptive testing test battery collateral information empirical prior test speededness

Get full access to this article

View all access options for this article.

References

Anselmi

Robusto

Stefanutti

de Chiusole

(2016). An upgrading procedure for adaptive assessment of knowledge. Psychometrika, 81(2), 461–482, https://doi.org/10.1007/s11336-016-9498-9

Bock

R. D.

Mislevy

R. J.

(1988). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6(4), 431–444, https://doi.org/10.1177/014662168200600405

Brown

Weiss

(1977). An adaptive testing strategy for achievement test batteries (Research report No. 77-6). University of Minnesota – Minneapolis, MN 55455: Department of Psychology.

Cheng

(2009). When cognitive diagnosis meets computerized adaptive testing: CD-CAT. Psychometrika, 74(4), 619–632, https://doi.org/10.1007/s11336-009-9123-2

Colledani

Anselmi

Robusto

(2018). Using item response theory for the development of a new short form of the Eysenck Personality Questionnaire-Revised. Frontiers in Psychology, 9, 1834, https://doi.org/10.3389/fpsyg.2018.01834

Colledani

Anselmi

Robusto

(2019a). Development of a new abbreviated form of the Eysenck Personality Questionnaire-Revised with multidimensional item response theory. Personality and Individual Differences, 149, 108–117, https://doi.org/10.1016/j.paid.2019.05.044

Colledani

Anselmi

Robusto

(2019). Using multidimensional item response theory to develop an abbreviated form of the Italian version of Eysenck’s IVE questionnaire. Personality and Individual Differences, 142, 45–52, https://doi.org/10.1016/j.paid.2019.01.032

Colledani

Robusto

Anselmi

(2018). Development of a new abbreviated form of the Junior Eysenck personality questionnaire-revised. Personality and Individual Differences, 120, 159–165, https://doi.org/10.1016/j.paid.2017.08.037

Deville

(1993). Flow as a testing ideal. Rasch Measurement Transactions, 7(3), 308.

10.

Falmagne

J.-C.

Doignon

J.-P.

(1988). A class of stochastic procedures for the assessment of knowledge. British Journal of Mathematical and Statistical Psychology, 41(1), 1–23, https://doi.org/10.1111/j.2044-8317.1988.tb00884.x

11.

Fuller

W. A.

(1987). Measurement error models. John Wiley & Sons.

12.

Gialluca

K. A.

Weiss

(1979). Efficiency of an adaptive inter-subtest branching strategy in the measurement of classroom achievement (Research report No. 79-6). University of Minnesota – Minneapolis, MN 55455: Department of Psychology.

13.

Haberman

S. J.

(2008). When can subscores have value? Journal of Educational and Behavioral Statistics, 33(2), 204–229, https://doi.org/10.3102/1076998607302636

14.

Huebner

(2010). An overview of recent developments in cognitive diagnostic computer adaptive assessments. Practical Assessment, Research & Evaluation, 15(3).

15.

Liu

(2015). Comparisons of subscoring methods in computerized adaptive testing: A simulation study. Unpublished doctoral dissertation. https://libres.uncg.edu/ir/uncg/f/Liu_uncg_0154D_11840_pdf

16.

Liu

(2007). Comparing multi-dimensional and uni-dimensional computer adaptive strategies in psychological and health assessment. Unpublished doctoral dissertation, http://d-scholarship.pitt.edu/8092/

17.

Maurelli

Weiss

(1981). Factors influencing the psychometric characteristics of an adaptive testing strategy for test batteries (Research report No. 81-4). University of Minnesota – Minneapolis, MN 55455: Department of Psychology.

18.

Reise

S. P.

Henson

J. M.

(2000). Computerization and adaptive administration of the NEO PI-R. Assessment, 7(4), 347–364, https://doi.org/10.1177/107319110000700404

19.

Rossi Ferrario

Panzeri

Anselmi

Vidotto

(2019). Development and psychometric properties of a short form of the Illness Denial Questionnaire. Psychology Research and Behavior Management, 12, 727–739, https://doi.org/10.2147/PRBM.S207622

20.

Sinharay

(2010). How often do subscores have added value? Results from operational and simulated data. Journal of Educational Measurement, 47(2), 150–174, https://doi.org/10.1111/j.1745-3984.2010.00106.x

21.

van der Linden

W. J

. (1999). Empirical initialization of the trait estimator in adaptive testing. Applied Psychological Measurement, 23(1), 21–29, https://doi.org/10.1177/01466219922031149

22.

van der Linden

W. J

. (2008a). Sequencing an adaptive test battery (Research report No. 08-05). Law School Admission Council, Inc.

23.

van der Linden

W. J

. (2008b). Some new developments in adaptive testing technology. Zeitschrift für Psychologie/Journal of Psychology, 216(1), 3–11, https://doi.org/10.1027/0044-3409.216.1.3

24.

van der Linden

W. J

. (2008c). Using response times for item selection in adaptive testing. Journal of Educational and Behavioral Statistics, 33(1), 5–20, https://doi.org/10.3102/1076998607302626

25.

van der Linden

W. J.

Glas

C. A. W.

(Eds.). (2000). Computerized adaptive testing: Theory and practice. Kluwer.

26.

Weiss

D. J.

(Ed.). (1983). New horizons in testing: Latent trait test theory and computerized adaptive testing. Academic Press, Inc.

27.

Xie

(2019). The impact of collateral information on ability estimation in an adaptive test battery. Unpublished doctoral dissertation, https://ir.uiowa.edu/cgi/viewcontent.cgi?article=8424&context=etd

28.

Yao

(2013). Comparing the performance of five multidimensional CAT selection procedures with different stopping rules. Applied Psychological Measurement, 37(1), 3–23, https://doi.org/10.1177/0146621612455687

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.14 MB

Enhancing Computerized Adaptive Testing with Batteries of Unidimensional Tests

Abstract

Keywords

Get full access to this article

References

Supplementary Material