Assessing individual change is an important topic in both psychological and educational measurement. An adaptive measurement of change (AMC) method had previously been shown to exhibit greater efficiency in detecting change than conventional nonadaptive methods. However, little work had been done to compare different procedures within the AMC framework. This study introduced a new item selection criterion and two new test statistics for detecting change with AMC that were specifically designed for the paradigm of hypothesis testing. In two simulation sets, the new methods for detecting significant change improved on existing procedures by demonstrating better adherence to Type I error rates and substantially better power for detecting relatively small change.
Bartroff, J., Finkelman, M., & Lai, T.L. ( 2008). Modern sequential analysis and its applications to computerized adaptive testing. Psychometrika, 73, 473-486.
4.
Bereiter, C. ( 1963). Some persisting dilemmas in the measurement of change . In C. Harris (Ed.), Problems in measuring change (pp. 3-20). Madison: University of Wisconsin Press.
5.
Bock, R.D. ( 1976). Basic issues in the measurement of change. In D. N. M. de Gruijter & L. J. T. van der Kamp (Eds.), Advances in psychological and educational measurement (pp. 75-96). New York: John Wiley.
6.
Burr, J.A., & Nesselroade, J.R. (1990). Change measurement. In A. von Eye (Ed.), Statistical methods in longitudinal research (vol. 1) (pp. 3-34). Boston: Academic Press.
7.
Chang, H.H., Qian, J., & Ying, Z. ( 2001). a-stratified multistage CAT with b-blocking. Applied Psychological Measurement, 25, 333-341.
8.
Chang, H.H., & Stout, W. ( 1993). The asymptotic posterior normality of the latent trait in an IRT model. Psychometrika, 58, 37-52.
9.
Chang, H.H., & Ying, Z. ( 1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20, 213-229.
10.
Collins, L. M., & Sayer, A. G. (Eds.). (2001). New methods for the analysis of change. Washington, DC: American Psychological Association.
11.
Cover, T.M., & Thomas, J.A. ( 1991). Elements of information theory. New York: John Wiley.
12.
Cronbach, L.J., & Furby, L. ( 1970). How we should measure ‘‘change’’-Or should we? Psychological Bulletin, 74, 68-80.
13.
Eggen, T.J.H.M. ( 1999). Item selection in adaptive testing with the sequential probability ratio test. Applied Psychological Measurement , 23, 249-261.
14.
Embretson, S.E. ( 1991a). A multidimensional latent trait model for measuring learning and change. Psychometrika, 56, 495-515.
15.
Embretson, S.E. ( 1991b). Implications of a multidimensional latent trait model for measuring change. In L. M. Collins & J. L. Horn (Eds.), Best methods for the analysis of change (pp. 184-197). Washington, DC: American Psychological Association.
16.
Embretson, S.E. ( 1995). A measurement model for linking individual learning to processes and knowledge: Application to mathematical reasoning. Journal of Educational Measurement, 32, 277-294.
17.
Embretson, S.E., & Reise, S.P. ( 2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.
18.
Fischer, G.H. ( 1976). Some probabilistic models for measuring change. In D. N. M. de Gruijter & L. J. T. van der Kamp (Eds.), Advances in psychological and educational measurement (pp. 97-110). New York: John Wiley.
19.
Hummel-Rossi, B., & Weinberg, S.L. (1975). Practical guidelines in applying current theories to the measurement of change. I. Problems in measuring change and recommended procedures (Ms. No. 916). JSAS Catalog of Selected Documents in Psychology , 5, 226.
20.
Kim-Kang, G., & Weiss, D.J. ( 2007). Comparison of computerized adaptive testing and classical methods for measuring individual change. In D. J. Weiss (Ed.), Proceedings of the 2007 GMAC Conference on Computerized Adaptive Testing. Available from www.psych.umn.edu/psylabs/CATCentral/
Kingsbury, G.G., & Zara, A.R. ( 1989). Procedures for selecting items for computerized adaptive tests. Applied Measurement in Education, 2, 359-375.
23.
Lai, T.L., & Shih, M.C. ( 2004). Power, sample size and adaptation considerations in the design of group sequential trials. Biometrika, 91, 507-528.
24.
Lord, F.M. ( 1963). Elementary models for measuring change. In C. W. Harris (Ed.), Problems in measuring change (pp. 21-38). Madison: University of Wisconsin Press.
25.
Lord, F.M. ( 1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
26.
Lord, F.M., & Novick, M.R. ( 1968). Statistical theories of mental test scores. Menlo Park, CA: Addison-Wesley.
27.
Manning, W.H., & DuBois, P.H. (1962). Correlation methods in research on human learning. Perceptual and Motor Skills, 15, 287-321.
28.
McDonald, R.P. ( 1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum.
29.
McNemar, Q. ( 1958). On growth measurement. Educational and Psychological Measurement, 18, 47-55.
30.
Muraki, E. ( 1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159-176.
31.
Samejima, F. ( 1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, No. 17.
32.
Siegmund, D. ( 1985). Sequential analysis: Tests and confidence intervals. New York: Springer.
33.
Stocking, M.L., & Lewis, C. ( 1998). Controlling item exposure conditional on ability in computerized adaptive testing. Journal of Educational and Behavioral Statistics , 23, 57-75.
34.
Sympson, J.B., & Hetter, R.D. ( 1985). Controlling item exposure rates in computerized adaptive testing. In Proceedings of the 27th annual meeting of the Military Testing Association (pp. 937-977). San Diego, CA: Navy Personnel Research and Development Center .
35.
Tucker, L.R., Damarin, F., & Messick, S. ( 1966). A base-free measure of change. Psychometrika , 31, 457-473.
36.
van der Linden, W.J. ( 2000). Constrained adaptive testing with shadow tests. In W. J. van der Linden & C. A. W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 27-52). Boston: Kluwer.
37.
van der Linden, W. J., & Glas, C. A. W. (Eds.). (2000). Computerized adaptive testing: Theory and practice. Boston: Kluwer.
38.
Wainer, H. (Ed.). (2000). Computerized adaptive testing: A primer (2nd ed.). Mahwah, NJ: Lawrence Erlbaum.
39.
Weiss, D.J. ( 1983). Introduction. In D. J. Weiss (Ed.), New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 1-9). New York: Academic Press.
40.
Weiss, D.J., & Kingsbury, G.G. (1984). Application of computerized adaptive testing to educational problems. Journal of Educational Measurement, 21, 361-375.
41.
Willett, J.B. ( 1994). Measurement of change. In T. Husen & T. N. Postlethwaite (Eds.), The international encyclopedia of education (2nd ed., pp. 671-678). Oxford, UK: Pergamon .
42.
Willett, J.B. ( 1997). Measuring change: What individual growth modeling buys you. In E. Amsel & K. A. Renninger (Eds.), Change and development: Issues of theory, method, and application (pp. 213-243). Mahwah, NJ : Lawrence Erlbaum.
43.
Yen, W.M. ( 1986). The choice of scale for educational measurement: An IRT perspective. Journal of Educational Measurement, 23, 299-325.
