The present study illustrates an extension of Kamata's (2001) restricted form of the hierarchical generalized linear model that provides a multilevel longitudinal Rasch measurement model appropriate for use with polytomous responses. This extension can be used to assess average and interindividual change in the latent trait of interest, concurrently with an assessment of the invariance of item locations over time. Responses of 1,353 college students to three subscales (namely, measures of depression/ anxiety, stress, and well-being) of the Outcome Questionnaire were used for demonstration purposes. Interpretation of the results is provided, and the benefits of using a multilevel item response theory model with longitudinal data are discussed.
Get full access to this article
View all access options for this article.
References
1.
Adams, R. J., Wilson, M., & Wu, M. (1997). Multilevel item response models: An approach to errors in variables regression. Journal of Educational and Behavioral Statistics, 22, 47-76.
2.
Andrich, A. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561-573.
3.
Beretvas, S. N., Kearney, L. K., & Barón, A. (2002). A shortened form of the Outcome Questionnaire: A validation of scores across ethnic groups. Unpublished manuscript.
4.
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 397-424). Reading, MA: Addison-Wesley.
5.
Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29-51.
6.
Bock, R. D., Muraki, E., & Pfeiffenberger, W. (1988). Item pool maintenance in the presence of item parameter drift. Journal of Educational Measurement, 25, 275-285.
7.
Bryk, A. G., & Raudenbush, S. W. (1987). Application of hierarchical linear models to assessing change. Psychological Bulletin, 101, 147-158.
8.
Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items. Thousand Oaks, CA: Sage.
9.
Cheong, Y. F., & Raudenbush, S. W. (2000). Measurement and structural models for children's problem behaviors. Psychological Methods, 5, 477-495.
10.
Donoghue, J. R., & Isham, S. P. (1998). A comparison of procedures to detect item parameter drift. Applied Psychological Measurement, 22, 33-51.
11.
Fischer, G. H. (1973). Linear logistic test model as an instrument in educational research. Acta Psychologica, 27, 359-374.
12.
Fischer, G. H. (1989). An IRT based model for dichotomous longitudinal data. Psychometrika, 54, 599-624.
13.
Fischer, G. H., & Parzer, P. (1991). An extension of the rating scale model with an application to the measurement of treatment effects. Psychometrika, 56, 637-651.
14.
Fischer, G. H., & Ponocny, I. (1994). An extension of the partial credit model with an application to the measurement of change. Psychometrika, 59, 177-192.
15.
Fox, J. P., & Glas, C. A. W. (2003). Bayesian modeling of measurement error in predictor variables using item response theory. Psychometrika, 68, 169-191.
16.
Glück, J., & Spiel, C. (1997). Item response models for repeated measures designs: Application and limitations of four different approaches. Methods of Psychological Research Online, 2. Retrieved June 8, 2005, from http://www.mpr-online.de
17.
Goldstein, H. (1983). Measuring changes in educational attainment over time: Problems and possibilities. Journal of Educational Measurement, 25, 31-45.
18.
Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer Nijhoff.
19.
Kamata, A. (1998). Some generalizations of the Rasch model: An application of the hierarchical generalized linear model. Doctoral dissertation, Michigan State University, East Lansing.
20.
Kamata, A. (2001). Item analysis by the hierarchical generalized linear model. Journal of Educational Measurement, 38, 79-93.
21.
Kreft, I., & De Leeuw, J. (1998). Introduction to multilevel modeling. London: Sage.
22.
Lambert, M. J., Burlingame, G. M., Umphress, V., Hansen, N. B., Vermeersch, D. A., Clouse, G. C., et al. (1996). The reliability and validity of a new psychotherapy outcome questionnaire. Clinical Psychology and Psychotherapy, 3, 249-258.
23.
Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
24.
Mellenbergh, G. J. (1995). Conceptual notes on models for discrete polytomous item responses. Applied Psychological Measurement, 19, 91-100.
25.
Motl, R. W., & DiStefano, C. (2002). Longitudinal invariance of self-esteem and method effects associated with negatively worded items. Structural Equation Modeling, 9, 562-578.
26.
Mueller, R. M., Lambert, M. J., & Burlingame, G. M. (1998). Construct validity of the Outcome Questionnaire: A confirmatory factor analysis. Journal of Personality Assessment, 70, 248-262.
27.
Muraki, E. (1990). Fitting a polytomous item response model to Likert-type data. Applied Psychological Measurement, 14, 59-71.
28.
Pastor, D. A. (2003). The use of multilevel IRT modeling in applied research: An illustration. Applied Measurement in Education, 16, 223-243.
29.
Ponocny, I. (2002). On the applicability of some IRT models for repeated measurement designs: Conditions, consequences, and goodness-of-fit tests. Methods of Psychological Research Online, 7. Retrieved June 8, 2005, from http://www.mpr-online.de
30.
Raudenbush, S. W., Brennan, R. T., & Barnett, R. C. (1995). A multivariate hierarchical model for studying psychological change within married couples. Journal of Family Psychology, 9, 161-174.
31.
Raudenbush, S. W., & Bryk, A. G. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.
32.
Raudenbush, S. W., Bryk, A. G., Cheong, Y. F., & Congdon, R. (2004). HLM 6: Hierarchical linear and nonlinear modeling [Computer software]. Chicago: Scientific Software International.
33.
Rijmen, F., & De Boeck, P. (2002). The random weights linear logistic test model. Applied Psychological Measurement, 26, 271-285.
34.
Rijmen, F., Tuerlinckx, F., De Boeck, P., & Kuppens, P. (2003). A nonlinear mixed model framework for item response theory. Psychological Methods, 8, 185-205.
35.
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph, No. 17.
36.
Samejima, F. (1972). A general model for freeresponse data. Psychometric Monograph, No. 18.
37.
SAS Institute. (2004). The GLIMMIX procedure (experimental) [Computer program and manual]. Retrieved from http://www.sas.com/statistics/doc.html
38.
Shin, S.-H. (2003). A polytomous nonlinear mixed model for item analysis. Unpublished doctoral dissertation, University of Texas, Austin.
39.
Snijders, T. A. B., & Bosker, R. J. (2000). Multilevel analysis: An introduction to basic and advanced multilevel modeling. London: Sage.
40.
Stevens, J. (1999). Intermediate statistics: A modern approach (2nd ed.). Mahwah, NJ: Lawrence Erlbaum.
41.
Wallace, D., & Green, S. B. (2002). Analysis of repeated measures designs with linear mixed models. In D. S. Moskowitz & S. L. Hershberger (Eds.), Modeling intraindividual variability with repeated measures data: Methods and applications. Mahwah, NJ: Lawrence Erlbaum.
42.
Wells, C. S., Subkoviak, M. J., & Serlin, R. C. (2002). The effect of item parameter drift on examinee ability estimates. Applied Psychological Measurement, 26, 77-87.
43.
Williams, N. J., & Beretvas, S. N. (2006). DIF identification using HGLM for polytomous items. Applied Psychological Measurement, 30, 22-42.
