This pedagogical article is intended to explain the similarities and differences between the parameterizations of two multilevel measurement model (MMM) frameworks. The conventional two-level MMM that includes item indicators and models item scores (Level 1) clustered within examinees (Level 2) and the two-level cross-classified MMM (in which item scores are cross-classified by two Level 2 classifications including item and examinee) are discussed. A small subset of National Assessment of Educational Progress 4th grade mathematics item scores is used to demonstrate use of the MMMs for assessing facets of the validity of accommodated test scores. The models’ similarities and distinctions are emphasized as well as the flexibility of the models’ extensions.
AbediJ. (2002). Standardized achievement tests and English language learners: Psychometric issues. Educational Assessment, 8, 231-257.
2.
AbediJ.BaileyA.ButlerF.Castellon-WellingtonM.LeonS.MirochaJ. (2005). The validity of administering large-scale content assessments to English language learners: An investigation from three perspectives (CSE Report No. 663). Los Angeles, CA: National Center for Research on Evaluation, Standards, and Student Testing.
3.
AbediJ.LordC.PlummerJ. (1997). Final report of language background as a variable in NAEP mathematics performance (CRESST Technical Report No. 429). Los Angeles: University of California.
4.
AdamsR. J.WilsonM. (1996). Formulating the Rasch model as a mixed coefficients multinomial logit. In EngelhardG.WilsonM. (Eds.), Objective measurement: Theory and practice (Vol. 3, pp. 143-166). Norwood, NJ: Ablex.
5.
AdamsR. J.WilsonM.WuM. (1997). Multilevel item response models: An approach to errors in variables regression. Journal of Educational and Behavioral Statistics, 22, 47-76.
6.
BeretvasS. N. (2008). Cross-classified random effects models. In O’ConnellA. A.Betsy McCoachD. (Eds.), Multilevel modeling of educational data (pp. 161-197). Charlotte, SC: Information Age.
7.
BeretvasS. N. (2010). Cross-classified and multiple membership random effects models. In HoxJ.RobertsJ. K. (Eds.), The handbook of advanced multilevel analysis (pp. 313-334). New York, NY: Routledge.
8.
BeretvasS. N.KamataA. (2005). The multilevel measurement model: Introduction to the special issue. Journal of Applied Measurement, 6, 247-254.
9.
BeretvasS. N.MeyersJ. L.RodriguezR. A. (2005). The cross-classified multilevel measurement model: An explanation and demonstration. Journal of Applied Measurement, 6, 322-341.
10.
BoltS. E.ThurlowM. L. (2004). Five of the most frequently allowed testing accommodations in state policy. Remedial and Special Education, 25, 141-152.
11.
CheongY. F.RaudenbushS. W. (2000). Measurement and structural models for children’s problem behaviors. Psychological Methods, 5, 477-495.
12.
ClapperA. T.MorseA. B.LazarusS. S.ThompsonS. J.ThurlowM. L. (2005). 2003 state policies on assessment participation and accommodations for students with disabilities (Synthesis Report 56). Minneapolis: University of Minnesota, National Center on Educational Outcomes. Retrieved from http://cehd.umn.edu/NCEO/OnlinePubs/Synthesis56.html
13.
FischerG. H. (1995). Linear logistic test model. In FischerG. H.MolenaarI. W. (Eds.), Rasch models: Foundations, recent developments and applications (pp. 131-155). New York, NY: Springer-Verlag.
HoxJ. J. (2002). Multilevel analysis: Techniques and applications. Mahwah, NJ: Erlbaum.
16.
KamataA. (2001). Item analysis by the hierarchical generalized linear model. Journal of Educational Measurement, 38, 79-93.
17.
LepikM. (1990). Algebraic word problems: Role of linguistic and structural variables. Educational Studies in Mathematics, 21(1), 83-90.
18.
LuppescuS. (2002, April). DIF detection in HLM. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.
19.
McDonnellL. M.McLaughlinM. J.MorisonP. (Eds). (1997). Educating one and all: Students with disabilities and standards-based reform. Washington, DC: National Academy Press.
20.
MeuldersM.XieY. (2004). Person-by-item predictors. In De BoeckP.WilsonM. (Eds.), Explanatory item response models: A generalized linear and nonlinear approach (pp. 213-240). New York, NY: Springer.
21.
National Center for Education Statistics. (2003, April). NAEP validity studies: An agenda for NAEP validity research (Working Paper No. 2003-07). Washington, DC: Author.
22.
National Center for Education Statistics. (2007, May). NAEP 2005 mathematics, reading, and science assessments restricted-use data files data companion, Mathematics (NCES 2007-485, NCES 2007-486); Reading (NCES 2007-487, NCES 2007-488); Science (NCES 2007-489, NCES 2007-490); and Grade 12 (NCES 2007-484) by RogersA.M.StoeckelJ. J.Washington, DC: Author.
23.
PhillipsS. E. (1994). High-stakes testing accommodations: Validity versus disabled rights. Applied Measurement in Education, 7, 93-120.
24.
RasbashJ.BrowneW. J. (2001). Modeling non-hierarchical structures. In LeylandA. H.GoldsteinH. (Eds.), Multilevel modeling of health statistics (pp. 93-105). Chichester, England: Wiley.
25.
RasbashJ.BrowneW. J.GoldsteinH.YangM.PlewisI.HealyM.. . . LewisT. (2000). A user’s guide to MLwiN (2nd ed.). London, England: Institute of Education.
26.
RijmenF.TuerlinckxF.De BoeckP.KuppensP. (2003). A nonlinear mixed model framework for item response theory. Psychological Methods, 8, 185-205.
27.
SwansonD. B.ClauserB. E.CaseS. M.NungsterR. J.FeathermanC. (2002). Analysis of differential item functioning (DIF) using hierarchical logistic regression models. Journal of Educational and Behavioral Statistics, 27, 53-75.
28.
Van den NoortgateW.De BoeckP. (2005). Assessing and explaining differential item functioning using logistic mixed models. Journal of Educational and Behavioral Statistics, 30, 443-464.
29.
Van den NoortgateW.De BoeckP.MeuldersM. (2003). Cross-classification multilevel logistic models in psychometrics. Journal of Educational and Behavioral Statistics, 28, 369-386.
30.
WilliamsN. J.BeretvasS. N. (2006). DIF identification using HGLM for polytomous items. Applied Psychological Measurement, 30, 22-42.
