Using Item Response Theory and Model—Data Fit to Conceptualize Differential Item and Person Functioning for Students With Disabilities

Abstract

The major purpose of this study is to describe a conceptual framework for examining differential item functioning (DIF) and differential person functioning (DPF) as types of model—data misfit within the context of assessing students with disabilities. Specifically, DIF and DPF can be viewed through the lens of residual analyses. Residual analyses can be used to explore DIF (item fit) as well as extended to explore DPF (person fit). One of the advantages of this conceptual framework is that the size of the subgroups can be quite small with interpretable results produced even for individuals. To illustrate this conceptual framework, Rasch measurement theory is used as the item response theory model. Methodological and theoretical issues are discussed. Data from a high-stakes assessment in mathematics in Georgia (Grade 7, geometry items) are used to illustrate the conceptual framework for students with disabilities. The substantive research questions used to illustrate the conceptual framework addresses whether modifications in test administration (use of resource guides or calculators) influence item functioning for students with disabilities (DIF analyses) and whether students with various disabilities differentially benefit from these modifications (DPF analyses).

Keywords

differential item functioning differential person functioning invariant measurement Rasch measurement theory

Get full access to this article

View all access options for this article.

References

Andrich, D. (1989). Distinctions between assumptions and requirements of measurement in social sciences. In J. A. Keats , H. Taft, R. A. Heath , & S. H. Lovibond (Eds.), Mathematical and theoretical systems (pp. 7-16). North-Holland: Elsevier Science.

Bock, R.D. , & Jones, L.V. (1968). The measurement and prediction of judgment and choice. San Francisco: Holden-Day .

Bock, R.D. , & Moustaki, I. (2007). Item response theory in a general framework. In C. R. Rao & S. Sinharay (Eds.), Psychometrics: Vol. 26. Handbook of statistics (pp. 469-514). Amsterdam: Elsevier .

Camilli, G. (2006). Test fairness. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 221-256) Westport, CT: Praeger.

De Boeck, P. , & Wilson, M. (2004). Explanatory item response models: A generalized linear and nonlinear approach. New York: Springer.

Emons, W.H.M. , Sijtsma, K. , & Meijer, R.R. (2005). Global, local, and graphical person-fit analysis using person-response functions. Psychological Methods, 10, 101-119.

Engelhard, G. (1994). Historical views of the concept of invariance in measurement theory. In M. Wilson (Ed.), Objective measurement: Theory into practice (Vol. 2, pp. 73-99). Norwood, NJ: Ablex .

Engelhard, G. , Fincher, M. , & Domaleski, C.S. (2006). Examining the reading and mathematics performance of students with disabilities under modified conditions: The Georgia Department of Education modification research study. Atlanta: Georgia Department of Education.

Hambleton, R.K. (1989). Principles and selected applications of item response theory. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 147-200). New York: American Council on Education and Macmillan .

10.

Karabatsos, G. (2000). A critique of Rasch residual fit statistics. Journal of Applied Measurement, 1, 152-176.

11.

Karabatsos, G. (2003). Comparing the aberrant response detection performance of thirty-six person-fit statistics. Applied Measurement in Education, 16, 277-298.

12.

Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.

13.

Linacre, J.M. (2007). A user's guide to FACETS: Rasch-model computer programs. Chicago: MESA Press.

14.

Rasch, G. (1977). On specific objectivity: An attempt at formalizing the request for generality and validity of scientific statements. Danish Yearbook of Philosophy, 14, 58-94.

15.

Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests. Chicago: University of Chicago Press. (Original work published 1960 by the Danish Institute for Educational Research)

16.

Reckase, M.D. (1979). Unifactor latent trait models applied to multifactor tests: Results and implications. Journal of Educational Statistics , 4, 207-230.

17.

Smith, R.M. (1991). The distributional properties of Rasch standardized residuals. Educational and Psychological Measurement, 48, 657-667.

18.

Sijtsma, K. , & Molenaar, I.W. (2002). Introduction to nonparametric item response theory. Thousand Oaks, CA: Sage.

19.

Swaminathan, H. , Hambleton, R.K. , & Rogers, H.J. (2007). Assessing the fit of item response theory models . In C. R. Rao & S. Sinharay (Eds.), Psychometrics: Vol. 26. Handbook of statistics (pp. 683-718). Amsterdam: Elsevier.

20.

Tashakkori, A. , & Creswell, J.W. (2007). Exploring the nature of research questions in mixed methods research. Journal of Mixed Methods Research, 1, 207-211.

21.

U.S. Department of Education. (2007, April 9). Title I: Improving the academic achievement of the disadvantaged: Individuals with Disabilities Act (IDEA): Final rule. Federal Register, 72(67), 17748-17781.

22.

van der Linden, W. J. , & Hambleton, R. K. (1997). (Eds.). Handbook of modern item response theory. New York: Springer-Verlag.

23.

Wright, B.D. (1984). Despair and hope for educational measurement. Contemporary Education Review, 3, 281-288.

24.

Wright, B.D. , & Masters, G. (1982). Rating scale analysis: Rasch measurement. Chicago: MESA Press.