The idea that item response theory (IRT) models yield invariant parameter estimates is widely accepted among scholars interested in achieving truly scientific measurements in social and behavioral sciences. Starting from a conceptual and mathematical definition of invariance, this article presents a critical examination of the theoretical and empirical support for the property of invariance with regard to populations and samples of items and subjects by means of simulated data. The distinction between internal and external invariance is introduced to clarify the meaning and limitations of invariance in IRT models. Furthermore, the consequences of “giving in to the sirens’ call” of achieving invariant measurements in behavioral sciences are also discussed.
AdedoyinO. O.NentyH. J.ChilisaB. (2008). Investigating the invariance of item difficulty parameter estimates based on CTT and IRT. Educational Research and Reviews, 3(2), 83–93.
2.
BreithauptK.ZumboB. D. (2002). Sample invariance of the structural equation model and the item response model: A case study. Structural Equation Modeling, 9(3), 390–412. doi: 10.1207/S15328007SEM0903_5
3.
CamilliG.ShepardL. A. (1994). Methods for identifying biased test items. Thousand Oaks, CA: SAGE.
4.
ChernyshenkoO. S.StarkS.DrasgowF.RobertsB. W. (2007). Constructing personality scales under the assumptions of an ideal point response process: Toward increasing the flexibility of personality measures. Psychological Assessment, 19(1), 88–106. doi: 10.1037/1040–3590.19.1.88
5.
De AyalaR. J. (2009). Theory and practice of item response theory. New York, NY: Guilford Press.
6.
DeMarsC. (2010). Item response theory. Oxford, UK: Oxford University Press.
7.
EmbretsonS. E. (1996). Item response theory models and spurious interaction effects in factorial ANOVA designs. Applied Psychological Measurement, 20(3), 201–212. doi:10.1177/014662169602000302
8.
EmbretsonS. E. (1999). Issues in the measurement of cognitive abilities. In EmbretsonS.HershbergerS. (Eds.), The new rules of measurement: What every psychologist and educator should know (pp. 1–15). Mahwah, NJ: Psychology Press.
9.
EmbretsonS. E.ReiseS. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum Associates.
10.
EngelhardG. (1984). Thorndike, Thurstone, and Rasch: A comparison of their methods of scaling psychological and educational tests. Applied Psychological Measurement, 8(1), 21–38. doi: 10.1177/014662168400800104
11.
EngelhardG. (2008). Historical perspectives on invariant measurement: Guttman, Rasch, and Mokken. Measurement, 6(3), 155–189. doi: 10.1080/15366360802197792
12.
FanX. (1998). Item response theory and classical test theory: An empirical comparison of their item/person statistics. Educational and Psychological Measurement, 58(3), 357–381. doi: 10.1177/0013164498058003001
13.
FeyerabendP. (1975). Against method: Outline of an anarchistic theory of knowledge. London, UK: New Left Books.
14.
FischerG. H.MolenaarI. W. (Eds.). (1995). Rasch models: Foundations, recent developments, and applications. New York, NY: Springer.
15.
GratzerW. (2001). The undergrowth of science: Delusion, self-deception, and human frailty. Oxford, UK: Oxford University Press.
16.
GuttmanL. (1950). The basis for scalogram analysis. In StoufferS. A.GuttmanL.SuchmanE. A.LazarsfeldP. F.StarS. A.ClausenJ. A. (Eds.), Measurement and prediction (Vol. IV, pp. 60–90). Princeton, NJ: Princeton University Press.
17.
HambletonR. K. (1994). Item response theory: A broad psychometric framework for measurement advances. Psicothema, 6(3), 535–556.
18.
HambletonR. K.RussellJ. W. (1993). Comparison of classical test theory and item response theory and their applications to test development. Educational Measurement: Issues and Practice, 12(3), 38–47. doi: 10.1111/j.1745–3992.1993.tb00543.x
19.
HambletonR. K.SwaminathanH. (1985). Item response theory: Principles and applications. New York, NY: Springer.
20.
HambletonR. K.SwaminathanH.RogersH. J. (1991). Fundamentals of item response theory. Newbury Park, CA: SAGE.
21.
HollandP. W.HoskensM. (2003). Classical test theory as a first-order item response theory: Application to true-score prediction from a possibly nonparallel test. Psychometrika, 68(1), 123–149. doi: 10.1007/BF02296657
JonesL. V. (1960). Some invariant finding under the method of successive intervals. In GulliksenH.MessickS. (Eds.), Psychological scaling: Theory and applications (pp. 7–20). New York, NY: John Wiley & Sons.
24.
LevineM. V.RubinD. B. (1979). Measuring the appropriateness of multiple-choice test scores. Journal of educational statistics, 4(4), 269–290.
25.
LiuY.Maydeu-OlivaresA. (2013). Local dependence diagnostics in IRT modeling of binary data. Educational and Psychological Measurement, 73(2), 254–274. doi: 10.1177/0013164412453841
26.
LohrS. (2009). Sampling: Design and analysis. Boston, MA: Cengage Learning.
27.
LordF. (1952). A theory of test scores. Psychometric monographs, No.7. Richmond, VA: Psychometric Corporation.
28.
LordF. M. (1980). Applications of item response theory to practical testing problems. Mahwah, NJ: Lawrence Erlbaum Associates.
29.
Maydeu-OlivaresA.MoreraO.D’ZurillaT. J. (1999). Using graphical methods in assessing measurement invariance in inventory data. Multivariate Behavioral Research, 34(3), 397–420. doi: 10.1207/S15327906MBR3403_5
30.
McDonaldR. P. (1982). Linear versus models in item response theory. Applied Psychological Measurement, 6(4), 379–396. doi: 10.1177/014662168200600402
31.
McDonaldR. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum Associates.
MillsapR. E. (2008). Model-implied invariance in psychometrics: Be skeptical when theory suggests data are not needed. Measurement: Interdisciplinary Research and Perspectives, 6(3), 195–197. doi: 10.1080/15366360802265979
34.
MuñizJ.HambletonR. K. (1992). Medio siglo de teoría de respuesta a los ítems [Half a century of item response theory]. Anuario de Psicología, 52, 41–66.
35.
OrlandoM.ThissenD. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24(1), 50–64. doi: 10.1177/01466216000241003
36.
R Development Core Team. (2012). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing.
37.
RaschG. (1980). Probabilistic models for some intelligence and attainment tests (Expanded ed.). Chicago, IL: University of Chicago Press. (Original work published 1960)
38.
ReiseS. P.AinsworthA. T.HavilandM. G. (2005). Item response theory: Fundamentals, applications, and promise in psychological research. Current Directions in Psychological Science, 14(2), 95–101. doi: 10.1111/j.0963–7214.2005.00342.x
39.
ReiseS. P.HavilandM. G. (2005). Item response theory and the measurement of clinical change. Journal of Personality Assessment, 84(3), 228–238. doi: 10.1207/s15327752jpa8403_02
40.
RizopoulosD. (2006). LTM: An R package for latent variable modeling and item response theory analyses. Journal of Statistical Software, 17(5), 1–25. doi: 10.18637/jss.v017.i05
41.
RuppA. A.ZumboB. D. (2004). A note on how to quantify and report whether IRT parameter invariance holds: When Pearson correlations are not enough. Educational and Psychological Measurement, 64(4), 588–599. doi: 10.1177/0013164403261051
42.
RuppA. A.ZumboB. D. (2006). Understanding parameter invariance in unidimensional IRT models. Educational and Psychological Measurement, 66(1), 63–84. doi: 10.1177/0013164404273942
43.
ThorndikeE. L. (1922). On finding equivalent scores in tests of intelligence. Journal of Applied Psychology, 6, 29–33.
44.
ThurstoneL. L. (1927). The unit of measurement in educational scales. Journal of Educational Psychology, 18(8), 505–524. doi: 10.1037/h0072880
45.
van der LindenW. J.HambletonR. K. (Eds.). (1997). Handbook of modern item response theory. New York, NY: Springer.
46.
WrightB. D. (1968). Sample-free test calibration and person measurement. In Proceedings of the 1967 invitational conference on testing problems (pp. 85–101). Princeton, NJ: Educational Testing Service.
47.
WrightB. D. (1999). Fundamental measurement for psychology. In EmbretsonS.HershbergerS. (Eds.). The new rules of measurement: What every psychologist and educator should know (pp. 65–104). Mahwah, NJ: Psychology Press.
