In this article, the authors illustrate the use of mixed-model item response theory (MM-IRT) and explain its usefulness for analyzing organizational surveys. The authors begin by giving an overview of MM-IRT, focusing on both technical aspects and previous organizational applications. Guidance is provided on how researchers can use MM-IRT to check scoring assumptions, identify the influence of systematic responding that is unrelated to item content (i.e., response sets), and evaluate individual and group difference variables as predictors of class membership. After summarizing the current body of research using MM-IRT to address problems relevant to organizational researchers, the authors present an illustration of the use of MM-IRT with the Job Descriptive Index (JDI), focusing on the use of the ‘‘?’’ response option. Three classes emerged, one most likely to respond in the positive direction, one most likely to respond in the negative direction, and another most likely to use the ‘‘?’’ response. Trust in management, job tenure, age, race, and sex were considered as correlates of class membership. Results are discussed in terms of the applicability of MM-IRT and future research endeavors.
Abrams, L.C., Cross, R., Lesser, E., & Levin, D.Z. ( 2003). Nurturing interpersonal trust in knowledge-sharing networks . Academy of Management Executive, 17, 64-71.
2.
Adams, J.E., Carter, N.T., Wolford, K., Zickar, M.J., & Highhouse, S. ( 2009). The job descriptive index: A reliability generalization study. Presented at the 24th Annual Meeting of the Society for Industrial and Organizational Psychology, New Orleans, LA.
3.
Akaike, H. ( 1973). Information theory and the extension of the maximum likelihood principle. In V. N. Petrov & F. Scáki (Eds.), Second international symposium of information theory (pp. 267-281). Budapest: Akadémiai Kiadó.
4.
Andrich, D. ( 2005). The Rasch model explained. In S. Alagumalai, D. D. Durtis & N. Hungi (Eds.), Applied Rasch measurement: A book of exemplars. New York: Springer.
5.
Balzer, W.K., Kihm, J.A., Smith, P.C., Irwin, J.L., Bachiochi, P.D., Robie, C., et al. (1997). User’s manual for the Job Descriptive Index (JDI; 1997 Revision) and the Job in General (JIG) scales. OHBowling Green State University.
6.
Bock, R.D. ( 1972). Estimating item parameters and latent ability when the responses are scored in two or more nominal categories. Psychometrika , 37, 29-51.
7.
Bock, R., & Jones, L.V. ( 1968). The measurement and prediction of judgment and choice . San Francisco: Holden Day.
8.
Borsboom, D. ( 2005). Measuring the mind: Conceptual issues in contemporary psychometrics. New York: Cambridge University Press.
9.
Bowling, N.A., Hendricks, E.A., & Wagner, S.H. ( 2008). Positive and negative affectivity and facet satisfaction: A meta-analysis. Journal of Business and Psychology, 23, 115-125.
10.
Bozdogan, H. ( 1987). Model selection and Akaike’s Information Criteria: The general theory and its analytic extensions. Psychometrika , 52, 345-370.
11.
Chan, K.-Y., Drasgow, F., & Sawin, L.L. ( 1999). What is the shelf life of a test? The effect of time on the psychometrics of a cognitive ability test battery. Journal of Applied Psychology, 84, 610-619.
12.
Chernyshenko, O.S., Stark, S., Chan, K.-Y., Drasgow, F., & Williams, B. ( 2001). Fitting item response theory models to two personality inventories: Issues and insights. Multivariate Behavioral Research, 36, 523-562.
13.
Clark, M.C., & Payne, R.L. ( 1997). The nature and structure of workers’ trust in management . Journal of Organizational Behavior, 18, 205-224.
14.
Cohen, J. ( 1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
15.
Collins, W.C., Raju, N.S., & Edwards, J.E. ( 2000). Assessing differential functioning in a satisfaction scale. Journal of Applied Psychology, 85, 451-461.
16.
Connolly, J.J., & Viswesvaran, C. (2000). The role of affectivity in job satisfaction: A meta-analysis. Personality and Individual Differences, 29, 265-281.
17.
Cooper-Hakim, A., & Viswesvaran, C. (2005). The construct of work commitment: Testing an integrative framework. Psychological Bulletin, 131, 241-259.
18.
Cramér, H. ( 1946). Mathematical methods of statistics. Uppsala, Sweden: Almqvist & Wiksells.
19.
Cruickshank, P.J. ( 1984). A stress arousal mood scale for low vocabulary subjects: A reworking of Mackay et al. (1978). British Journal of Psychology, 75, 89-94.
20.
Detert, J.R., & Burris, E.R. ( 2007). Leadership behavior and employee voice: Is the door really open? Academy of Management Journal, 50, 869-884.
21.
Dirks, K.T., & Ferrin, D.L. ( 2002). Trust in leadership: Meta-analytic findings and implications for research and practice. Journal of Applied Psychology, 87, 611-628.
22.
Donovan, M.A., Drasgow, F., & Probst, T.M. ( 2000). Does computerizing paper-and-pencil job attitude scales make a difference? New IRT analyses offer insight. Journal of Applied Psychology, 85, 305-313.
23.
Dubois, B., & Burns, J.A. ( 1975). An analysis of the meaning of the question mark response category in attitude scales. Educational and Psychological Measurement, 35, 869-884.
24.
Eid, M., & Rauber, M. ( 2000). Detecting measurement invariance in organizational surveys . European Journal of Psychological Assessment, 16, 20-30.
25.
Eid, M., & Zickar, M.J. ( 2007). Detecting response styles and faking in personality and organizational assessments by mixed Rasch models. In M. Von Davier & C. H. Carstensen (Eds.), Multivariate and mixture distribution Rasch models: Extensions and applications . New York: Springer.
26.
Embretson, S.E., & Reise, S.P. ( 2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.
27.
Goldberg, G. ( 1971). Response format in attitude scales. Unpublished manuscript , Northwestern University.
28.
Hanisch, K.A. ( 1992). The job descriptive index revisited: Questions about the question mark. Journal of Applied Psychology, 77, 377-382.
29.
Hart, R., & Hart, M. ( 2002). Statistical process control for health care. Pacific Grove, CA: Duxbury.
30.
Hernandez, A., Drasgow, F., & Gonzalez-Roma, V. (2004). Investigating the functioning of a middle category by means of a mixed-measurement model. Journal of Applied Psychology, 89, 687-699.
31.
Jablin, F.M. ( 1979). Superior-subordinate communication: The state of the art. Psychological Bulletin, 86, 1201-1222.
32.
Judge, T.A., Heller, D., & Mount, M.K. ( 2002). Five-factor model of personality and job satisfaction: A meta-analysis. Journal of Applied Psychology, 87, 530-541.
33.
Kaplan, K.J. ( 1972). On the ambivalence-indifference problem in attitude theory: A suggested modification of the semantic differential technique. Psychological Bulletin, 77, 361-372.
34.
Krosnick, J.A., Holbrook, A.L., Berent, M.K., Carson, R.T., Hanemann, W.M., Kopp, R.J., et al. (2002). The impact of ‘‘no opinion’’ response options on data quality: Non-attitude reduction or an invitation to satisfice?Public Opinion Quarterly, 66, 371-403.
35.
Kutner, M.H., Nachtsheim, C.J., Neter, J., & Li, W. ( 2005). Applied linear statistical models (5th ed.). Boston: McGraw-Hill.
Levin, D.Z., Whitener, E.M., & Cross, R. ( 2006). Perceived trustworthiness of knowledge sources: The moderating impact of relationship length. Journal of Applied Psychology , 91, 1163-1171.
38.
Maij-de Meij, A.M., Kelderman, H., & van der Flier, H. (2008). Fitting a mixture item response theory model to personality questionnaire data: Characterizing latent classes and investigating possibilities for improving prediction. Applied Psychological Measurement, 32, 611-631.
39.
Masters, G.N. ( 1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149-174.
Mislevy, R., & Huang, C.W. ( 2007). Measurement models as narrative structures. In M. Von Davier & C. H. Carstensen (Eds.), Multivariate and mixture distribution Rasch models: Extensions and applications. New York: Springer.
45.
Ostroff, C., & Kozlowski, S. ( 1992). Organizational socialization as a learning process: The role of information acquisition. Personnel Psychology, 45, 849-874.
46.
Payne, R., & Clark, M. ( 2003). Dispositional and situational determinants of trust in two types of managers. International Journal of Human Resource Management, 14, 128-138.
47.
Personnel Decisions International. (1997). Selection systems test scale manual. Unpublished document. Minneapolis, MN: Author.
48.
Reise, S.P., & Gomel, J.N. ( 1995). Modeling qualitative variation within latent trait dimensions: Application of mixed-measurement to personality assessment. Multivariate Behavioral Research, 30, 341-358.
49.
Reise, S.P., & Waller, N.G. ( 2003). How many IRT parameter does it take to model personality items? Psychological Methods, 8, 164-184.
50.
Rost, J. ( 1991). A logistic mixture distribution for polychotomous item responses. British. Journal of Mathematical and Statistical Psychology, 44, 75-92.
51.
Rost, J. ( 1997). Logistic mixture models. In W. J. van der Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 449-463). New York: Springer.
52.
Rost, J., & von Davier, M. (1994). A conditional item-fit index for Rasch models. Applied Psychological Measurement , 18, 171-182.
53.
Rousseau, D.M., Sitkin, S.B., Burt, R.S., & Camerer, C. ( 1998). Not so different after all: A cross-discipline view of trust. Academy of Management Review, 23, 393-404.
54.
Ryan, A.M., Horvath, M., Ployhart, R.E., Schmitt, N., & Slade, L. ( 2000). Hypothesizing differential item functioning in global employee opinion surveys. Personnel Psychology, 53, 531-562.
55.
Schwarz, G.E. ( 1978). Estimating the dimension of a model. Annals of Statistics, 6, 461-464.
56.
Sheskin, D.J. ( 2004). Handbook of parametric and non-parametric statistical procedures (3rd ed.). Boca Raton, FL: Chapman & Hall/CRC.
57.
Smit, A., Kelderman, H., & van der Flier, H. (2003). Latent trait latent class analysis of an Eysenck personality questionnaire. Methods of Psychological Research Online, 8, 23-50.
58.
Smith, P.C., Kendall, L.M., & Hulin, C.L. ( 1969). The measurement of satisfaction in work and retirement: A strategy for the study of attitudes. Skokie, IL: Rand-McNally.
59.
Tay, L., Drasgow, F., Rounds, J., & Williams, B. A. (2009). Fitting measurement models to vocational interest data: Are dominance models ideal? Journal of Applied Psychology , 94, 1287-1304.
60.
Thissen, D., Chen, W.-H., & Bock, R.D. ( 2003). Multilog (version 7) [Computer software]. Lincolnwood, IL: Scientific Software International .
61.
Thissen, D. & Steinberg, L. ( 1986). A taxonomy of item response models. Psychometrika, 51, 567-577.
62.
Vermunt, J.K., & Magidson, J. ( 2005). LatentGOLD (v4.5). Belmont, MA: Statistical Innovations, Inc.
63.
von Davier, M. ( 2001). WINMIRA 2001: Windows mixed Rasch model analysis [Computer software and User manual]. Kiel, the Netherlands: Institute for Science Education.
64.
von Davier, M., Rost, J., & Carstensen, C.H. (2007). Introduction: Extending the Rasch model. In M. von Davier & C. H. Carstensen (Eds.), Multivariate and mixture distribution Rasch models: Extensions and applications. New York: Springer.
65.
Wang, M., & Russell, S.S. ( 2005). Measurement equivalence of the Job Descriptive Index across Chinese and American workers: Results from confirmatory factor analysis and item response theory. Educational and Psychological Measurement , 65, 709-732.
66.
White, L.A., Nord, R.D., Mael, F.A., & Young, M.C. ( 1993). The Assessment of Background and Life Experiences (ABLE) . In T. Trent & J. H. Laurence (Eds.), Adaptability screening for the armed forces (pp. 101-162). Washington, DC: Office of Assistant secretary of Defense (Force Management and Personnel).
67.
Wilde, G. ( 1970). Neurotische labiliteit gemeten volgens de vragenlijstmethode [Neurotic ability measured by the questionnaire method]. Amsterdam: Van Rossen.
68.
Worthy, M. ( 1969). Note on scoring midpoint responses in extreme response style scores. Psychological Reports, 24, 189-190.
69.
Zickar, M.J. ( 2001). Conquering the next frontier: Modeling personality data with item response theory. In B. Roberts & R. Hogan (Eds.), Applied personality psychology: The intersection of personality and I/O psychology (pp. 141-158). Washington, DC: American Psychological Association .
70.
Zickar, M.J., & Broadfoot, A.A. (2009). The partial revival of a dead horse? Comparing classical test theory and item response theory. In C. E. Lance & R. J. Vandenberg (Eds.), Statistical and methodological myths and urban legends: Doctrine, verity, and fable in the organizational and social sciences (pp. 37-60). New York: Routledge.
71.
Zickar, M.J., Gibby, R.E., & Robie, C. ( 2004). Uncovering faking samples in applicant, incumbent, and experimental data sets: An application of mixed-model item response theory . Organizational Research Methods, 7, 168-190.
72.
Zickar, M.J., & Robie, C. ( 1999). Modeling faking good on personality items: An item-level analysis. Journal of Applied Psychology, 84, 551-563.
