Sage Journals: Discover world-class research

Abstract

Pseudo-guessing parameters are present in item response theory applications for many educational assessments. When sample size is not sufficiently large, the guessing parameters may be ignored from the analysis. This study examines the impact of ignoring pseudo-guessing parameters on measurement invariance analysis, specifically, on item difficulty, item discrimination, and mean and variance of ability distribution. Results show that when non-zero guessing parameters are ignored from the measurement invariance analysis, item discrimination estimates tend to decrease particularly for more difficult items, and item difficulty estimates decrease unless the items are highly discriminating and difficult. As the guessing parameter increases, the size of the decrease in item discrimination and difficulty tends to increase, and the estimated mean and variance of ability distribution tend to be inaccurate. When two groups have heterogeneous ability distributions, ignoring the guessing parameter affects the reference group and the focal group differently. Implications of result findings are discussed.

Keywords

measurement invariance analysis pseudo-guessing parameter differential item functioning item response theory multiple-group factor analysis

Get full access to this article

View all access options for this article.

References

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.

Bentler

P. M.

(1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246.

Brennan

R. L.

(2006). Perspectives on the evolution and future of educational measurement. In Brennan

R. L.

(Ed.), Educational measurement (pp. 1–16). Praeger Publishers.

Budgell

G. R.

Raju

N. S.

Quartetti

D. A.

(1995). Analysis of differential item functioning in translated assessment instruments. Applied Psychological Measurement, 19(4), 309–321.

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255.

Cohen

A. S.

Kim

S. H.

Wollack

J. A.

(1996). An investigation of the likelihood ratio test for detection of differential item functioning. Applied Psychological Measurement, 20(1), 15–26.

Crocker

Algina

(2008). Introduction to classical and modern test theory. Cengage Learning.

DeMars

C. E.

Jurich

D. P.

(2015). The interaction of ability differences and guessing when modeling differential item functioning with the Rasch model: Conventional and tailored calibration. Educational and Psychological Measurement, 75(4), 610–633.

Downing

S. M.

Haladyna

T. M.

(2006). Handbook of test development. Lawrence Erlbaum Associates.

10.

Finch

W. H.

(2005). The MIMIC model as a method for detecting DIF: Comparison with Mantel-Haenszel, SIBTEST, and the IRT likelihood ratio. Applied Psychological Measurement, 29(4), 278–295.

11.

Finch

W. H.

French

B. F.

(2014). The impact of group pseudo-guessing parameter differences on the detection of uniform and nonuniform DIF. Psychological Test and Assessment Modeling, 56(1), 25–44.

12.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.

13.

Hulin

C. L.

Lissak

R. I.

Drasgow

(1982). Recovery of two- and three-parameter logistic item characteristic curves: A Monte Carlo Study. Applied Psychological Measurement, 6(3), 249–260.

14.

Kamata

Bauer

D. J.

(2008). A note on the relation between factor analytic and item response theory models. Structural Equation Modeling, 15(1), 136–153.

15.

Koh

K. H.

Zumbo

B. D.

(2008). Multi-group confirmatory factor analysis for testing measurement invariance in mixed item format data. Journal of Modern Applied Statistical Methods, 7(2), 471–477.

16.

Maydeu-Olivares

Joe

(2006). Limited information goodness-of-fit testing in multidimensional contingency tables. Psychometrika, 71(4), 713–732.

17.

Meredith

(1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58(4), 525–543.

18.

Millsap

R. E.

(2011). Statistical approaches to measurement invariance. Routledge.

19.

Millsap

R. E.

Yun-Tein

(2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39(3), 479–515.

20.

Muthén

L. K.

Muthén

B. O.

(1998–2019). Mplus user’s guide (8th ed.).

21.

Paek

Cui

Gübeş

Yang

(2018). Estimation of an IRT model by Mplus for dichotomously scored responses under different estimation methods. Educational and Psychological Measurement, 78(4), 569–588.

22.

Paek

Lin

(2019). Detection rates of the M2 test for nonzero lower asymptotes under normal and nonnormal ability distributions in the applications of IRT. Applied Psychological Measurement, 43(1), 84–88.

23.

SAS Institute Inc. (2013). SAS 9.4 [Computer software].

24.

Steiger

J. H.

(1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25(2), 173–180.

25.

Takane

de Leeuw

(1987). On the relationship between item response theory and factor analysis of discretized variables. Psychometrika, 52(3), 393–408.

26.

Tucker

L. R.

Lewis

(1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38(1), 1–10.

27.

Woods

C. M.

Harpole

(2015). How item residual heterogeneity affects tests for differential item functioning. Applied Psychological Measurement, 39(4), 251–263.

28.

Yen

C. J.

C. H.

(2011). A multiple-group confirmatory factor analysis of the scores for online social presence: Do they measure the same thing across cultural groups? Journal of Educational Computing Research, 44(2), 219–242.

29.

Yen

W. M.

(1981). Using simulation results to choose a latent trait model. Applied Psychological Measurement, 5(2), 245–262.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

7.86 MB