Identifying Problematic Item Characteristics With Small Samples Using Mokken Scale Analysis

Abstract

Researchers frequently use Mokken scale analysis (MSA), which is a nonparametric approach to item response theory, when they have relatively small samples of examinees. Researchers have provided some guidance regarding the minimum sample size for applications of MSA under various conditions. However, these studies have not focused on item-level measurement problems, such as violations of monotonicity or invariant item ordering (IIO). Moreover, these studies have focused on problems that occur for a complete sample of examinees. The current study uses a simulation study to consider the sensitivity of MSA item analysis procedures to problematic item characteristics that occur within limited ranges of the latent variable. Results generally support the use of MSA with small samples (N around 100 examinees) as long as multiple indicators of item quality are considered.

Keywords

Mokken scale analysis small samples nonparametric IRT survey research

Get full access to this article

View all access options for this article.

References

Chou

Y.-H.

Lee

C.-P.

Liu

C.-Y.

Hung

C.-I.

(2017). Construct validity of the depression and somatic symptoms scale: Evaluation by Mokken scale analysis. Neuropsychiatric Disease and Treatment, 13, 205–211. https://doi.org/10.2147/NDT.S118825

Lee

C.-P.

Chou

Y.-H.

Liu

C.-Y.

Hung

C.-I.

(2017). Dimensionality of the Chinese hospital anxiety depression scale in psychiatric outpatients: Mokken scale and factor analyses. International Journal of Psychiatry in Clinical Practice, 21(4), 283–291. https://doi.org/10.1080/13651501.2017.1311350

Meijer

R. R.

Baneke

J. J.

(2004). Analyzing psychopathology items: A case for nonparametric item response theory modeling. Psychological Methods, 9(3), 354–368. https://doi.org/10.1037/1082-989X.9.3.354

Mokken

R. J.

(1971). A theory and procedure of scale analysis. De Gruyter.

Muraki

(2018). Generalized partial credit model. In van der Linden

W. J.

(Ed.), Handbook of item response theory (Vol. 1, pp. 127–138). CRC Press.

Santor

D. A.

Ramsay

J. O.

(1998). Progress in the technology of measurement: Applications of item response models. Psychological Assessment, 10(4), 345–359.

Sijtsma

Meijer

R. R.

Andries van der Ark

(2011). Mokken scale analysis as time goes by: An update for scaling practitioners. Personality and Individual Differences, 50(1), 31–37. https://doi.org/10.1016/j.paid.2010.08.016

Sijtsma

Molenaar

I. W.

(2002). Introduction to nonparametric item response theory (Vol. 5). SAGE.

Sijtsma

van der Ark

L. A.

(2017). A tutorial on how to do a Mokken scale analysis on your test and questionnaire data. British Journal of Mathematical and Statistical Psychology, 70(1), 137–158. https://doi.org/10.1111/bmsp.12078

10.

Straat

J. H.

van der Ark

L. A.

Sijtsma

(2014). Minimum sample size requirements for Mokken scale analysis. Educational and Psychological Measurement, 74(5), 809–822. https://doi.org/10.1177/0013164414529793

11.

Tillema

Bouwmeester

Verkoeijen

Heijltjes

(2021). Psychometric properties of the short-form CART: Investigating its dimensionality through a Mokken scale analysis. Thinking Skills and Creativity, 39, 100793. https://doi.org/10.1016/j.tsc.2021.100793

12.

van der Ark

L. A.

(2007). Mokken scale analysis in R. Journal of Statistical Software, 20(11), 1–19.

13.

van der Ark

L. A.

(2012). New developments in Mokken scale analysis in R. Journal of Statistical Software, 48(5), 1–27.

14.

Vaughan

Grace

(2018). A Mokken scale analysis of the peer physical examination questionnaire. Chiropractic & Manual Therapies, 26(1), 6. https://doi.org/10.1186/s12998-018-0176-0

15.

Watson

Egberink

I. J. L.

Kirke

Tendeiro

J. N.

Doyle

(2018). What are the minimal sample size requirements for Mokken scaling? An empirical example with the Warwick- Edinburgh Mental Well-being Scale. Health Psychology and Behavioral Medicine, 6(1), 203–213. https://doi.org/10.1080/21642850.2018.1505520

16.

Wind

S. A.

(2017). An instructional module on Mokken scale analysis. Educational Measurement Issues and Practice, 36(2), 50–66. https://doi.org/10.1111/emip.12153

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.12 MB