The traditional application of the Montreal Cognitive Assessment uses total scores in defining cognitive impairment levels, without considering variations in item properties across populations. Item response theory (IRT) analysis provides a potential solution to minimize the effect of important confounding factors such as education. This research applies IRT to investigate the characteristics of Montreal Cognitive Assessment items in a randomly selected, culturally homogeneous sample of 1,873 older persons with diverse educational backgrounds. Any formal education was used as a grouping variable to estimate multiple-group IRT models. Results showed that item characteristics differed between people with and without formal education. Item functioning of the Cube, Clock Number, and Clock Hand items was superior in people without formal education. This analysis provided evidence that item properties vary with education, calling for more sophisticated modelling based on IRT to incorporate the effect of education.
Alzheimer’s Disease International. (2013). The global impact of dementia 2013–2050: Policy brief for heads of Government. London, England: Author.
2.
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: Author.
3.
BalsisS.ChoudhuryT. K.GeraciL.BengeJ. F.PatrickC. J. (2018). Alzheimer’s disease assessment: A review and illustrations focusing on item response theory techniques. Assessment, 25, 360-373. doi:10.1177/1073191117745125
4.
BentlerP. M. (2007). Covariance structure models for maximal reliability of unit-weighted composites. In Sik-YumL. (Ed.), Handbook of latent variable and related models (Vol. 1, pp. 3-20). Amsterdam, Netherlands: Elsevier.
5.
BezdicekO.CervenkovaM.MooreT. M.Stepankova GeorgiH.SulcZ.WolkD. A.WeintraubD. A.MobergP. J.JechR.KopecekM.RoalfD. R. (2018). Determining a short form Montreal Cognitive Assessment (s-MoCA) Czech version: validity in mild cognitive impairment Parkinson’s disease and cross-cultural comparison. Assessment. doi: 10.1177/1073191118778896.
6.
BockR. D.AitkinM. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443-459.
ChalmersR. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6), 1-29.
9.
ChengY.YuanK.-H.LiuC. (2012). Comparison of reliability measures under factor analysis and item response theory. Educational and Psychological Measurement, 72, 52-67.
10.
ChoS. J.SuhY.LeeW. Y. (2016). An NCME instructional module on latent DIF analysis using mixture item response models. Educational Measurement: Issues and Practice, 35(1), 48-61.
11.
ChuL. W.NgK. H.LawA. C.LeeA. M.KwanF. (2015). Validity of the Cantonese Chinese Montreal Cognitive Assessment in Southern Chinese. Geriatrics & Gerontology International, 15, 96-103.
12.
ColeM. W.LaurentP.StoccoA. (2013). Rapid instructed task learning: A new window into the human brain’s unique capacity for flexible cognitive control. Cognitive, Affective, & Behavioral Neuroscience, 13, 1-22. doi:10.3758/s13415-012-0125-7
13.
DavisD. H.CreavinS. T.YipJ. L.Noel-StorrA. H.BrayneC.CullumS. (2015). Montreal Cognitive Assessment for the diagnosis of Alzheimer’s disease and other dementias. Cochrane Database of Systematic Reviews, 2015, CD010775. doi:10.1002/14651858.CD010775.pub2
14.
de AyalaR. J. (2009). The theory and practice of item response theory. New York, NY: Guilford Press.
15.
DeMarsC. (2010). Item response theory (Series in understanding measurement). Oxford; NY: Oxford University Press.
16.
EmbretsonS. E.ReiseS. P. (2013). Item response theory for psychologists. New York, NY: Psychology Press.
17.
JulayanontP.BrousseauM.ChertkowH.PhillipsN.NasreddineZ. S. (2014). Montreal Cognitive Assessment Memory Index Score (MoCA-MIS) as a predictor of conversion from mild cognitive impairment to Alzheimer’s disease. Journal of American Geriatric Society, 62, 679-684. doi:10.1111/jgs.12742
18.
JulayanontP.PhillipsN.ChertkowH.NasreddineZ. S. (2013). Montreal Cognitive Assessment (MoCA): Concept and clinical review. In LarnerA. (Ed.), Cognitive screening instruments (pp. 111-152). London, England: Springer.
19.
KangT.ChenT. T. (2008). Performance of the generalized S-X2 item fit index for polytomous IRT models. Journal of Educational Measurement, 45, 391-406.
20.
KimS.-H.CohenA. S.ChoS.-J.EomH. J. (2019). Use of information criteria in the study of group differences in trace lines. Applied Psychological Measurement, 43, 95-112. doi:10.1177/0146621618772292
21.
KoskiL.XieH.FinchL. (2009). Measuring cognition in a geriatric outpatient clinic: Rasch analysis of the Montreal Cognitive Assessment. Journal of Geriatric Psychiatry and Neurology, 22, 151-160.
22.
LamB.MiddletonL. E.MasellisM.StussD. T.HarryR. D.KissA.BlackS. E. (2013). Criterion and convergent validity of the Montreal cognitive assessment with screening and standardized neuropsychological testing. Journal of American Geriatric Society, 61, 2181-2185. doi:10.1111/jgs.12541
23.
LiewT. M.FengL.GaoQ.NgT. P.YapP. (2015). Diagnostic utility of Montreal Cognitive Assessment in the fifth edition of Diagnostic and Statistical Manual of Mental Disorders: Major and mild neurocognitive disorders. Journal of American Medical Directors Association, 16, 144-148. doi10.1016/j.jamda.2014.07.021
24.
LiuT.WongG. H.LuoH.TangJ. Y.XuJ.ChoyJ. C.LumT. Y. (2017). Everyday cognitive functioning and global cognitive performance are differentially associated with physical frailty and chronological age in older Chinese men and women. Aging & Mental Health, 22, 942-947. doi:10.1080/13607863.2017.1320700
LuJ.LiD.LiF.ZhouA.WangF.ZuoX.. . . JiaJ. (2011). Montreal cognitive assessment in detecting cognitive impairment in Chinese elderly individuals: A population-based study. Journal of Geriatric Psychiatry and Neurology, 24, 184-190.
27.
Maydeu-OlivaresA.JoeH. (2006). Limited information goodness-of-fit testing in multidimensional contingency tables. Psychometrika, 71, 713. doi:10.1007/s11336-005-1295-9
28.
McDonaldR. P. (2013). Test theory: A unified treatment: Boulder, CO: NetLibrary.
29.
MurakiE. (1992). A generalized partial credit model: Application of an EM algorithm. ETS Research Report Series, 1992(1), i-30.
30.
NasreddineZ. S.PhillipsN. A.BédirianV.CharbonneauS.WhiteheadV.CollinI.. . . ChertkowH. (2005). The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society, 53, 695-699.
31.
National Institute for Health and Clinical Excellence. (2011). Dementia: A NICE-SCIE guideline on supporting people with dementia and their carers in health and social care (Revised). Leicester, England: Author.
32.
PetersenR. C.RobertsR. O.KnopmanD. S.BoeveB. F.GedaY. E.IvnikR. J.. . . JackC. R. (2009). Mild cognitive impairment: Ten years later. Archives of Neurology, 66, 1447-1455.
33.
PrinceM.BryceR.FerriC. (2011). World Alzheimer Report 2011: The benefits of early diagnosis and intervention. London, England: Alzheimer’s Disease International.
34.
R Core Team. (2017). R: A Language and environment for statistical computing [Computer software]. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from https://www.R-project.org/
35.
ReiseS. P.MooreT. M.HavilandM. G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92, 544-559.
36.
ReiseS. P.VenturaJ.KeefeR. S.BaadeL. E.GoldJ. M.GreenM. F.. . . SeidmanL. J. (2011). Bifactor and item response theory analyses of interviewer report scales of cognitive impairment in schizophrenia. Psychological Assessment, 23, 245-261. doi:10.1037/a0021501
37.
RevelleW.ZinbargR. E. (2009). Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma. Psychometrika, 74, 145. doi:10.1007/s11336-008-9102-z
38.
RoalfD. R.MooreT. M.WolkD. A.ArnoldS. E.Mechanic-HamiltonD.RickJ.. . . ChahineL. M. (2016). Defining and validating a short form Montreal Cognitive Assessment (s-MoCA) for use in neurodegenerative disease. Journal of Neurology, Neurosurgery & Psychiatry, 87, 1303-1310. doi:10.1136/jnnp-2015-312723
39.
RodriguezA.ReiseS. P.HavilandM. G. (2016). Applying bifactor statistical indices in the evaluation of psychological measures. Journal of Personality Assessment, 98, 223-237.
40.
RosseelY. (2012). Lavaan: An R package for structural equation modeling and more: Version 0.5-12 (BETA). Journal of Statistical Software, 48(2), 1-36.
41.
SamejimaF. (2016). Graded response models. In van der LindenW. J. (Ed.), Handbook of item response theory (Vol. 1, pp. 95-108). Boca Raton, FL: CRC Press.
42.
SchwarzG. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461-464.
43.
SinhaP.PoggioT. (1996). Role of learning in three-dimensional form perception. Nature, 384, 460-463. doi:10.1038/384460a0
44.
TangJ. Y.WongG. H.NgC. K.KwokD. T.LeeM. N.DaiD. L.LumT. Y. (2016). Neuropsychological profile and dementia symptom recognition in help-seekers in a community early-detection program in Hong Kong. Journal of American Geriatric Society, 64, 584-589. doi:10.1111/jgs.13938
45.
ThomasM. L. (2011). The value of item response theory in clinical assessment: A review. Assessment, 18, 291-307.
46.
TsaiC.-F.LeeW.-J.WangS.-J.ShiaB.-C.NasreddineZ.FuhJ.-L. (2012). Psychometrics of the Montreal Cognitive Assessment (MoCA)and its subscales: Validation of the Taiwanese version of the MoCA and an item response theory analysis. International Psychogeriatrics, 24, 651-658.
47.
TsoiK. K.ChanJ. Y.HiraiH. W.WongS. Y.KwokT. C. (2015). Cognitive tests to detect dementia: A systematic review and meta-analysis. JAMA Internal Medicine, 175, 1450-1458. doi:10.1001/jamainternmed.2015.2152
48.
WongA.LawL. S.LiuW.WangZ.LoE. S.LauA.. . . MokV. C. (2015). Montreal Cognitive Assessment: One cutoff never fits all. Stroke, 46, 3547-3550. doi:10.1161/STROKEAHA.115.011226
49.
XuJ. Q.ChoyJ. C. P.TangJ. Y. M.LiuT. Y.LuoH.LouV. W. Q.. . . WongG. H. Y. (2017). Spontaneously reported symptoms by informants are associated with clinical severity in dementia help-seekers. Journal of American Geriatric Society, 65, 1946-1952. doi:10.1111/jgs.14931
50.
Yang-WallentinF.JöreskogK. G.LuoH. (2010). Confirmatory factor analysis of ordinal variables with misspecified models. Structural Equation Modeling, 17, 392-423.
51.
ZhengL.TengE. L.VarmaR.MackW. J.MungasD.LuP. H.ChuiH. C. (2012). Chinese-language Montreal Cognitive Assessment for Cantonese or Mandarin speakers: Age, education, and gender effects. International Journal of Alzheimer’s Disease, 2012, 204623. doi:10.1155/2012/204623
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
