A Comparison of Test-Retest Reliability and Practice Effects of Short Portable Mental State Questionnaire and Montreal Cognitive Assessment in Patients with Stroke
Restricted accessResearch articleFirst published online January, 2025
A Comparison of Test-Retest Reliability and Practice Effects of Short Portable Mental State Questionnaire and Montreal Cognitive Assessment in Patients with Stroke
To compare the test-retest reliabilities and minimal detectable change (MDC) of the Short Portable Mental State Questionnaire (SPMSQ) and the Montreal Cognitive Assessment (MoCA) in patients with stroke.
Methods
63 patients were recruited from 1 medical center. The SPMSQ and MoCA were administered twice, 2 weeks apart.
Results
Both measures showed high intraclass correlation coefficients (SPMSQ: 0.87; MoCA: 0.89) and acceptable MDC%s (SPMSQ: 14.8%; MoCA: 19.6%). A small correlation (r = 0.30) was found between the absolute difference and average in each pair of assessments in the SPMSQ, which was close to the criterion of heteroscedasticity. A small practice effect was observed in the MoCA (Cohen’s d = 0.30).
Conclusion
The SPMSQ demonstrated smaller random measurement error and an absence of practice effect. When comparing the psychometric properties of the SPMSQ and MoCA as outcome measures for assessing cognitive function in patients with stroke, the SPMSQ appears to be a more suitable choice than the MoCA.
SunJHTanLYuJT. Post-stroke cognitive impairment: epidemiology, mechanisms and management. Ann Transl Med. 2014;2(8):80. doi:10.3978/j.issn.2305-5839.2014.08.05.
2.
de HaanEHNysGMVan ZandvoortMJ. Cognitive function following stroke and vascular cognitive impairment. Curr Opin Neurol. 2006;19(6):559-564. doi:10.1097/01.wco.0000247612.21235.d9.
3.
NysGMvan ZandvoortMJde KortPL, et al.The prognostic value of domain-specific cognitive abilities in acute first-ever stroke. Neurology. 2005;64(5):821-827. doi:10.1212/01.WNL.0000152984.28420.5A.
4.
PfeifferE. A short portable mental status questionnaire for the assessment of organic brain deficit in elderly patients. J Am Geriatr Soc. 1975;23(10):433-441. doi:10.1111/j.1532-5415.1975.tb00927.x.
5.
NasreddineZSPhillipsNABedirianV, et al.The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005;53(4):695-699. doi:10.1111/j.1532-5415.2005.53221.x.
6.
ChitiGPantoniL. Use of Montreal cognitive assessment in patients with stroke. Stroke. 2014;45(10):3135-3140. doi:10.1161/STROKEAHA.114.004590.
7.
SchweizerTAAl-KhindiTMacdonaldRL. Mini-Mental State Examination versus Montreal Cognitive Assessment: rapid assessment tools for cognitive and functional outcome after aneurysmal subarachnoid hemorrhage. J Neurol Sci. 2012;316(1-2):137-140. doi:10.1016/j.jns.2012.01.003.
8.
PendleburySTMarizJBullLMehtaZRothwellPM. MoCA, ACE-R, and MMSE versus the national institute of neurological disorders and stroke-Canadian stroke network vascular cognitive impairment harmonization standards neuropsychological battery after TIA and stroke. Stroke. 2012;43(2):464-469. doi:10.1161/STROKEAHA.111.633586.
9.
FariaALPinhoMSBermudezIBS. A comparison of two personalization and adaptive cognitive rehabilitation approaches: a randomized controlled trial with chronic stroke patients. J NeuroEng Rehabil. 2020;17(1):78. doi:10.1186/s12984-020-00691-5.
10.
TaravatiSCapaciKUzumcugilHTanigorG. Evaluation of an upper limb robotic rehabilitation program on motor functions, quality of life, cognition, and emotional status in patients with stroke: a randomized controlled study. Neurol Sci. 2022;43(2):1177-1188. doi:10.1007/s10072-021-05431-8.
11.
Markle-ReidMOrridgeCWeirR, et al.Interprofessional stroke rehabilitation for stroke survivors using home care. Can J Neurol Sci. 2011;38(2):317-334. doi:10.1017/s0317167100011537.
12.
YinMLiuYZhangL, et al.Effects of rTMS treatment on cognitive impairment and resting-state brain activity in stroke patients: a randomized clinical trial. Front Neural Circuits. 2020;14:563777. doi:10.3389/fncir.2020.563777.
13.
CummingTBBrodtmannADarbyDBernhardtJ. Cutting a long story short: reaction times in acute stroke are associated with longer term cognitive outcomes. J Neurol Sci. 2012;322(1-2):102-106. doi:10.1016/j.jns.2012.07.004.
14.
MarzoliniSOhPMcIlroyWBrooksD. The effects of an aerobic and resistance exercise training program on cognition following stroke. Neurorehabil Neural Repair. 2013;27(5):392-402. doi:10.1177/1545968312465192.
MokkinkLBTerweeCBPatrickDL, et al.The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010;63(7):737-745. doi:10.1016/j.jclinepi.2010.02.006.
17.
CooleySAHeapsJMBolzeniusJD, et al.Longitudinal change in performance on the Montreal cognitive assessment in older adults. Clin Neuropsychol. 2015;29(6):824-835. doi:10.1080/13854046.2015.1087596.
18.
BruijnenCDijkstraBAGWalvoortSJW, et al.Psychometric properties of the Montreal Cognitive Assessment (MoCA) in healthy participants aged 18-70. Int J Psychiatry Clin Pract. 2020;24(3):293-300. doi:10.1080/13651501.2020.1746348.
19.
LeeYCLinYTChiuEC. A comparison of test-retest reliability of four cognitive screening tools in people with dementia. Disabil Rehabil. 2022;44(15):4090-4095. doi:10.1080/09638288.2021.1891466.
20.
WuCYHungSJLinKCChenKHChenPTsayPK. Responsiveness, minimal clinically important difference, and validity of the MoCA in stroke rehabilitation. Occup Ther Int. 2019;2019:2517658. doi:10.1155/2019/2517658.
21.
LauHYLinYHLinKC, et al.Reliability of the Montreal cognitive assessment in people with stroke. Int J Rehabil Res. 2024;47(1):46-51. doi:10.1097/MRR.0000000000000612.
22.
BartelsCWegrzynMWiedlAAckermannVEhrenreichH. Practice effects in healthy adults: a longitudinal study on frequent repetitive cognitive testing. BMC Neurosci. 2010;11:118. doi:10.1186/1471-2202-11-118.
23.
CalamiaMMarkonKTranelD. Scoring higher the second time around: meta-analyses of practice effects in neuropsychological assessment. Clin Neuropsychol. 2012;26(4):543-570. doi:10.1080/13854046.2012.680913.
24.
AaronsonNAlonsoJBurnamA, et al.Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res. 2002;11(3):193-205. doi:10.1023/a:1015291021312.
25.
BujangMABaharumN. A simplified guide to determination of sample size requirements for estimating the value of intraclass correlation coefficient: a review. Archives of orofacial science. 2017;12(1):1-11.
26.
MokkinkLBPrinsenCPatrickDL, et al.COSMIN Study Design checklist for Patient-reported outcome measurement instruments. Amsterdam, The Netherlands. 2019;2019:1-32.
27.
KasnerSEChalelaJALucianoJM, et al.Reliability and validity of estimating the NIH stroke scale score from medical records. Stroke. 1999;30(8):1534-1537. doi:10.1161/01.str.30.8.1534.
28.
DeweyHMDonnanGAFreemanEJ, et al.Interrater reliability of the National Institutes of Health Stroke Scale: rating by neurologists and nurses in a community-based stroke incidence study. Cerebrovasc Dis. 1999;9(6):323-327. doi:10.1159/000016006.
29.
ZandiehAKahakiZZSadeghianH, et al.The underlying factor structure of National Institutes of Health Stroke scale: an exploratory factor analysis. Int J Neurosci. 2012;122(3):140-144. doi:10.3109/00207454.2011.633721.
30.
KooTKLiMY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155-163. doi:10.1016/j.jcm.2016.02.012.
31.
BlandJMAltmanDG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307-310. doi:DOI 10.1016/s0140-6736(86)90837-8
32.
ChhapolaVKanwalSKBrarR. Reporting standards for Bland-Altman agreement analysis in laboratory research: a cross-sectional survey of current practice. Ann Clin Biochem. 2015;52(Pt 3):382-386. doi:10.1177/0004563214553438.
33.
BlandJMAltmanDG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8(2):135-160. doi:10.1177/096228029900800204.
34.
CohenJ. A power primer. Psychol Bull. 1992;112(1):155-159. doi:10.1037//0033-2909.112.1.155.
35.
SchuckPZwingmannC. The ‘smallest real difference’ as a measure of sensitivity to change: a critical analysis. Int J Rehabil Res. 2003;26(2):85-91. doi:10.1097/00004356-200306000-00002.
36.
SchreudersTARoebroeckMEGoumansJvan NieuwenhuijzenJFStijnenTHStamHJ. Measurement error in grip and pinch force measurements in patients with hand injuries. Phys Ther. 2003;83(9):806-815.
37.
SmidtNvan der WindtDAAssendelftWJ, et al.Interobserver reproducibility of the assessment of severity of complaints, grip strength, and pressure pain threshold in patients with lateral epicondylitis. Arch Phys Med Rehabil. 2002;83(8):1145-1150. doi:10.1053/apmr.2002.33728.
38.
CohenJ. Statistical Power Analysis for the Behavioral Sciences. 2nd ed.New Jersey: Lawrence Erlbaum Associates; 1988.
39.
YangCMWangYCLeeCHChenMHHsiehCL. A comparison of test-retest reliability and random measurement error of the Barthel Index and modified Barthel Index in patients with chronic stroke. Disabil Rehabil. 2022;44(10):2099-2103. doi:10.1080/09638288.2020.1814429.
40.
CostaASFimmBFriesenP, et al.Alternate-form reliability of the Montreal cognitive assessment screening test in a clinical setting. Dement Geriatr Cogn Disord. 2012;33(6):379-384. doi:10.1159/000340006.
