Accuracy in Scoring Vignettes Using the Mini Mental State Examination and the Short Orientation Memory Concentration Test

Abstract

Objective: To compare the accuracy in scoring of vignettes of 2 cognitive screening tests—the Mini Mental State Examination (MMSE) and the Short Orientation Memory Concentration Test (S-OMC). Design: Within group comparison. Setting: Academic institution. Participants: A total of 74 student nurses. Method: Participants received instruction on the scoring of each test and were then asked to score 6 videotaped clinical vignettes. Main Outcome Measures: Scores allocated by participants on the vignettes for the MMSE and S-OMC. Results: The proportion of participants assigning the correct total score to the vignettes was low (MMSE 53.2%, S-OMC 66.7%), and the difference in accuracy between tests was significant (P = .005). The total scores that were assigned resulted in misclassification across all 4 caseness cutoffs within each test (MMSE 0.9%-5.9%; S-OMC 1.4%-2.7%). The differences in correct caseness placement between the 2 tests were significant only for the moderate impairment cutoff, where the S-OMC achieved greater correct caseness placement. Conclusions: The high proportion of scoring mistakes overall has implications for the use of both the tests in routine clinical practice but in particular for the MMSE. An increased awareness is needed in clinical settings and among tests developers, of the likelihood of, and implications arising from, variations in scoring accuracy. In the current study, the S-OMC was scored more accurately than the MMSE.

Keywords

scoring interrater Mini Mental State Examination MMSE 6 Item Cognitive Impairment Test 6-CIT Short Orientation Memory Concentration Test Short Blessed Test

Get full access to this article

View all access options for this article.

References

National Institute for Health and Clinical Excellence. Dementia: supporting people with dementia and their carers in health and social care. NICE Guideline 2006; 42. www.nice.org.uk . Accessed April 22 2009.

Gifford DR , Cummings JL Evaluating dementia screening tests: methodologic standards to rate their performance. Am Acad Neurol. 1999;52(2):224-227.

Boustani M. , Peterson B. , Hanson L. , et al. Screening for dementia in primary care: a summary of the evidence for the U.S. Preventive Services Task Force. Ann Intern Med . 2003;138(11):927-937.

Costa PT , Williams TF , Somerfield M. , et al. Recognition and initial assessment of Alzheimer’s disease and related dementias. Clinical Practice Guideline No. 19. Rockville, MD: U.S. Department of Health and Human Services, Public Health Service, Agency for Health Care Policy and Research 1996: Publication no.97-0702.

Folstein MF , Folstein SE , McHugh PR ‘Mini-Mental State.’ A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(13):189-198.

Katzman R. , Brown T. , Fuld P. , et al. Validation of a short orientation-memory-concentration test of cognitive impairment. Am J Psychiatry. 1983;140(6):734-739.

Fillenbaum GG , Heyman A. , Wilkinson WE , et al. Comparison of two screening tests in Alzheimer’s disease: The correlation and reliability of the mini mental state examination and the modified blessed test. Arch Neurol. 1987;44(9):924-927.

Petersen RC , Stevens JC , Ganguli M. , et al. Practice parameter: early detection of dementia: mild cognitive impairment (an evidence-based review). Neurology. 2001;56(9):1133-1142.

Molloy DW , Alemayehu E. , Roberts R. Reliability of a standardized Mini Mental State Examination compared with the traditional Mini Mental State Examination. Am J Psychiatry . 1991; 148(1):102-105.

10.

Lorentz WJ , Scanlan JM , Borson S. Brief screening tests for dementia. Can J Psychiatry. 2002;47(8):723-733.

11.

O’Connor DW , Pollitt PA , Hyde JB , et al. The reliability and validity of the mini-mental state in a British community sample. J Psychiatr Res. 1989;23(1):87-96.

12.

Bowie P. , Branton T. , Holmes J. Should the mini mental state examination be used to monitor dementia treatments ? Lancet. 1999;354:1527-1528.

13.

Davey RJ , Jamieson S. The validity of using the mini mental state examination in NICE dementia guidelines . J Neurol Neurosurg Psychiatry. 2004;75(2):343-344.

14.

Fabrigoule C. , Lechevallier N. , Crasborn L. , et al. Inter-rater reliability of scales and tests used to measure mild cognitive impairment by general practitioners and psychologists. Curr Medic Res Opin. 2003 ;19(7):603-608.

15.

Lenth RV Java Applets for Power and Sample Size. Test comparing two proportions, www.stat.uiowa.edu/∼rlenth/Power . Accessed July 23, 2006.

16.

Cullen B. , O’Neill B. , Evans JJ , et al. A review of screening tests for cognitive impairment. J Neurol Neurosurg Psychiatry. 2007; 78(8):790-799.

17.

McDowell I. Measuring Health: A Guide to Rating Scales & Questionnaires. New York, NY: Oxford University Press; 2006: 394-469.

18.

Blessed G. , Tomlinson BE , Roth M. The association between quantitative measures of dementia and of senile change in the cerebral grey matter of elderly subjects. Br J Psychiatry . 1968; 114(512):797-811.

19.

Stuss DT , Meiran N. , Guzman DA , et al. Do long tests yield a more accurate diagnosis of dementia than short tests? A comparison of 5 neuropsychological tests. Arch Neurol. 1996;53(10): 1033-1039.

20.

Davous P. , Lamour Y. , Debrand E. , et al. A comparative evaluation of the short orientation memory concentration test of cognitive impairment. J Neurol Neurosurg Psychiatry . 1987;50(10): 1312-1317.

21.

Brooke P. , Bullock R. Validation of a 6 item cognitive impairment test with a view to primary care usage. Int J Geriatr Psychiatry. 1999;14(11):936-940.

22.

Reisberg B. , Ferris SH , DeLeon MJ , et al. The Global Deterioration Scale for assessment of primary degenerative dementia. Am J Psychiatry . 1982;139(9):1136-1139.

23.

Brooke P. The 6 item cognitive impairment test (6CIT). In: VandeCreek L , Jackson TL , eds. Innovations in Clinical Practice: A Source Book 19. Sarasota, FL: Professional Resource Press; 2001:299-302.

24.

National Institute for Health and Clinical Excellence. Donepezil, rivastigmine, galantamine (review) and memantine for the treatment of Alzheimer’s disease (amended). NICE technology appraisal guidance 111 2006. www.nice.org.uk. Accessed September 28 2009, August 2009.

25.

Demakis GJ , Mercury MG , Sweet JJ Screening for cognitive impairments in primary care settings. In: Maruish ME , ed. Handbook of Psychological Assessment in Primary Care Settings. Mahwah, NJ: Lawrence Erlbaum; 2000:555-582.