Examining the construct validity of the Cognitive Competency Test for occupational therapy practice / Évaluer la validité de construit du Cognitive Competency Test en fonction de la pratique de l’ergothérapie

Abstract

Background.

Enhancing occupational therapy practice requires critical examination of assessment tools and the conclusions being drawn from their use. When working with cognitively impaired older individuals, judgments about occupational competence are often informed by an assessment of cognitive competence.

Purpose.

The Cognitive Competency Test (CCT) is a frequently used measure in Canada to inform predictions of occupational competence. However, there is an absence of published evidence that addresses its validity.

Methods.

To appraise validity of the CCT, a retrospective chart review (n = 107) of CCT reports for inpatient and outpatient clients with cognitive impairment was conducted. Data were subjected to exploratory factor analyses to examine the factor structure, and the measure was compared with commonly used clinical variables reflecting cognitive and occupational competence.

Findings.

Results suggest that the CCT measures a unitary construct and provide some support for its predictive capacity.

Implications.

CCT scores can add incremental validity to cognitive screens, such as the Mini Mental State Exam, when evaluating occupational competence.

Description.

L’amélioration de la pratique de l’ergothérapie fait appel à un examen critique des outils d’évaluation et des conclusions que l’on peut tirer face à leur usage. Lorsque l’on travaille auprès de personnes âgées ayant des troubles cognitifs, il arrive souvent que nos jugements sur la compétence occupationnelle soient éclairés par une évaluation de la compétence cognitive.

But.

Le Cognitive Competency Test (CCT) est une mesure fréquemment utilisée au Canada pour prédire la compétence occupationnelle. Toutefois, aucune étude sur la validité du CCT n’a été publiée à ce jour.

Méthodologie.

Les auteurs ont procédé à un examen rétrospectif des résultats au CCT de patients hospitalisés et non hospitalisés ayant des troubles cognitifs (n = 107), en vue d’évaluer la validité du CCT. Des analyses factorielles exploratoires ont été effectuées sur les données afin d’examiner la structure factorielle de la mesure; la mesure a ensuite été comparée aux variables cliniques utilisées couramment pour décrire la compétence cognitive et occupationnelle.

Résultats.

Les résultats suggèrent que le CCT mesure un construit unitaire et ils fournissent également certaines preuves de sa capacité prédictive.

Conséquences.

Lors de l’évaluation de la compétence occupationnelle, les scores du CCT peuvent donner une validité ajoutée aux tests de dépistage cognitifs comme le Mini Mental State Exam.

Keywords

Consequential validity Incremental validity Measurement Occupational competence Validity compétence occupationnelle mesure validité validité ajoutée validité de conséquence

Get full access to this article

View all access options for this article.

References

Allaire

Willis

(2006). Competence in everyday activities as a predictor of cognitive risk and mortality. Aging, Neuropsychology and Cognition, 13, 207–224. doi:10.1037/0882-7974.14.4.627

Aronson

Barr

Kyle

O’Keeffe

(2002, May). The use of cognitive assessments in occupational therapy for dementia. Paper presented at the Ontario Gerontological Association Annual Meeting, Toronto, Ontario, Canada.

Baum

Katz

(2010). Occupational therapy approach to assessing the relationship between cognition and function. In Marcotte

Grant

(Eds.), Neuropsychology of everyday functioning (pp. 62–90). New York, NY: Guilford Press.

Christensen

Mateer

(2005). Neuropsychological deficits, syndromes, and cognitive competency in schizophrenia. Cognitive Neuropsychology, 10, 361–378.

Cooney

L. M.

Kennedy

G. J.

Hawkins

K. A.

Hurme

S. B.

(2004). Who can stay at home? Assessing the capacity to choose to live in the community. Archives of Internal Medicine, 164, 357–360. doi:10.1001/archinte.164.4.357

Coster

(2008). Embracing ambiguity: Facing the challenge of measurement. American Journal of Occupational Therapy, 62, 743–751. doi:10.5014/ajot.62.6.743

De Lepeleire

Aertgeerts

Umbach

Pattyn

Tamsin

Nestor

Krekelbergh

(2004). The diagnostic value of IADL evaluation in the detection of dementia in general practice. Aging Mental Health, 8(1), 52–57. doi:10.1080/13607860310001613338

Douglas

Liu

Warren

Hopper

(2007). Cognitive assessments for older adults: Which ones are used by Canadian occupational therapists and why. Canadian Journal of Occupational Therapy, 74, 370–371. doi:10.2182/cjot.07.010

Farina

Fioravanti

Pignatti

Alberoni

Mantovani

Manzoni

G.,

… Nemni

(2010). Functional living skills assessment: A standardized measure of high-order activities of daily living in patients with dementia. European Journal of Physical and Rehabilitation Medicine, 46(1), 73–80.

10.

Folstein

(1975). Mini-Mental State: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12(3), 189–198. doi:10.1016/0022-3956(75)90026-6

11.

Haynes

S. N.

Lench

H. C.

(2003). Incremental validity of new clinical assessment measures. Psychological Assessment, 15, 456–466. doi:10.1037/1040-3590.15.4.456

12.

Hulley

S. B.

Cummings

S. R.

Browner

W. S.

Grady

Hearst

Newman

T. B.

(2007). Designing clinical research (3rd ed.). Philadelphia, PA: Lippincott, Williams, & Wilkins.

13.

Inman

Kulis

(1993, August). Concurrent validity and clinical utility of the Cognitive Competency Test. Paper presented at the American Psychological Association Convention, Toronto, Ontario, Canada.

14.

Iverson

G. L.

(1998). Interpretation of Mini-Mental State Examination scores in community-dwelling elderly and geriatric neuropsychology patients. International Journal of Geriatric Psychiatry, 13, 661–666. doi:10.1002/(SICI)1099-1166(1998100)13:10<661:: AID-GPS838>3.0.CO;2-0

15.

Jaeger

Berns

Davis-Conway

(2006). Neurocognitive deficits and disability in major depressive disorder. Psychiatry Research, 145(1), 39–48. doi:10.1016/j.psychres.2005.11.011

16.

Kelly

P. A.

O’Malley

K. J.

Kallen

M. A.

Ford

M. E.

(2005). Integrating validity theory with use of measurement instruments in clinical settings. Health Research and Educational Trust, 40, 1605–1619.

17.

Kielhofner

(2006). Research in occupational therapy: Methods of inquiry for enhancing practice. Philadelphia, PA: F. A. Davis.

18.

Kline

(1986). A handbook on test construction: Introduction to psychometric design. New York, NY: Methuen.

19.

Kline

(2000). Test construction: Factor analytic and item analysis methods (2nd ed.). New York, NY: Routledge.

20.

Klosses

D. N.

Alexopoulos

G. S.

(2005). IADL functions, cognitive deficits, and severity of depression: A preliminary study. American Journal of Geriatric Psychiatry, 13(3), 244–249. doi:10.1176/appi.ajgp.13.3.244

21.

Kuther

(1999). Competency to provide informed consent in older adulthood. Gerontology and Geriatrics Education, 20(1), 15–30. doi:10.1300/J021v20n01_03

22.

Law

(1987). Measurement in occupational therapy: Scientific criteria for evaluation. Canadian Journal of Occupational Therapy, 54, 133–138. doi:10.1177/000841748705400308

23.

Lehmann

S. W.

Black

B. S.

Shore

Kasper

Rabins

P. V.

(2010). Living alone with dementia: Lack of awareness adds to functional and cognitive vulnerabilities. International Psychogeriatrics, 22, 778–784. doi:10.1017/S1041610209991529

24.

Malloy

P. F.

McLaughlin

N. C.

(2010). Everyday functioning in dementia and mild cognitive impairment. In Marcotte

Grant

(Eds.), Neuropsychology of everyday functioning (pp. 264–279). New York, NY: Guilford Press.

25.

Mathuranath

P. S.

George

Cherian

P. J.

Mathew

Sarma

P. S.

(2005). Instrumental activities of daily living scale for dementia screening in elderly people. International Psychogeriatrics, 17, 461–474. doi:10.1017/S1041610205001547

26.

Messick

(1989). Validity. In Linn

R. L.

(Ed.), Educational measurement (3rd ed., pp. 13–103). New York, NY: MacMillan.

27.

Molloy

D. W.

Clarnette

(1999). Standardized Mini Mental State Examination. Troy, ON: New Grange Press.

28.

Molloy

D. W.

Darzins

Strang

(1999). Capacity to decide. Troy, ON: New Grange Press.

29.

Moore

D. J.

Moseley

Atkinson

J. H.

(2010). The influence of depression on cognition and daily functioning. In Marcotte

Grant

(Eds.), Neuropsychology and everyday functioning (pp. 419–440). New York, NY: Guilford Press.

30.

Polatajko

(1992). Naming and framing occupational therapy: A lecture dedicated to the life of Nancy B. Canadian Journal of Occupational Therapy, 59, 189–199.

31.

Polatajko

H. J.

Backman

Baptiste

Davis

Eftekhar

Harvey

A.,

… Connor-Schisler

(2007). Human occupation in context. In Townsend

E. A.

Polatajko

H. J.

, Enabling occupation II: Advancing an occupational therapy vision for health, well-being, and justice through occupation (pp. 37–61). Ottawa, ON: CAOT Publications ACE.

32.

Royall

D. R.

Lauterbach

E. C.

Kaufer

Malloy

Coburn

K. L.

Black

K. J.

(2007). The cognitive correlates of functional status: A review from the committee on research of the American Neuropsychiatric Association. Journal of Neuropsychiatry and Clinical Neurosciences, 19(3), 249–265. doi:10.1176/appi.neuropsych.19.3.249

33.

Rutman

Silberfeld

(1992). A preliminary report on the discrepancy between clinical and test evaluations of competence. Canadian Journal of Psychiatry, 37, 634–639.

34.

Sackett

Rosenberg

Gray

Haynes

Richardson

(1996). Evidence based medicine: What it is and what it isn’t. British Medical Journal, 312, 71–72. doi:10.1136/bmj.312.7023.71

35.

Sechrest

(1963). Incremental validity: A recommendation. Educational and Psychological Measurement, 23(1), 153. doi:10.1177/001316446302300113

36.

Streiner

D. L.

Norman

G. R.

(2003). Health measurement scales: A practical guide to their development and use (3rd ed.). New York, NY: Oxford University Press.

37.

Tabachnick

Fidell

(2007). Using multivariate statistics (5th ed.). Boston, MA: Pearson Education.

38.

Velicer

W. F.

Eaton

Fava

J. L.

(2000). Construct explication through factor or component analysis: A review and evaluation of alternative procedures for determining the number of factors or components. In Helme

R. D. G. E.

(Ed.), Problems and solutions in human assessment: Honoring Douglas N. Jackson at seventy (pp. 41–71). Boston, MA: Kluwer Academic.

39.

Wang

Ennis

(1986). Competency assessment in clinical populations: An introduction to the Cognitive Competency Test. In Uzzell

Gross

(Eds.), Clinical neuropsychology of intervention. Boston, MA: Martinus Nijhoff.

40.

Wang

Ennis

Copland

(1987). Cognitive Competency Test manual. Toronto, ON: Mount Sinai Hospital.

41.

Wasylenki

D. A.

(1989). Depression. In Wasylenki

D. A.

Martin

B. A.

Clark

D. M.

Lennox

E. A.

Perry

L. A.

Harrison

M. K.

(Eds.), Psychogeriatrics: A practical handbook (pp. 41–58). London, UK: Jessica Kingsley.

42.

Watkins

M. W.

(2000). MacParallel analysis [Computer software]. Retrieved from http://www.public.asu.edu/~mwwatkin/Watkins3.html

43.

Zur

Johnson

A. M.

Roy

E. A.

Laliberte Rudman

Wells

(2012). Beyond traditional notions of validity: Selecting appropriate measures for occupational therapy practice. Australian Occupational Therapy Journal, 59, 243–246.

44.

Zur

Laliberte Rudman

Johnson

A. M

Roy

E. A.

Wells

(2013). Components of cognitive competence predictive of occupational competence in persons with dementia: A Delphi study. Canadian Journal of Occupational Therapy, 80, 71–81. doi:10.1177/0008417413481574

45.

Zwick

W. L.

Velicer

W. F.

(1986). Comparison of five rules for determining the number of components to retain. Psychological Bulletin, 99, 432–442. doi:10.1037/0033-2909.99.3.432