Date Presented Accepted for AOTA INSPIRE 2021 but unable to be presented due to online event limitations.
The goal of this poster is to provide information and foster conversation about the development, clinical use, and content validity of the Occupational Therapy Anticipatory Awareness Test (OTAAT). The OTAAT is a newly developed, performance-based assessment tool that addresses a gap in the assessment of anticipatory awareness for individuals with neurological conditions. Discussion about participation in future research and validation processes will be facilitated.
Primary Author and Speaker: Danielle Mahoney
Additional Authors and Speakers: Lenin Grajo, Glen Gillen
PURPOSE: The American Occupational Therapy Association (AOTA; 2019) defines functional cognition as the ability to use and integrate thinking and processing skills to accomplish everyday activities in clinical and community living settings. Self-awareness is a component of functional cognition that is required to maintain safe and optimal participation in all daily life activities in a variety of environmental contexts. Self-awareness of functional deficits is defined as the cognitive ability to understand the functional impact of injury-related deficits (Kersey et al., 2019). Anticipatory awareness, the most advanced level of awareness, is the ability to anticipate the level of difficulties that might be experienced in daily life situations (Chesnel et al., 2018). Anticipatory awareness is a critical skill in occupational performance that is often impacted by a neurological condition. Decreased anticipatory awareness can translate into poor judgment and safety and inability to set realistic goals, and therefore, must be taken into consideration when discharge planning occurs. The development of assessments that measure functional skills is a priority in the profession (AOTA and American Occupational Therapy Foundation, 2011). A recently conducted scoping review of the literature has revealed a paucity of available assessment tools measuring anticipatory awareness (Mahoney et al., 2019). The Occupational Therapy Anticipatory Awareness Test (OTAAT) is a newly developed, performance-based assessment tool that can potentially addresses a gap in the assessment of anticipatory awareness for individuals with neurological conditions. The purpose of this study was to determine the initial content validity of the OTAAT.
DESIGN: In order to assess content validity, researchers utilized quantitative methods following Classical Test Theory (CTT). Content validity is defined as the comprehensiveness of an assessment and its inclusion of items that fully represent the attribute being measure (Law, 1997). The attribute being measured by the OTAAT is anticipatory awareness.
METHODS: An expert panel of five occupational therapists were purposefully recruited, as a gold standard in order to establish content validity (Gilbert & Prion, 2016). Using Lynn’s guidelines (1998), a minimum of 5 reviewers will minimize chance agreement and the total number should not exceed ten. The experts have an average combined 14 years of experience in practice (range = 8-20 years) and average of 16 publications related to adult neurological practice (range = 3-30 publications). The participating occupational therapists received the OTAAT scoring sheet, administration guide, and materials. Using an online form, the content experts rated each item on the OTAAT as either (1) essential, (2) useful but not essential, or (3) not necessary following guidelines by Lawshe (1975). Opportunities were provided for comments on how to improve the test item, as needed. Content Validity Ratio (CVR) was calculated.
RESULTS: A total of 10/14 items of the OTAAT were preserved. Eight out of fourteen items were rated with a CVR = 0.99 (1.00) indicating that all content experts believed these eight items to be essential. Two of the ten items addressing emergent awareness received a CVR = 0.6 and 0.2, meaning useful but not essential. Following discussion, review of supporting literature, and analysis of qualitative feedback, the two emergent awareness items were kept. Four items were removed (strategy implementation and awareness of strategy use subdomains) as these were deemed not essential.
CONCLUSION: This study established preliminary content validity of the OTAAT. Future research should focus on establishing construct validity and test reliability of the tool.
References
American Occupational Therapy Association. (2019). Cognition, Cognitive Rehabilitation, and Occupational Performance. American Journal of Occupational Therapy, 73, Supplement 2, 7312410010. https://doi.org/10.5014/ajot.2019.73S201
American Occupational Therapy Association and American Occupational Therapy Foundation. (2011). Occupational therapy research agenda. American Journal of Occupational Therapy, 65(Suppl.), S4–S7. https://doi.org/10.5014/ajot.2011.65S4
Kersey, J., Juengst, S. B., & Skidmore, E. (2019). Brief report-Effect of strategy training on self-awareness of deficits after stroke. American Journal of Occupational Therapy, 73(3), 7303345020. http://doi.org/10.5014/ajot.2019.031450
Mahoney, D., Gutman, S. A., & Gillen, G. (2019). A Scoping Review of Self-Awareness Instruments for Acquired Brain Injury. The Open Journal of Occupational Therapy, 7(2). https://doi.org/10.15453/2168-6408.1529
