Date Presented 04/01/2022
This study explores the impact of using keyforms on MSOT learners’ ability to analyze task items according to level of difficulty, select the just-right challenge interventions, and choose appropriate short-term and long-term goals. Outcomes revealed that keyform use significantly improved identification of level of difficulty and just-right challenges but not goal setting. Keyform use may help answer the call to action to personalize measurement and demonstrate the distinct value of OT services.
Primary Author and Speaker: Brad Egan
Additional Authors and Speakers: Cynthia Sears
PURPOSE: This study’s purpose was to identify changes in MSOT learners’ clinical reasoning after participating in a keyform map learning activity. This study came about in response to the recent call to action to personalize measurement in practice (Velozo, 2021). Keyform maps, based on Rasch analysis, are innovative tools that support clinicians in understanding the client’s performance on specific test items rather than generally understanding ability or disability levels from a total assessment score (Choi, 2014). Moreover, the keyform map enables clinicians to easily identify how each item on an assessment fits within an overall task-difficulty hierarchy so that interventions can be more therapeutically effective (Woodbury et al., 2016). The following research questions were used: 1) Does an educational learning activity on the use of keyforms help students better identify which items are more easy or more difficult than other items on the same assessment? 2) Does the use of a keyform support learners in more accurately determining the just-right-challenge interventions for a client? 3) Does the use of a keyform support learners in more accurately determining short-term and long-term goals for a client?
DESIGN: This study used a pre- and post-test design to explain the relationship between students clinical reasoning when not using and using a keyform map to determine item difficulty, just-right-challenge interventions, and goals based on the client’s ability level. Participants (n = 80) were learners enrolled in an entry-level MSOT program. This study was approved by the University’s IRB.
METHOD: Participants were given instruction on keyform maps and personalized measurement and then asked to review a clinical case designed by the research team. The case included the Berg Balance scores for a client who was experiencing occupational performance issues related to poor balance. After reviewing the case, participants complete a 10-question (non-graded) multiple choice quiz addressing task difficulty, just-right-challenge activities, and STGs and LTGs. Immediately following, participants were then given the Berg Balance keyform and asked to complete the same quiz. The research team disabled features in the LMS so that participants could not see their scores or which quiz items they answered correctly or incorrectly. SPSS software was used to analyze the pre and post data.
RESULTS: Paired t-test analysis indicated an extremely statistically significant positive overall change in pre- and post-test scores (p<0.0001). The quiz was subdivided into 3 parts to better determine the impact of keyform use on determining item difficulty, just-right challenge, and STGs and LTGs. Outcomes were statistically significant for analyzing task difficulty (p<.001) and selecting the just-right challenge (p<.01) but not for setting goals (p = 0.69).
CONCLUSION: The objective of this study was to analyze the impact of a keyform map on participants’ ability to determine the level of difficulty for items on a commonly used assessment, just-right challenge interventions, and STGs and LTGs. The use of a keyform demonstrated statistically significant improvements on students’ ability to determine the level of difficulty of test items and just-right challenge interventions but not with setting goals. Moreover, the results suggest that keyforms provided participants with useful measurement outputs that translated to dramatic increases in the accuracy and personalization of occupational therapy services.
IMPACT STATEMENT: Supporting pre-service clinicians to develop habits around keyform use may be one way to answer the call to action to personalize measurement and demonstrate the distinct value of OT services.
References
Velozo, C.A. (2021, April 6). 2021 Eleanor Clarke Slagle Lecture – Using Measurement to Highlight OT’s Distinct Value. AOTA Inspire 2021, virtual conference.
Choi, B. (2014). Keyform analysis of Rasch measurement accessible to clinicians in rehabilitation clinics. Physical Therapy Korea, 21(2), 74-81. http://dx.doi.org/10.12674/ptk.2014.21.2.074
Woodbury, M., Anderson, K., Finetto, C., Fortune, A., Dellenbach, B., Grattan, E., & Hutchison, S. (2016). Archives of Physical Medicine and Rehabilitation, 97(11), 1863–1871. https://doi.org/10.1016/j.apmr.2016.03.022
