Date Presented 04/13/21
This presentation describes the preliminary psychometric properties of the TeleWrite, a handwriting assessment tool for children in first through third grade administered via telehealth. A series of pilot studies have been completed to determine initial interrater reliability, content validity, and clinical utility using classical test theory. Conversation about clinical use and research plans for construct validation will be facilitated.
Primary Author and Speaker: Julia M. Guzman
PURPOSE: This presentation will introduce the TeleWrite, an occupation-based handwriting assessment specifically designed to be administered through a remote web-based platform, and discuss studies determining its preliminary psychometric properties.
DESIGN: We used classical test theory in a series of pilot studies to determine initial interrater reliability, content validity, and clinical utility using classical test theory. Intraclass correlation was computed for inter-rater reliability. Content validity index with modified kappa statistic were calculated to determine content validity. A mixed-methods approach was used to determine whether the TeleWrite has potential clinical usefulness.
METHODS: To measure interrater reliability, we used a convenience sample of eight children in grades 1-3 and 9 raters. We also used a mixed-methods approach by collecting quantitative descriptive demographical data and qualitative data through survey results of 52 occupational therapists’ perceptions of the clinical usefulness of the instrument. The participants represented 12 countries and the United States from 22 states. We assessed the content validity of the TeleWrite through a review by a panel of 9 experts. All content experts had a doctoral degree, publications, and an average of 20.7 years of experience in pediatric OT or academia. This study analyzed two content validity indices: item level content validity index (I-CVI) and scale level validity (S-CVI).
RESULTS: We analyzed the intraclass correlation (ICC) estimate of inter-rater reliability resulting in total scores of .928 (excellent) at a 95% confidence interval and the Cronbach's Alpha (α) measure for internal consistency was .953 (excellent). Data from the clinician’s responses indicated that (61%; n = 33) are quite or extremely likely to use the TeleWrite in their practice. Regarding the perceived accuracy of the TeleWrite in assessing handwriting skills, most of the participants (64%; n = 34) indicated that TeleWrite was quite/extremely accurate in measuring handwriting. The content experts indicated that all 10 TeleWrite subscales rated are essential items. To achieve an excellent content validity rating, the I-CVI should be .78 or higher and the S-CVI should be 0.90 or higher (Polit, et al., 2007), the TeleWrite tool surpassed these thresholds at I-CVI of 0.88 and S-CVI 0.94 respectively.
CONCLUSION: The Ecology of Human Performance (EHP) framework postulates that occupational performance is a product of the interaction among the person, the context, the task, and performance (Dean, Wallisch, & Dunn, 2019). The TeleWrite is an ipsative, occupation-based assessment that uses an ecological framework to assess the child, environment, and the transaction between the two. Assessment of children's handwriting should include evaluation of fine motor, perceptual skills, and motor planning (Fancher et al, 2018). The TeleWrite includes all these elements as a handwriting assessment designed to be administered through a telehealth platform. Telehealth has significant benefits for OT in terms of expanding our scope of practice, providing client centered services to those living in remote areas or for those without access to evaluation and rehabilitation. Telehealth is needed urgently nationwide given the COVID-19 crisis and this medium of delivering services is more widely used; however, it is especially urgent in rural and less populated areas where health care access is more limited (AOTA, 2020).There is a lack of evaluation tools validated for telehealth administration and there is no handwriting assessment specifically designed for telehealth use. The TeleWrite assessment was developed to bridge this gap and to expand the use of telehealth-based OT assessment in pediatrics.
References
American Occupational Therapy Association (2020). Information pertaining to occupational therapy in the era of Coronavirus (COVID-19). https://www.aota.org/Practice/Health-Wellness/COVID19.aspx.
Dean, E. Wallisch, A., & Dunn, W. (2019). Adaptation as a transaction with the environment. Perspectives from the Ecology of Human Performance model. In L.C. Grajo and A. Boisselle (Eds). Adaptation through occupation: Multidimensional perspectives. Thorofare, NJ: SLACK, Inc.
Fancher, A., Priestley-Hopkins, D. A. & Jeffries, L. M. (2018). Handwriting acquisition and intervention: A systematic review. Journal of Occupational Therapy, Schools & Early Intervention, 11(4), 454-473. https://doi.org/10.1080/19411243.2018.1534634.
Polit, D.F., Beck, C.T. & Owen, S.V. (2007) Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health, 30, 459-467. http://dx.doi.org/10.1002/nur.20199.
