Date Presented 04/9/21
The Pediatric Evaluation of Disability Inventory-Patient Reported Outcome (PEDI-PRO) is a self-report of functional performance designed for transition-age youth with developmental disabilities (DD) ages 14–22. We evaluated the usability of the PEDI-PRO software in clinical practice and established the reliability and accessibility of the PEDI-PRO user interface. Results indicate that the PEDI-PRO software is easy to use in clinical practice, and the accessibility features appear to support reliable reporting of daily activities, social–cognitive, and mobility function by youth with DD.
Primary Author and Speaker: Jessica Kramer
PURPOSE: Assessment developers recognize that technology can enhance the accessibility of educational tests for youth with disabilities (Beddow, 2012), but this vision has yet to be fully realized for self-reports (patient reported outcome measures (PROMs)) in occupational therapy practice. Most PROMs, which are predominately paper based, present many barriers to completion for transition-age youth with developmental disabilities (DD). Paper based PROMs can be cumbersome or impossible to modify to reduce the visual-perceptual, motor, and cognitive demands required for completion. The PEDI-PRO is a PROM designed for transition age youth with DD ages 14-22; it measures perceived performance of Daily Activities, Social/Cognitive, and Mobility functional skills. Previous research established that youth with DD interpreted the initial PEDI-PRO items in the intended manner (Kramer & Schwartz, 2017). However, additional research is needed to evaluate the usability and accessibility of these items when administered via an accessible software interface. Research Aim 1 (RQ1) Evaluate the usability of the PEDI-PRO software in clinical contexts. Research Aim 2 (RQ2) Establish the reliability and acceptability of the PEDI-PRO user interface with transition age youth with DD.
DESIGN: RQ1: We used survey methodology to evaluate clinical usability. We recruited clinicians across the US through professional networks and a database of former research participants. A total of 14 clinicians (OT = 12, PT = 2) administered the PEDI-PRO in diverse practice contexts (School-based = 9, outpatient = 4, other setting = 1) to 39 youth with DD ages 14-22. RQ2: Test-retest design with a convenience sample of youth with DD recruited through schools and agencies in a large metropolitan area, and database of former research participants. 54 youth completed the PEDI-PRO: M age = 19.7 yrs, sd = 1.7 yrs; 60% male; 64% Caucasian; 64% intellectual disability.
METHOD: RQ1: Clinicians administered the PEDI-PRO to three youth, and then completed the System Usability Survey (SUS) (Bangor et al., 2008). SUS scores range from 0-100, with higher scores indicating greater usability; the accepted industry standard for usability systems is a mean score of >68. RQ2: Research staff administered the PEDI-PRO software two times, 1-2 weeks apart. Items were counterbalanced across testing occasions to avoid order effects. Youth also completed a questionnaire about the features of the PEDI-PRO. We used a Rasch rating scale model to obtain scores from the original Likert response scale, then used ICC consistency model (3,1) (Trevethan, 2017) to investigate the reliability of scores across testing occasions; ICCs > .75 indicate fair reliability.
RESULTS: RQ1: SUS rating M = 84 (sd = 11.68), exceeding industry standards. RQ2: Results reported by PEDI-PRO domain. Daily Activities: ICC = .83. Social/Cognitive: ICC = .85. Mobility: ICC = .81. Almost all youth reported they ‘liked’ the accessibility features, including interface images, button sounds, item read-aloud audio, and rating category choices with images: Range = 84%- 98%; M = 88.8% (sd = 5.1%).
CONCLUSIONS: Clinicians and youth with DD found the interface easy to use and accessible. Our reliability results provide initial evidence that the PEDI-PRO user interface produces relatively stable scores during a time period in which change in functional performance is not expected to occur.
IMPACT: PROMs that incorporate technology, such as the PEDI-PRO, can provide access to PROMs for youth with DD and a range of functional needs. Access to PROMs can help address the current under representation of this population in the evaluation of rehabilitation and healthcare outcomes and support client-centered practice.
References
Beddow, P. A. (2012). Accessibility theory for enhancing the validity of test results for students with special needs. International Journal of Disability, Development and Education, 59(1), 97-111. https://doi.org/10.1080/1034912X.2012.654966
Kramer, J. & Schwartz, A. (2017). Refining the Pediatric Evaluation of Disability Inventory–Patient-Reported Outcome (PEDI-PRO) item candidates: Interpretation of a self-reported outcome measure of functional performance by young people with neurodevelopmental disabilities. Developmental Medicine and Child Neurology, 59(10), 1083-1088. https://doi.org/10.1111/dmcn.13482.
Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the system usability scale. Intl. Journal of Human–Computer Interaction, 24(6), 574-594. https://doi.org/10.1080/10447310802205776
Trevethan, R. (2017). Intraclass correlation coefficients: clearing the air, extending some cautions, and making some requests. Health Services and Outcomes Research Methodology, 17(2), 127-143. https://doi.org/10.1007/s10742-016-0156-6
