Abstract
Despite a rapid increase in telehealth utilization, older adults disproportionately experience disparities to services. To promote telehealth accessibility among this population, there is a need for specific training to increase user perceived competency. For this sequential mixed-methods study, we designed telehealth education modules through consultation with older adults. We then evaluated their impact on older adults’ perceived telehealth competency. To solicit feedback on preliminary modules, we administered a semi-structured interview to a sample (n = 5) of older adults; then, we assessed the revised modules’ impact on perceived competency among a sample of older adults (n = 53). Participants critiqued the preliminary training materials as having limited information on telehealth privacy and advised increasing the accessibility of design. Those that completed the revised modules demonstrated significant improvements in perceived telehealth competence. Telehealth training modules may be a promising method to increase perceived telehealth competency among older adults.
Telehealth, which is the process of providing health care services through internet enabled technologies (American Occupational Therapy Association [AOTA], 2018), is an increasingly adopted practice innovation. The onset of the COVID-19 pandemic was a pivotal catalyst for this trend, wherein April 2020 (i.e., the first month following declaration of a global pandemic by the World Health Organization [WHO]) saw an 8,336% increase in utilization from the year prior (FAIR Health, 2020). Beyond the COVID-19 pandemic, telehealth continues to account for 5.4% of health care claims (FAIR Health, 2022). Key drivers for adoption of telehealth include improved health informatics technologies, health care policy and financing, consumer demand, as well as a growing body of evidence supporting outcomes (Center for Connected Health Policy [CCHP], 2014; Gajarawala & Pelkowski, 2021). However, adoption of telehealth among older adults is challenged by more barriers to access and usability than other age groups (Kalicki et al., 2021).
With the population of older adults expected to double by the year 2050 (WHO, 2022), equitable and accessible health services will be a critical need. If present trends continue, during this timeframe, approximately two thirds of older adults will be managing multiple chronic conditions (Boersma et al., 2020). However, older adults disproportionately experience disparities to health care access (American Geriatrics Society, 2020; Peckham et al., 2021). Targeting these inequities, the Institute for Healthcare Improvement first proposed the Triple Aim as a treatise to improve the U.S. health care system by targeting population health outcomes, while simultaneously lowering associated costs, and improving the patient experience (Berwick et al., 2008; Dirnberger & Waisbren, 2020). Bodenheimer and Sinsky (2014) have proposed an additional aim of enhancing health care provider work life to reduce incidence of professional burnout. Now recognized as the Quadruple aim, it is widely embraced as a foregoing framework for evidence-based health care improvement (Nundy et al., 2022). Telehealth is one means to help achieve the Quadruple Aim, and adoption of telehealth by allied health services, particularly occupational therapy, will be critical to achieve this goal (Cason, 2015).
Growing evidence supports the role of occupational therapy and other allied health services in addressing the Quadruple Aim. Studies by Little, Pope, et al. (2018), Gately et al. (2020), Saragih et al. (2022), and Scott et al. (2022) have identified improved or equitable patient outcomes through the use of telehealth. Furthermore, decreased cost and travel demands, as well as increased scheduling flexibility and timeliness, result in improved client (Charters et al., 2022; Eddison et al., 2022; Goldberg et al., 2022) and provider experiences (Eddison et al., 2022; Ervin et al., 2021; Little, Wallisch, et al., 2018; Smith et al., 2022). Despite these benefits, the forecast for telehealth remains tentative for occupational therapy, clouded by an absence of long-term reimbursement policies and contextual barriers limiting adoption.
Among contextual factors, access to high-speed internet–enabled technology is a requisite for telehealth utilization. While internet usage is at its highest point in history, 7% of individuals within the United States remain offline (Perrin & Atske, 2021). Age is the primary discriminating factor among this cohort, wherein 25% of individuals above 65 years report never going online (Perrin & Atske, 2021). Furthermore, internet nonuse, respectively, accounts for 14% individuals with fewer than 12 years of education or from a household with less than US$30,000 in income (Perrin & Atske, 2021).
Once online, additional barriers that delay the adoption of telehealth among older adults include a lack of perceived competency in using requisite technologies and processes associated with telehealth visits. Edwards et al. (2014) identified confidence to be a cross-cutting determinant of telehealth adoption across demographics of health status, age, and socioeconomic factors. This is supported by Hawley et al. (2020), who found 68% of older adults sampled were interested in telehealth, yet only 42% reported confidence in using it. In addition to confidence, Vaportzis et al. (2017) found a lack of accessible training and technology-specific knowledge to be additional barriers. Despite these barriers, older adults express a willingness to adopt telehealth to access services that would facilitate independent living (Edwards et al., 2014; Qin, 2022; Vaportzis et al., 2017). Targeted and accessible training is critical for adoption among this population (Carroll et al., 2022).
Occupational therapy practitioners (OTPs) have unique professional expertise in promoting accessible participation, including with telehealth (Abbott-Gaffney et al., 2022). We feel this expertise may be leveraged to develop telehealth training for older adults. Therefore, to address this need, the purpose of this exploratory sequential mixed-methods study was to (a) describe the development of a website housing asynchronous and accessible telehealth education modules (TEMs) targeting the needs of older adults, and (b) evaluate the extent to which TEMs improve self-perceived telehealth competency for older adults. In the quantitative phase, we hypothesized that participants would report higher competency after completion of TEMs. Findings of this study will align with the PACE Framework, which seeks to advance evidence-based telehealth service delivery for occupational therapy through research (Little et al., 2021), thus enhancing participation for our future clients.
Method
The current study used an exploratory sequential mixed-methods approach (Creswell et al., 2003), in which we began with a qualitative approach to evaluate of the content and format of initial TEMs (i-TEMs) followed by a quantitative approach to evaluate the extent to which revised TEMs may influence competency in telehealth use among older adults. Within mixed-methods, decisions are made related to timing of data collection and mixing (relating the two data sets) of each approach (Creswell & Clark, 2007). For the current study, we collected qualitative data prior to implementing changes to the revised TEMs based on stakeholder feedback, and then collected the quantitative data from a larger sample of older adults. We did not mix the two data sets; instead, we revised the TEMs based on the qualitative feedback. Then, in the quantitative phase, we evaluated the extent to which the revised TEMs influenced perceived competency among older adults who completed the training.
TEM Development and Refinement
We designed i-TEMs content through a review of literature on telehealth education and training recommendations provided by the National Academic Consortium of Telehealth (Hollander et al., 2018), Papanagnou et al. (2015), and the American Telemedicine Association (ATMA, 2011). Literature on efficacious telehealth training programs for allied health providers, supplemental best practice guidelines for clinical interactions, and interprofessional communications all served as a theoretical foundation for the i-TEMs. We adapted these clinically oriented recommendations to target the learning needs of telehealth care recipients. We developed the initial modules (prior to qualitative participant review) between October 15, 2020 and December 30, 2020. We obtained approval from our university’s institutional review board, and all participants provided electronic consent via REDCap (Harris et al., 2019) to participate in the study.
Participants
For the qualitative phase of the study, we recruited a convenience sample of older adults from our university’s Center for Excellence in Aging’s list of community volunteers. To be eligible, qualitative participants needed to be 60 years or older, be community-dwelling, and have access to an internet enabled device to complete the i-TEMs. We sent email invitations to n = 8 potential participants, and n = 5 agreed to participate. Demographics of this sample included mean age 71 years (range: 60–78 years); two participants identified as Black or African American, one as White, and two preferred not to respond.
For the quantitative phase of the study, we recruited a convenience sample of older adults with one or more chronic health conditions from a list of volunteers participating with a service-learning, university-wide interprofessional education course at Rush University. To be eligible quantitative participants needed to be 60 years or older, community-dwelling, and have one or more chronic health condition. We excluded any individual who lacked consistent access to an internet-enabled device. Quantitative participants needed to complete pre- and poststudy questionnaires and all TEMs to be classified as completing all study procedures. Quantitative sample demographics and level of study procedure completion may be found in Table 1.
Participant Demographics Grouped by Study Procedures Completion.
Procedures and Measures
Qualitative
After qualitative participants viewed our i-TEMs, we used a phone-delivered semi-structured questionnaire to evaluate acceptability and solicit any recommended modifications. Specifically, we asked six open-ended questions to garner feedback on improvements to the content of each module in addition to a question about how we could improve the accessibility of the modules overall (available as Supplemental Table 1). The first author (ST) administered the questionnaire over the phone and recorded all participant responses.
Quantitative
We collected quantitative participants’ demographic information and administered the Telehealth Competency Questionnaire-Consumer (TCQ-C; Taylor & Little, 2023) before viewing and following completion of the revised TEMs. The TCQ-C is a 13-item assessment of client or patient perceptions of their telehealth knowledge, skills, and attitudes related to five domains: (a) telehealth practice fundamentals, (b) troubleshooting, (c) care planning, (d) therapeutic rapport, and (e) team-based care. The TCQ-C is scored on a 5-point Likert-type scale (1 = highly agree to 5 = highly disagree; a lower score is favorable) and demonstrates moderate, positive concurrent validity with the Telehealth Usability Questionnaire—Usefulness subscale (r = .728, p < .001) and factor analysis reveals the 13-item questionnaire items load on one factor which explains 66.66% of sample variance (Taylor & Little, 2023).
Data Analysis
Qualitative
Three authors (ST, LL, JO) read all participant feedback separately and quantified common topics (Chi, 1997). This approach allowed us to quantify the number of instances in which respondents reported that specific changes were needed. The three authors separately created a coding scheme, met to discuss and finalized a coding scheme, then went back to the data to recode the responses based on the coding scheme. Then, all authors discussed how participants’ suggestions could be synthesized, brainstormed strategies to implement changes, and agreed on specific changes to implement for the TEMs. Finally, we revised TEMs into a final form for the quantitative phase of the study.
Quantitative
We used chi-square analyses to investigate the extent to which demographic characteristics (i.e., gender, annual median household income of zip code, minority race, and ethnicity status) and prior experience (i.e., prior telehealth use) may have differed between the group that completed all study procedures versus those who did not. Quantitative participants were considered to have completed all study procedures if they completed both pre- and poststudy questionnaires and reported fully completing all five TEMs. Then, to address our primary research question, we ran nonparametric and parametric paired tests using individual item and the total mean TCQ-C scores to examine the extent to which older adults’ engagement with revised TEMs influenced their perceived self-competency to use telehealth.
Results
Qualitative
As a result of the qualitative older adults’ feedback on i-TEMs, our research team made specific changes to the content and quality of the modules as outlined below.
Accessibility
The primary concern reported by four of the five respondents was regarding the accessibility of the i-TEMs. Overall, participants suggested that the narration be slower, less complex, and asked for closed captioning as an automatic feature (vs. having to change settings to enable). Therefore, we revisited all language used throughout the modules to ensure a seventh-grade reading level (to align with average-difficulty health literacy accessibility recommendations from the U.S. Department of Health and Human Services, 2010), reduced the speed of narration, and enabled closed captioning. Furthermore, based on feedback, we introduced definitions for technical jargon words such as “Wi-Fi” and “login.”
Beyond the content of the i-TEMs, qualitative participants provided additional suggestions for the hosting website. Two individuals expressed the need for written directions to use distinct types of videoconferencing software. Therefore, we created step-by-step guides that were available through word documents, PDFs, and embedded within the training website with written and visual descriptions. One participant suggested that we add video examples of telehealth for additional learning tools, so we collaborated with a simulation center to create vignettes for each module, exemplifying learning objectives through a simulated telehealth encounter. In addition, we introduced accessibility tools within the website to support navigation.
Privacy
Two qualitative participants reported unresolved concerns related to the privacy of their information when using telehealth, and the overall privacy of the i-TEMs hosting website itself. Participants suggested i-TEMs include specific information that addressed technical methods for ensuring security of private health information (e.g., not using public networks, means of determining whether a communication is encrypted). Furthermore, there was a concern regarding our original web domain (i.e., “.training”), which was thought to potentially be malicious as it was not a more commonly used “.com” or “.org.” Therefore, we changed the revised TEMs hosting website’s domain to be “.org.” See Table 2 for the revised TEMs, learning materials, and associated learning objectives.
Revised Telehealth Education Modules’ (TEMs) Learning Materials and Objectives.
Note. CC = closed captioning enabled.
Quantitative
In the second phase of the study, we evaluated the extent to which revised TEMs impacted the perceived competency for telehealth utilization among older adults. Fifty-three (56.4%) quantitative participants completed all study procedures (i.e., both pre- and poststudy questionnaires, as well as all five TEMs) of our original sample of 94. Overall, chi-square analyses indicated that those that completed all study procedures did not significantly differ from those that did not on the basis of racial minority status (χ2(1) = 3.040, p = .081), Hispanic ethnicity (χ2(1) = 1.101, p = .294), or gender (χ2(1) = 0.741, p = .389). In addition, analyses showed no significant differences among these groups based on previous use of telehealth (χ2(1) = .002, p = .961). Median household income of zip code had a significant association with study procedures completion (χ2(2) = 7.693, p = .021), wherein lower median income was associated with lower completion and higher median income demonstrated the opposite effect. To further explore the impact of median household income of zip code, we ran an analysis of variance (ANOVA) on baseline TCQ-C mean scores and found no significant difference among groups (F(2, 93) = .280, p = .757).
Paired pre- and poststatistics were run on individual TCQ-C items and median score (nonparametric and parametric tests, respectively). Data showed that participants reported significantly improved perceived competency to use telehealth following completion of revised TEMs (see Table 3). Specifically, participants reported improvements in all but one item of the TCQ-C, as well as a significant improvement on the mean total TCQ-C score. Cohen’s d for the change in mean TCQ-C score (0.608) indicated a medium effect size, with individual items’ effect sizes ranging from 0.302 to 0.834 (i.e., small [Cohen’s d > 0.2] medium [Cohen’s d > 0.5] to large [Cohen’s d > 0.8]; Cohen, 1998; Table 3).
Telehealth Competency Questionnaire—Consumer (TCQ-C) Results.
Note. The TCQ-C utilizes a five-level Likert-type scale for all items (“1” = strongly agree, “2” = agree, “3” = neutral, “4” = disagree, “5” = strongly disagree).
Expressed as Z unless otherwise indicated.
Small effect size. **Medium effect size. ***Large effect size (Cohen, 1988).
Discussion
The purpose of this study was to describe the development and exploratory evaluation of TEMs on client-perceived telehealth competency targeting the needs of older adults. We found that consultation with this population throughout the developmental process of the TEMs website allowed for valuable insights into the unique needs and perspectives of this population. Specifically, stakeholders shared strategies that would increase the accessibility of the materials as well as ways to address privacy concerns within the modules. Accessibility of training materials and resources to promote competence of telehealth is critical to reducing disparities to health service access experienced by this population (Carroll et al., 2022) and improving client experiences as part of the Quadruple Aim (Bodenheimer & Sinsky, 2014). Novel findings from this study also showed that online training materials, when adapted to meet older adults’ needs, may be efficacious in increasing their perceived competence and confidence to use telehealth.
During the qualitative phase, consultation with the stakeholder group of older adults for i-TEMs review revealed two key areas of need. The first was a concern regarding accessibility, here defined as the presentation of information within the i-TEMs website, including format, speech, and language use. This reported concern aligns with previous findings by Vaportzis et al. (2017), Eddison et al. (2022), and Charters et al. (2022), who found that perceived complexity with technologies increased feelings of inadequacy among this population. The second was a concern about the availability of telehealth privacy and information security safeguards. Unlike the first area of concern, which was related to the presentation of information within the i-TEMs, this critique was due to insufficient content related to this specific knowledge area. While this report for our study was specific to the concerns of older adults, this finding aligns with Edwards et al. (2014), who found that concerns related to security was a cross-cutting perceived disadvantage of telehealth for all age groups.
Beyond the present study and associated TEM development, telehealth concerns expressed by older adults bear relevance to OTPs. Among health care professionals, OTPs are experts in occupational analysis and promotion of accessibility, expertise that can be leveraged to promote health service access through this medium. Furthermore, through application of these professional skills to secure telehealth access and provide subsequent services, OTPs are distinctly suited to advance population health and advance the Quadruple Aim (Abbott-Gaffney et al., 2022; Cason, 2015; Proffitt et al., 2021). The role of OTPs in addressing these concerns and providing these services is further supported by AOTA (2018) in a position paper concerning the subject, and aligns with the PACE Framework research agenda (Little et al., 2021).
Through the quantitative phase of our investigation, we were able to examine the impact of revised TEMs through both levels of study completion and change in perceived competency. A majority of this sample completed all study procedures (i.e., both pre- and postquestionnaires, and all TEMs). With the exception of median household income of zip code, no participant demographic feature was significantly associated with attrition. For demographic categories of gender, race, ethnicity, this suggests random study attrition, and that none of these participant subgroups disproportionately experienced difficulty with study procedures (i.e., completion of questionnaires and TEM trainings). As lower median household income of zip code was associated with study procedure attrition, we feel this suggests this group experienced additional challenges in completing study procedures comprising electronic survey completion and online TEMs. This barrier, additionally found by Eddison et al. (2022), should be recognized by providers adopting similar telehealth trainings.
With the majority of the quantitative sample completing all study procedures, we feel this suggests a general interest among our sample in completion of the revised TEM trainings, a finding also found among the samples of Edwards et al. (2014) and Qin (2022). We feel this cross-cutting level of interest in gaining telehealth competency demonstrates a need within this field and suggests practitioners have resources to support this need among potential clients.
Our data provided support for preliminary evaluation of TEMs toward increasing perceived competency to utilize telehealth services among older adults. Significant improvements in the mean total and all but one individual TCQ-C items indicated that TEMs may be effective at increasing perceived competency to participate with telehealth services among this population. Furthermore, the medium magnitude of change for the TCQ-C mean suggests that the TEMs are more likely to have generated a meaningful change in perceived competency among our sample.
Our process for development of TEMs and the subsequent investigation into larger-scale efficacy must be interpreted with recognition of study limitations. One limitation is a sampling bias, as our recruitment procedures included acquiring two convenience samples from a pool of volunteers for a program internal to our home institution (i.e., Rush University). Participation with previous programming among these cohorts may have biased their response to TEMs, as well as could a geographical bias (as many center volunteers are within the Chicago metropolitan area). In addition, the sample for our qualitative semi-structured interviews was small and may not adequately represent the diverse range of perceptions of the older adult population. Future inquiries into the efficacy of targeted telehealth training should aim to capture a broader range of participants.
This study’s findings cannot be separated from the time period during which the TEMs were created and evaluated—specifically, December 31, 2020 to February 28, 2021. This period is associated with the COVID-19 pandemic, wherein Telehealth utilization and use of teleconferencing platforms significantly increased (American Telehealth Association [ATA], 2021; Perrin & Atske, 2021). This historical context poses a threat to our study’s external validity. With trends of increasing Telehealth adoption and utilization expected to continue (Perrin & Atske, 2021), telehealth technologies, practices, and requisite-associated consumer training will similarly need to evolve. Therefore, we recognize our TEMs, and similar client-focused telehealth trainings, should iteratively reflect contemporary practices to best meet older adults’ unique needs. Furthermore, as this population becomes more competent with entry-level telehealth practices, more advanced training and future investigation may be beneficial. We recommend involvement of this population throughout the development process of such training.
To the authors’ knowledge, this is the first study describing the development and influence of telehealth training targeting the unique needs of older adults. Important within the context of this study, we did not use perceived competency (a self-report, quantified by the TCQ-C) as a proxy metric for true competency to use telehealth services. We believe perceived competency to be analogous to confidence, a construct associated with telehealth interest (Edwards et al., 2014) and technology adoption (Vaportzis et al., 2017).
All study procedures, including development of TEMs, were completed by an interprofessional team inclusive of multiple OTPs. We feel the diversity of professional expertise, including consultation with older adults during the qualitative phase, contributed to the study’s outcomes. Furthermore, we feel OTPs offer a unique professional perspective for this area of practice, recognizing aspects of universal design, accessibility, and client factors that restrict access among populations. We feel that OTPs can additionally contribute to this area by applying their unique understanding of the therapeutic relationship and the influence of rapport on telehealth outcomes through future studies. Such investigations may target training for providers, or instead be directed at telehealth clients, enabling them to advocate for a therapeutic relationship with their telehealth providers.
Conclusion
Findings of this study suggest that the barriers to telehealth services disproportionately experienced by older adults may be mitigated through targeted client-focused training. A team of interprofessional health care practitioners, led by and inclusive of multiple OTPs, and in consultation with older adults, created the TEMs and hosting website (available at https://accesstelehealthtraining.org/) developed and evaluated in this study. We feel that the unique perspectives of both OTPs and older adults contributed to the trainings’ influence on older adults’ perceived competency in using telehealth by targeting the specific needs of this population. Furthermore, our findings provide empirical support that older adults who completed TEMs reported increased perceived competency to use telehealth. These findings are especially critical given the combined effects of an increasing population of aging adults, advancements in health care technologies, and a need for increased health care accessibility.
Supplemental Material
sj-docx-1-otj-10.1177_15394492231153115 – Supplemental material for Enhancing Telehealth Competency: Development and Evaluation of Education Modules for Older Adults
Supplemental material, sj-docx-1-otj-10.1177_15394492231153115 for Enhancing Telehealth Competency: Development and Evaluation of Education Modules for Older Adults by Steven Taylor, Samantha Souza, Lauren Little and Jan Odiaga in OTJR: Occupation, Participation and Health
Footnotes
Ethical Approval
This study was approved by Rush University Medical Center’s Institutional Review Board, #18010401-IRB01.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Funding from this study was provided by Rush University Medical Center’s Center for Excellence in Aging.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
