Abstract
Research has shown that using telehealth for rehabilitation assessment can be an effective approach. The COVID-19 pandemic and subsequent lockdowns led to many rehabilitation counselors pivoting to telehealth assessment with their clients. This study explores rehabilitation counselors’ use of rehabilitation assessments and telehealth since the beginning of the COVID-19 pandemic. Using a mixed-methods approach, the researchers analyzed data from 41 rehabilitation counselors across Australia. Participants were asked which measures they used prior to the pandemic, how their use of the measures changed during telehealth, and how their work changed due to the COVID-19 pandemic. The Depression Anxiety Stress Scales and the Occupational Search Inventory were the most commonly used tests. Theoretical analysis demonstrated that participants utilized tests based on their usefulness in comprehensive assessment and rehabilitation planning, for engaging the client in the assessment process, out of necessity (mandated tests), and due to attributes of the test the counselor valued. Participants described the impact of COVID-19 on assessment practice, demonstrating that despite challenges to telehealth, there were also benefits and that assessment measures could be altered for use in telehealth. Although telehealth had an impact on how rehabilitation counselors provided assessments, many found ways to make it work at a distance.
The COVID-19 pandemic and subsequent lockdowns resulted in many changes in rehabilitation service delivery (Doraiswamy et al., 2020; Provenzi et al., 2020; Schmid, 2020). In some cases, these changes led to the use of telehealth services for clients with disabilities that has been met with great success and client satisfaction (Eannucci et al., 2020; Provenzi et al., 2020). Therefore, utilizing telehealth to conduct assessment in rehabilitation has the potential to provide important services to individuals with disabilities in a way that increases access and improves accessibility. Although some research on assessment delivered by telehealth exists, the COVID-19 pandemic forced many rehabilitation assessment providers into telehealth, providing the perfect opportunity to examine assessment in a telehealth context on a larger scale.
Research prior to the COVID-19 pandemic has shown that using telehealth for rehabilitation assessment can be effective for some assessments and with some clients (Loh et al., 2004; McEachern et al., 2008; Schieltz & Wacker, 2020; Taylor et al., 2014). It can make assessment more accessible for clients, especially those in rural areas (Müller et al., 2017) or for clients who experience transportation barriers (Valdez et al., 2021), saving time by not necessitating that individuals travel to access services (Dorsey et al., 2013; Helleman et al., 2020). Clients may appreciate the opportunity to undertake assessment from the comfort of their home and providers may benefit from seeing clients in their home environments, leading to a more robust view of day-to-day functioning (Cary et al., 2016). Younger clients with disabilities also express direct interest in increased access to telehealth services for health care provision (Noel & Ellison, 2020).
Of course, conducting assessments by telehealth is not without challenges. Providers may find that it is difficult to assess without in-person physical cues (Luxton et al., 2014). They may also find that problems with technology and inexperienced technology users can lead to complications in the assessment process (Luxton et al., 2014). Providers may have ethical and privacy concerns (Luxton et al., 2014; Schopp et al., 2000) as well as cultural concerns (Chao et al., 2020; Luxton et al., 2014) in the assessment process. All these present important challenges to be aware of when conducing telehealth assessments in rehabilitation. Although previous work in telehealth has demonstrated similar efficacy to in-person services (Bernocchi et al., 2019; Riegler et al., 2017; Speyer et al., 2018), the pandemic and subsequent lockdowns have meant that telehealth has been accessed by a range of new users in the absence of other alternatives to face-to-face meetings available to undertake rehabilitation assessments.
There has been very little research on the use of telehealth in rehabilitation counseling settings (Lewis et al., 2017). This is especially true regarding rehabilitation assessment, despite its importance (Strauser & Greco, 2019b). Rehabilitation assessment provides important information about a client’s abilities, personality, interests, career goals, among other things (Strauser & Greco, 2019a). This information can be gathered through a comprehensive and targeted assessment process. In Australia, rehabilitation counselors are trained in assessment and are expected to conduct assessments with clients (Buys et al., 2015), similar to how assessment in rehabilitation counseling is conducted in the United States.
Rehabilitation counselors in Australia and all over the world have been required to adapt their assessments to telehealth due to the recent pandemic and the lockdown restrictions that prevent in-person interactions, yet little is known about how telehealth has affected their use of rehabilitation assessment measures. This study aimed to explore rehabilitation counselors’ use of rehabilitation assessments and telehealth since the COVID-19 pandemic. We were interested in finding out what measures rehabilitation counselors used and the reasons why they were used. We were also interested in exploring whether or not the use of telehealth required that rehabilitation counselors change or alter their assessment measures and whether this affected the overall quality of their assessments. Last, we were interested in exploring to what degree rehabilitation assessment may have changed due to the COVID-19 pandemic. Having a better understanding of rehabilitation counselors’ experiences of telehealth assessments can help us to better plan for the implementation of large-scale telehealth services that may enable better reach to rural areas and increased access for clients with disabilities.
Method
We collected data for this exploratory mixed-methods study between October 2020 and July 2021. Prior to undertaking the study, ethical approval was obtained by the University of Sydney’s Human Research Ethics Committee. Data were collected through use of an online Qualtrics survey. The research questions underpinning this study were the following:
Participants
Currently, there are approximately 1,200 counselors in Australia who are affiliated with professional rehabilitation counseling associations (Australian Society of Rehabilitation Counsellors [ASORC], 2021; Rehabilitation Counselling Association of Australasia [RCAA], 2021), and any currently practicing rehabilitation counselor in Australia was eligible to participate in the study. Seventy-six people accessed the online survey: six (7.9%) did not start the survey, 17 (22.4%) were not eligible to participate (i.e., indicated that they were not rehabilitation counselors or did not do assessments), and 12 (15.8%) provided incomplete responses. Those who provided incomplete data were all from New South Wales and did not differ significantly in age or gender from those who completed the survey. Forty-one (54%) rehabilitation counselors completed the online quantitative survey and open-text qualitative response items and these data were included in analyses. The majority were women (87.8%), which reflects the dominance of women in the profession. They had a mean age of 36.79 (SD = 9.49) years and average length of employment in the field of rehabilitation counseling was 8.44 (SD = 6.22) years. Responses came from six Australian states/territories, with the majority from New South Wales (n = 24, 60%). Participants were able to choose multiple responses to indicate which field they worked in, but the fields of worker’s compensation and psychological claims were most common (both n = 33), followed by private occupational rehabilitation (n = 17). These participant data are presented in Table 1.
Participant Characteristics for Telehealth Study Sample.
Participants could nominate more than one field of practice.
Procedure
LinkedIn, Twitter, and Facebook posts promoting the study invited practicing rehabilitation counselors to complete a Qualtrics survey on rehabilitation assessments and COVID-19. These were posted on rehabilitation counseling groups and pages as well as on the personal pages of the authors. Finally, the professional organizations promoted the survey through their social media presence, as well. To gain entry to the survey, participants were required to read the study’s ethics-approved information sheet and click a button to confirm that they agreed to participate. The survey comprised a set of quantitative set response items on personal demographic background, years of experience, and field of employment and a set of qualitative open-text items for the top three assessment measures used and why they were used, how their use of each measure had changed since the COVID-19 pandemic began, and finally, how their work in assessment had changed since the beginning of the COVID-19 pandemic. Survey responses were downloaded to SPSS (IBM Corp, 2016) for analysis.
Analysis
Quantitative data were analyzed using SPSS (IBM Corp, 2016). Descriptive analyses were used for demographic variables, field of employment, and years of experience. Qualitative data were analyzed inductively, using content analysis (Mayring, 2004). The first author and senior author analyzed the qualitative data separately, coding each individual response and then comparing codes and discussing themes. This inductive coding process developed codes to elements and then to themes relevant to the research questions.
Results
Measures Utilized Prior to COVID-19
Participants were asked to report on the three tests they most used in their practice. This resulted in 35 unique tests, 19 of which were mentioned only once. Tests that were utilized by more than one respondent can be found in Table 2. The most commonly used assessments by this cohort were the Depression Anxiety Stress Scales (Akin & Çetın, 2007) and the Occupational Search Inventory (Pryor, 2001). Tests that assessed psychological function and career interest/skill inventories featured highly in rehabilitation assessments (see Table 3).
Tests Most Used in Participants’ Practice.
Nature of Tests Most Used in Participants’ Practice.
Reason for Using the Measures
Participants’ responses to the reason why they used the identified measures in their rehabilitation assessments resulted in four themes. Two related to the usefulness of the assessments in comprehensive assessment and rehabilitation planning and their use in engaging clients in the process. Another two focused on more practical concerns, including necessity (tests that are mandated by their employer) and attributes of the test that were valued in their assessments.
Comprehensive assessment and rehabilitation planning
These measures allowed participants to obtain the data needed for a comprehensive vocational assessment and subsequent counseling. Specific measures provided a “comprehensive picture of symptom impact/severity,” of “health, life, work and wellbeing,” allowing them to “ascertain barriers, aptitudes and how well a client will transition back into employment.”
Symptoms and conditions could be measured and tests could be “re-administered over time to observe changes in symptom experience.” This information assisted in the tailoring of support and services provided and gave “insight to guide how to approach many case management activities.” Results from assessments were also useful in providing crosschecks to determine “whether responses align with presentation and self-reports.” In all, the assessments used by participants provided them with the level of information they needed to prepare a vocational evaluation that would underpin their client’s rehabilitation plan.
Client engagement
Participants also raised the importance of developing rapport to ensure client engagement in the assessment process. Some utilized the measure as a “tool to open conversations” with clients, to further develop rapport, to prompt clients “to think about where their skills and interests lie,” and to participate in “brainstorming” activities to identify vocational options available to them. Others mentioned the importance of using assessments that were easy for their clients to use, for example, selecting measures “validated for [clients with] non-English speaking backgrounds.”
Mandated assessments
Some participants were required to use specific assessment measures for various providers or insurers. These included a “required tool by DVA [Department of Veterans’ Affairs, Australia]” and “NDIS [National Disability Insurance Scheme] endorsed tools.” Others mentioned “in house assessments” and assessments that were a “requirement but also useful to open up discussions.” The use of these assessments enabled participants to provide the required outcome reporting data to the relevant authority but also provided the information required for a vocational assessment.
Assessment attributes
Several qualities were noted by participants as contributing to their continued use of an assessment. One quality was “commonly used.” Commonly used assessments promoted comparability between providers and different health professionals: “It is commonly used, so may be used as a measure of progress between various appointments, for example, a psych appointment, vocational counseling session, and vocational assessment.” Being “easy to use” was mentioned in the context of the test’s use and availability: “[easy] and workplace has it available”; administration: “easy to administer, especially online”; and communication: “easy to communicate, able to observe.” Another sought quality for using an assessment was being “evidence-based” because it ensured results were “scientifically valid and accepted by mental health practitioners.” The last quality was that an assessment was able to “measure change” over time, to document “actual changes with perceived changes in function,” and therefore be able to report the “success of RC interventions.”
Influence of COVID-19 and Telehealth
Participants were asked specifically about how use of the three most used tools in their practice had changed due to COVID-19 and whether this affected their work in assessment. Their responses to these questions fit together thematically and therefore were combined during analysis. This produced three themes relating to the way the tool could be altered for use in telehealth, the challenges, and the benefits of telehealth assessment.
Same tool, different delivery
Many participants reported altering how they delivered their assessments and particular tests during the COVID-19 stay-at-home restrictions that accompanied the pandemic. Instead of face-to-face delivery, assessments were delivered by telehealth: online, by email, telephone, Microsoft Teams, Zoom, SurvStar, or other web-based platforms that their organization offered. The flexibility in participants’ approach to making assessments work through telehealth was evident, even though it made “life a little harder, not impossible though.” Some sent materials prior to assessment rather than completing during the assessment, some used “share screen to carry out assessments,” and others met with clients more regularly, emailed tests to be completed prior to meeting, read questionnaires to clients, and marked responses over the phone.
A few reported that telehealth formats of tests were “cumbersome to use” or “impractical” and they were therefore used less, while others were used “more often.” Some indicated that assessments had “been well adapted to telehealth” and that “use of technology found the tool still has similar benefits [when] done online.”
Assessments via telehealth: Challenges
Several challenges associated with telehealth assessment and service delivery were raised by participants. It took more time and was “slightly trickier” to build trust and rapport with clients using telehealth, particularly by telephone, because the “personal interaction side of it” and the ability to “observe the client to gather non-verbal information or cues” had been removed. The lack of personal connection also increased incidents of “communication overlap or misunderstanding.”
Participants noted that not all clients were literate in the mechanisms that enable telehealth. People with brain injury or other cognitive conditions, for example, had “more challenges engaging via the telehealth platform.” A similar challenge exists for clients who have “psychological claims that require more regular, hands-on and in-person approaches.”
The final challenge was that participants were “unable to complete comprehensive observations of function” specifically a client’s function “in different settings” and “across different domains.” What this meant is that some assessments relied more on “reports of other allied health professionals, client, family and support workers” rather than their direct observation.
Assessments via telehealth: Benefits
The change to telehealth assessments also presented opportunities by expanding to new areas of work with clients from areas “that would not otherwise have access to our services.” For others, changing the way they did the assessments resulted in unanticipated benefits. Simple decisions like sending assessments to clients to complete prior to the telehealth meeting had great benefit by “creat[ing] more time for conversation with the client” and allowing for “more meaningful consideration of thoughts and aspirations related to work for clients.” Some participants also advised that timeframes for assessments from rehabilitation counselors and other health professionals “are quicker now that all services are delivered online.”
Discussion
The findings of this exploratory study suggest that the COVID-19 pandemic and lockdown periods have had a notable impact on the delivery of assessments in rehabilitation counseling. Notwithstanding, many measures that were preferred and previously used when meeting face-to-face with clients have been continued to be administered by different methods. Preferred measures were tests that assessed psychological function and career interest/skill inventories, specifically the Depression Anxiety Stress Scales (DASS) and Occupational Search Inventory (OSI) II/III, and these reflected the dominance of psychological claims and worker’s compensation work being done by participants. Measures were selected because of their ability to provide the information needed for a comprehensive vocational assessment and to provide direction for counseling. Some were mandated and others had qualities that were valued by rehabilitation counselors, in particular the replicability of the measures, and their ability to measure change over time, and their use by several professionals involved in the rehabilitation process.
Perhaps advantaged by the self-administered assessment design of the most frequently preferred assessment tools (Akin & Çetın, 2007; Kessler et al., 2003; Linton, 1999; Pryor, 2001), rehabilitation counselors reported ongoing successful collection of psychological and career interest/skill data required for comprehensive personal vocational assessment and subsequent counseling following the onset of the COVID-19 pandemic. Labor market assessments, on the contrary, were undertaken with greater uncertainty, influenced by the impact of COVID-19 on local labor markets (Australian Government, 2020).
Following the onset of the COVID-19 pandemic and the use of telehealth for assessments, creativity on the part of the rehabilitation counselor saw tests delivered in different ways: by email, by telephone, or even screen-sharing during a teleconference to go through the assessment with the client. Like others previously, rehabilitation counselors adapted and found ways to work using the tools they preferred that were also beneficial to their clients (Cary et al., 2016; Tenforde et al., 2017). These adaptations were important in allowing telehealth assessment to move forward in the context of COVID-19, on very short notice. Rehabilitation counselors, accustomed to providing accommodations, may have been ideally situated to make these changes (Wehman, 2017).
In addition, telehealth allowed service delivery in areas where clients generally may not have had access to local services, a benefit noted by telehealth researchers in other settings (Müller et al., 2017; Speyer et al., 2018). Rehabilitation counselors found ways to spend more time on discussion with clients by having them complete tests ahead of their scheduled session when possible. They also reported that assessment turnarounds were faster in a telehealth environment, which could lead to improved access to services for clients (Al-Rashaida et al., 2018).
Doing rehabilitation assessments using telehealth was not without its challenges. Although most assessments reported through this study were adaptable to telehealth administration, participants reported difficulty in developing rapport in a telehealth environment. This is not only a problem for rehabilitation counselors doing telehealth assessments. For example, Luxton et al. (2014) previously reported difficulties in rapport building when conducting telehealth psychological assessments. The absence of nonverbal cues and inability to see client functioning in person that was reported by our participants appears a common experience when using telehealth. Similar findings have been reported in areas outside of assessment, such as medical treatment (Helleman et al., 2020; Martinez et al., 2017). Other socioenvironmental factors, such as exposure to family and domestic violence, can also reduce reliability and increase risk in telehealth contexts for assessment (Jack et al., 2021).
Participants noted that clients with certain types of disabilities, such as brain injuries and psychological conditions, may struggle with assessment delivered by telehealth. Pre-appointment assessment in telehealth has previously identified risks in populations experiencing psychological distress (Cowan et al., 2019; Gilmore & Ward-Ciesielski, 2019), with clients potentially requiring greater time resources during appointments for completion of mandated outcome measures. However, with time, practice, and outside of the stressors of a global pandemic, rehabilitation counselors may find creative ways to address these problems and see these barriers to telehealth assessment differently. Future research that explores this area of assessments by telehealth is warranted.
Limitations
Ultimately, although this study presents important information on how rehabilitation counselors have adapted to the COVID-19 pandemic through telehealth assessment, it does have a number of limitations. The COVID-19 pandemic and accompanying lockdown periods have not made the most ideal of settings to move to telehealth, and as such, research conducted during this time should be taken within the context in which it was conducted, clearly a limitation of this work. Furthermore, working from home during COVID-19 has generated increased workloads (Wang et al., 2021; Yang et al., 2020), unpaid work demands (Craig & Churchill, 2021), and technical overload (Molino et al., 2020), which may, in part, have affected the uptake of this survey by rehabilitation counselors.
Although rehabilitation counseling is a small profession in Australia, with approximately 1,200 rehabilitation counselors who are members of professional rehabilitation counseling organizations (ASORC, 2021; RCAA, 2021), the small sample size of participants limits the generalizability of the work. The fact that the study was solely conducted in Australia and lacked sufficient responses from participants in particular fields (not-for-profits, National Disability Insurance Scheme [NDIS]) and particular states and territories (Western Australia, South Australia, Northern Territory, and Tasmania) presents a further limitation. Similarly, using social media as a means to recruit participants likely limits the generalizability of the work, as it is not representative of all rehabilitation counselors in Australia. Some rehabilitation counselors may be less likely than others to access the social media accounts and groups where the survey was marketed, while others may be more likely to respond to a social media post. Last, the results of this study cannot address the effects of the changes that were made to assessments and how those affected the reliability and validity of the assessments in question. However, with limited research on telehealth assessment available in the literature, and specifically telehealth assessment in rehabilitation counseling, this article provides an important baseline insight into the benefits and barriers faced by this profession.
Future Research
Future research that addresses how the changes to the use of these assessments in a telehealth context will be important as we progress to a world where telehealth becomes more ubiquitous. In addition, future research should explore the use of telehealth assessment in fields of rehabilitation counseling and geographical areas not well represented in this survey. For example, not-for-profits, National Disability Insurance Scheme (NDIS), and participants from some states and territories were not well represented in this study’s cohort and they could provide additional insight into the use of telehealth assessments by rehabilitation counselors in Australia. In-depth interviews with clients—who may view telehealth assessments differently to participants in this study—and with rehabilitation counselors could help us better understand the barriers and facilitators to telehealth in rehabilitation counseling. Studies that identify creative ways that assessments and rehabilitation counseling practices have been adapted over time would add to the literature on this topic. Findings from all these areas of research will help guide the development of improved processes and supports that alleviate barriers and improve both client–counselor experience of telehealth assessments.
Implications for Practice
This study provides important insight into telehealth assessment as a result of COVID-19. Importantly, and unsurprisingly, rehabilitation counselors adapted well to the challenges of the COVID-19 pandemic. Knowing how rehabilitation counselors adapted measures for use in telehealth during an emergency can provide important information for test developers and psychometricians who can then validate these measures for their use in telehealth in the future. In addition, understanding the preference for self-administered tests for telehealth practice can also guide test developers and psychometricians in future practice.
Conclusion
Ultimately, this inaugural study examining the use of telehealth in rehabilitation counseling assessment in Australia suggests that it is possible. Research on telehealth assessment, including this work, has demonstrated marked benefits in the process. When considering new ways of working and assessment practices that have greater equity and reach, especially to rural and remote clients with disabilities who may experience transportation barriers compounded by the intersection of geographic location and disability. To remedy this, the rehabilitation counseling profession needs to consider how and where telehealth features into the assessment landscape while taking these factors into consideration. Just because we can go back to in-person assessment does not necessarily mean that we should.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
