Abstract
Introduction
Care coordinators (CCs) are specialized healthcare providers and often the primary point of contact for patients with multiple medical and mental health comorbidities in integrated healthcare settings. Prior work shows CCs have lower comfort addressing mental health than physical health concerns. Digital mental health interventions can support CCs’ management of patient mental health needs, but training gaps must be addressed prior to a digital mental health intervention's implementation.
Methods
As part of a quality improvement initiative, a 1-hour training focused on the assessment and management of depression and suicide-related thoughts and behaviors was delivered to CCs within a large midwestern healthcare system's Division of Ambulatory Care Coordination. CCs completed online surveys prior to and following the training.
Conclusion
Training resulted in increased comfort working with clinical populations, including patients who experience suicide-related thoughts and behaviors. Gains around screening for suicide risk were modest. Brief trainings for CCs can address training gaps, however, ongoing training and case consultation may also be indicated.
Introduction
Brief Review/Discussion of Topic
Healthcare systems in the United States are increasingly adjusting their care delivery models to adopt value-based care payment structures. In value-based care models, payers (i.e., insurance companies, government insurers, etc.) incentivize health systems and providers to improve patient outcomes by compensating these systems and providers based on patient outcomes rather than on the number or type of services provided (Burwell, 2015; Cutler, 2015; Rosenthal & Dudley, 2007). By incentivizing outcomes, payers drive hospital systems to deliver cost-efficient, patient-centered, evidence-based care. Multimorbidity (defined as patients having two or more medical conditions)—which is estimated to affect over 25% of all adults (Ward et al., 2014)—is one of the most pressing challenges to value-based care (Tinetti et al., 2012; Zulman et al., 2014). The management of multiple comorbid conditions adds significant complexity to incentivizing patient oriented outcomes (rather than fee-for-service models) since these conditions impact one another and treatment must be based on patients’ unique combination of conditions (Zulman et al., 2014). Among the multiple chronic conditions U.S. adults manage, mental health conditions have been among the top five costliest conditions for more than two decades (Agency for Healthcare Research and Quality (AHRQ), 2020).
Care coordination programs are increasingly offered within integrated healthcare systems to more effectively address the needs of patients with multiple physical and mental health comorbidities including depression and suicide-related thoughts and behaviors (Archer et al., 2012). Care coordinators (CCs) are specialized healthcare service providers tasked with helping patients with multiple comorbidities navigate the complexities of modern healthcare systems. They achieve this goal by helping patients share relevant information with the appropriate providers, make appointments for different inter-related conditions, adhere to treatment regimens, and implement behavioral or lifestyle changes that are linked to improved patient outcomes. In this way, CCs play a critical role in maximizing patient outcomes and minimizing health system costs (Thorpe et al., 2017). CCs are also often the primary point of contact for patients with medical and mental health comorbidities and can operate as a bridge between services.
One way CCs may be able to support the comorbid mental and physical health needs of their patients is by integrating digital mental health interventions (DMHIs; e.g., apps, web-based treatments, provider portals, etc.) into their workflows. DMHIs are increasingly being used in healthcare settings to augment existing care structures, increase efficiencies, and provide both patients and providers with resources that can prevent patient symptom worsening between provider visits. DMHIs can provide CC staff with resources to help address patients’ complex and interrelated needs, especially comorbid mental health needs (Bauer et al., 2018; Rollman et al., 2018). DMHIs that have both patient- and CC-facing features (1) allow patients to engage in relevant therapeutic modules between provider visits, increasing the resources for symptom self-management; (2) enhance CC's provision of care by providing decision support tools that can help CCs follow appropriate procedures when working with patients who present with comorbid mental health concerns; (3) provide CCs with psychoeducation about common comorbid mental health conditions and how they interact with physical health conditions; and (4) facilitate the collection and monitoring of relevant patient data at a higher frequency than patient visits with medical providers, allowing for earlier intervention (Bauer et al., 2018; Mohr et al., 2018; Rodriguez-Villa et al., 2020; Rollman et al., 2018).
While there is great potential for DMHIs integrated in Care Coordination departments to improve patient outcomes and reduce cost of care, past work demonstrates that many CCs, who are primarily nurses by training, feel more comfortable addressing physical health concerns than mental health concerns (Lattie et al., 2021) and have less formal mental health training (Au et al., 2018). Despite being outside of many CC's formal training, CCs roles increasingly demand assessing and addressing patients’ comorbid mental health symptoms (Lattie et al., 2021).
This study represents a collaborative quality improvement initiative led by a major regional healthcare system's Division of Ambulatory Care Coordination. Recognizing that implementations of DMHIs within healthcare contexts often fail due to the complex nature of healthcare organizations and differing demands of health systems (Graham et al., 2020), this initiative aimed to: (1) elicit training needs prior to the implementation of a CC-administered DMHI to support patients with comorbid mental health symptoms, and to (2) increase CC's comfort and perceived skillfulness working with patients experiencing comorbid mental health conditions (e.g., depression, suicide-related thoughts and behaviors). This quality improvement initiative relied on a modified version of the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework (Aarons et al., 2011; Moullin et al., 2019) to first explore the needs of CCs (Lattie et al., 2021), collaborate with the division to explore appropriate interventions that would support improved care and outcomes for patients (i.e., integrating a DMHI within the Division of Ambulatory Care Coordination), identify and address needs and barriers to implementation (the subject of this brief report), implement the new intervention while monitoring for any additional supportive infrastructure, and provide ongoing support within the organization and Division of Ambulatory Care Coordination to facilitate sustained implementation of the planned DMHI. The focus of this study is addressing key identified barriers to a DMHI's implementation in the Division of Ambulatory Care Coordination through training. Subsequently, the objective of this brief report is to evaluate the ability of a brief training workshop to address previously identified CC training needs and increase CC's self-reported comfort and skillfulness working with patients experiencing comorbid mental health conditions within the context of a large regional healthcare setting.
Methods
Participants and Procedures
This mixed methods study was part of an ongoing quality improvement initiative in collaboration with the Division of Ambulatory Care Coordination at a large midwestern hospital system. The hospital system operates more than 10 hospitals, several medical groups, and several hundred ambulatory service locations. The system maintains a staff of over 30,000 and is one of the largest care systems in the Midwest delivering care across over 120,000 inpatient visits and 3 million outpatient visits, annually (Northwestern Memorial HealthCare, 2021). The Division of Ambulatory Care Coordination planned to implement a DMHI to be administered by CCs in an effort to improve patient outcomes and increase care efficiencies, however, prior quality improvement studies conducted with CCs (Lattie et al., 2021) revealed that additional training and policy infrastructure was necessary before a DMHI was implemented. In particular, ambulatory care coordination leadership determined that a 1-hour training should be conducted to increase knowledge, confidence, and skill among CCs who work with patients with depression and suicide risk. Brief trainings are a common method addressing mental health knowledge and skill gaps among both behavioral and non-behavioral healthcare staff, especially when trainings target deficits in suicide prevention skills (Berlim et al., 2007; Cross et al., 2010; Pisani et al., 2011). A 1-hour skills training workshop for all CCs in the Division of Ambulatory Care Coordination (amounting to approximately 30 CCs) was conducted in December of 2019. Using a repeated cross-sectional study design, voluntary online surveys (Harris et al., 2009) were distributed to CCs 2 weeks prior to the training and in the week following the training.
Sample
Pre-training sample consisted of 16 staff-level CCs and post-training sample consisted of 13 staff-level CCs. Due to the nature of the quality improvement initiative, no formal inclusion or exclusion criteria were established. However, all included CCs were employed by the large regional healthcare organization, were within the Division of Ambulatory Care Coordination, and had a role of CC. CCs must have attended the in-person training to have been sent the post-training survey.
Training Workshop
Training content was delivered by JM
Measures
Surveys to assess CCs training needs and impact were developed by the quality improvement team, comprised of two clinical psychologists (JM and EGL) and leadership from the Division of Ambulatory Care Coordination including one member of the team who is a licensed clinical social worker and accredited case manager. Survey items were designed to elicit insights about CC comfort working with different patient populations, self-assessments of skills, knowledge, and confidence with a particular clinical duty or population within the context of a major healthcare system. Items were measured using a Likert scale from 1–5 with 1 = very uncomfortable/very low/strongly disagree and 5 = very comfortable/very high/strongly agree. Survey items also assessed whether CCs incorporated or planned to incorporate screening for suicide-related thoughts or behaviors into their workflows, these relied on a dichotomized “yes” or “no” response pattern. Finally, CCs categorized their preferences regarding working with patients who experience suicide-related thoughts or behaviors, responses were mutually exclusive categories (e.g., I make it a practice not to work with suicidal patients; If I could, I would prefer not to see suicidal patients; When I find out a new patient is suicidal, I refer them to someone else, etc.). Additional items assessed CCs mental health training backgrounds. Qualitative data collection included free-response questions about perceived barriers to assessing mental health symptoms and suicide risk and additional trainings desired to facilitate comfort managing needs of patients with mental health concerns. 1 Where possible, descriptive statistics between pre- and post-training workshop were examined.
Quality Improvement and Institutional Oversight
This study was conducted as part of a quality improvement initiative to improve the care coordination services at a large regional healthcare organization within a specific Division of Ambulatory Care Coordination. Quality improvement projects that do not meet the US Health and Human Services definition of research fall outside of the purview of the authors’ institution's institutional review board.
Analysis
Pre-Training Results
Sixteen members of the care coordination team (15 registered nurses and 1 licensed clinical social worker) completed pre-training surveys. The majority (11/16; 68.8%) of the sample endorsed some formal training working with patients experiencing mental health problems, but most (11/16; 68.8%) had not received any formal training in suicide risk assessment. To this end, the vast majority of participants noted feeling as if their training in mental health assessment (14/16; 87.5%), broadly, and suicide prevention (13/16; 81.3%), specifically, was insufficient. In open-ended responses, 50% (8/16) of CCs noted some lack of training or knowledge about how to assess for suicide risk as the primary barrier to assessing risk during telephone contacts with patients. For example, one CC whose response was representative noted, “…[I’m] not sure what questions to ask, how direct to be, and how to respond based on the answers provided.”
Post-Training Results
Thirteen CCs completed post-training surveys (11 registered nurses and 2 licensed clinical social workers). Following the training, CC's median comfort level working with patients with various disorders increased across each disorder as did their knowledge of risk and protective factors for suicide, and self-reported skill level when working with patients with suicidal ideation (Table 1). CCs reported median agreement with statements that they possessed the knowledge, skill, and confidence to recognize suicide risk, and appropriately respond to patients with suicide risk, and follow organizational procedures when patients presented with suicide risk (Table 1; note these questions were only assessed at post-training). Importantly, however, CCs reported that following the training they neither agreed nor disagreed that they had the knowledge and skill to screen patients for suicide risk (Table 1).
Likert Scale Survey Responses.
Items rated on a Likert scale of 1 (very uncomfortable/very low knowledge, skill, or confidence/strongly disagree) to 5 (very comfortable/very high knowledge, skill, or confidence/strongly agree).
A 1-hour training had minimal impact on whether CCs asked or planned to ask patients about suicide-related thoughts or behaviors during their first encounter. While most CCs planned to ask all patients about suicide-related thoughts or behaviors, some planned not to ask patients about these experiences (Table 2). Similarly, this 1-hour training had minimal impact with regard to CCs preferences for working with patients experiencing suicide-related thoughts or behaviors, with many CCs preferring to refer patients with these experiences to other providers (Table 3).
Plans to Ask Patients About Suicide-Related Thoughts or Behaviors on First Encounter.
Care Coordinators’ Stance Toward Seeing Patients Presenting With Suicide-Related Thoughts or Behaviors.
Following the training, the biggest anticipated barriers to implementing regular mental health and suicide risk assessments included: time constraints, ill-defined protocols for patient follow-up based on responses to mental health screening, and the telephonic nature of care. CCs also reported that concrete policies and protocols would assist them in executing their responsibilities and implementing additional mental health assessments and interventions. One CC noted, “Structure – defined protocols [are necessary to feel more comfortable working with patients with mental health and suicide-related concerns]. Management team needs to be clear on what their expectations are with medical nurses managing BHS [Behavioral Health Service] cases…” This response highlights the need for a top-down shift of the aims and expectations of the CC role.
Discussion
The present study was part of a larger implementation effort to deploy a DMHI in the context of Coordinated Care within a large healthcare system. To increase the chances of successful DMHI implementation, we followed guidance from Graham et al. (2020), and used a modified version of the EPIS framework tailored to integrating DMHIs into complex healthcare organizations. Prior to deployment, the Division of Ambulatory Care Coordination conducted exploratory needs assessments with patients and providers to determine if a DMHI would be an acceptable and appropriate tool to meet the needs of patients and providers (Lattie et al., 2021). At the exploratory phase, a DMHI was thought to be a viable solution with potential to improve patient outcomes and increase cost-effectiveness of care (Lattie et al., 2021). Preparation focuses on ensuring training and ongoing supervision is established to ensure a workforce has the skills and knowledge needed to deliver a DMHI when it gets implemented. In the context of our work, this meant having clear guidelines around how CCs would practically deliver the DMHI and the appropriate cadence of patient engagement with the DMHI (e.g., weekly, daily, etc.). It is also important to establish clear workflows and minimize the need to use multiple disconnected electronic systems. CCs in our sample were required to engage in mental health assessment and symptom management, however, these tasks can be a substantial disruption to CCs workflow and often requires using and monitoring multiple electronic health record systems to document and track patients’ mental health information (Lattie et al., 2021). Ensuring CCs do not have to balance multiple electronic health records and have the information they need readily available will reduce strain and resistance to using the DMHI. Efforts to address structural barriers in addition to training-related barriers will improve CCs capacity to address patient mental health concerns prior to the implementation of a new DMHI.
To this end, this study highlights the feasibility and utility of identifying and addressing mental health training needs within a care coordination team of a major hospital system prior to the planned implementation of a DMHI. Specifically, results suggest that low-cost, but well-planned, training sessions are feasible to implement, and contribute to increases in CC knowledge, skill, and confidence in their ability to perform new duties. However, following the training, CCs largely did not report increases in their preferences for working with patients with suicide-related thoughts or behaviors or confidence in their ability to screen patients for suicide risk. This finding suggests that a single 1-hour training session may be insufficient to entirely address certain implementation gaps.
Our finding that training is necessary but insufficient to fully address knowledge and skill gaps in preparation for the implementation of a DMHI, is consistent with findings from prior work. When implementing measurement based mental health monitoring within a large healthcare system, Steinfeld et al. (2016) noted skill gaps among staff who would be administering mental health measures and deployed a 90-min training to address these gaps. In addition Steinfeld et al. (2016) also ensured that staff had minimal disruption to their workflow and concrete organizational sponsorship which consisted of hospital system leadership setting clear implementation targets. To this end, it is important to consider our findings in the context of the organizational environment in which we conducted our study: as part of preparatory efforts by the Division of Ambulatory Care Coordination to implement a CC administered DMHI.
Our data suggest that while trainings can improve CC skill and comfort assessing mental health and suicide risk, they are limited. They cannot provide CCs with hands-on experience, time, or organizational policy support. As is done with others who routinely work with individuals with suicide-related thoughts or behaviors, it will be necessary to build organization-level scaffolding for ongoing trainings to support continued skill building, knowledge acquisition, and increase adoption of best-practices when working with patients with suicide-related thoughts and behaviors. It will also be necessary to implement strong ongoing group supervision or case consultation models that provide CCs with greater exposure to, and experience working with, patients who have complex needs. Finally, it will be necessary to have extensive training in using the DMHI and facilitating CC comfort with using all the features and pragmatic tasks that must be completed within the DMHI (e.g., patient communications, sending and viewing questionnaire data, accessing decision support tools, etc.).
When implemented in a setting that has the proper scaffolding, DMHIs hold promise for supporting CCs in working with patients with comorbid mental health symptoms and for improving patients’ mental health-related outcomes. DMHIs can have provider-facing dashboards that display patient generated data. DMHIs can push depression and risk assessments to patients via Ecological Momentary Assessments at regular intervals as has been done in other digital services (Glenn et al., 2020; Gratch et al., 2021; Kaurin et al., 2022; Kleiman & Nock, 2018; Morgiève et al., 2020). They can also assist CCs who have lower levels of experience with patients who have mental health comorbidities by providing policy-backed evidence-based decision support tools for CCs within the DMHI dashboards. While these digital tools hold promise, it is necessary to first develop the training and administrative scaffolding necessary to support CCs as administrators and users of such a DMHI.
Limitations
A primary limitation of this study is that these data are not longitudinal, but rather use a repeated cross-sectional design. The data from this study comprise two cross-sectional samples of CCs and are not within person data and, subsequently, must be interpreted with caution. An additional limitation is that this work was conducted within a single healthcare institution and the specific efforts and insights described may not be feasible or generalizable to other healthcare systems.
Conclusions/Implications to Nursing Profession
Brief workshops that center on specific mental health training gaps can increase CCs’ knowledge and comfort working with patients with mental health comorbidities. However, prior to a full implementation of a DMHI, CCs must have ongoing training mechanisms enabling them to have the knowledge and skills to address patients’ mental health needs.
Footnotes
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Jonah Meyerhoff has accepted consulting fees from Boehringer Ingelheim. David C. Mohr has accepted honoraria and consulting fees from Otsuka Pharmaceuticals, Optum Behavioral Health, Centerstone Research Institute, and the One Mind Foundation. He also receives royalties from Oxford University Press and has an ownership interest in Adaptive Health, Inc. Emily G. Lattie has accepted consulting fees from Modern Health and honoraria from Streamline Healthcare Solutions. None of the other authors have competing interests to declare.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by NIMH [P50MH119029, T32MH115882, R34MH124960, R34MH128410, K08MH112878, K08MH128640] and AHRQ [R01HS028003].
