Evaluating the Barriers and Facilitators to Implementing a Novel Referral System for Outpatient Geriatric Services: The Geri-Hub Quality Improvement Initiative

Abstract

Background:

In healthcare systems prioritizing care of older adults, resource limitations and escalating demand often impede access to outpatient specialized geriatric services.

Objectives:

This study, theoretically guided by the Consolidated Framework for Implementation Research (CFIR), aimed to explore barriers and facilitators in implementing a centralized “Geri-Hub.” The Geri-Hub is a centralized intake system established within 2 hospital systems to coordinate outpatient and community-based services for older adults, aiming to connect them with the most appropriate care in a timely manner.

Methods:

Qualitative insights were gathered from healthcare professionals at 2 academic institutions in the process of consolidating services. Through open-ended surveys and semi-structured interviews, we solicited feedback on referral management, waiting times, and overall work experiences.

Results:

Thirteen frequently referring providers and a cohort of 9 geriatricians, along with 4 administrators, contributed to the study. Geriatricians emphasized streamlined referrals, flexible scheduling for urgent cases, and a target wait time of 3 months. Administrators stressed standardized referral procedures, defined roles, and accessible referral information.

Discussion:

The findings underscored the need for straightforward referral processes, enhanced communication on referral statuses, and reduced wait times. Optimizing these processes could potentially mitigate resource utilization issues and improve patient outcomes in healthcare systems. This research highlights the critical role of timely access to geriatric services during transformative phases in healthcare delivery.

Keywords

Specialized geriatric services older adults implementation research Consolidated Framework for Implementation Research (CFIR)centralized intake system Geri-Hub healthcare professionals referral management waiting times resource utilization patient outcomes

Background

There is an urgent need to improve the delivery of healthcare for older adults in Canada. Forecasts indicate a 68% rise in those aged 65+ and a doubled proportion for those 75+ by 2037, significantly impacting healthcare spending.¹ Despite constituting one-fifth of the population, older adults consume nearly half of healthcare expenditures.² While it is reasonable to attribute the disproportionate cost as a result of the increasing number of medical issues that come with aging requiring more medical attention, it is important to examine the inefficiencies driving high costs and inappropriate delivery.³ Tertiary care delivery notably incurs high expenses, with prolonged hospital stays contributing significantly.³ Moreover, the direct cost per emergency visit escalates annually, emphasizing the need for more suitable care delivery.^4-6

Specialized geriatric services (SGS) are defined as “a comprehensive, coordinated system of health services that assess, diagnose, and treat older adults living with complex health conditions.”⁷ Outpatient geriatric medicine clinics play a key role within SGS,⁸ allowing geriatricians and multidisciplinary teams to optimize care for older adults based on the principles of a comprehensive geriatric assessment (CGA).⁹ A referral system for outpatient geriatric services should meet several requirements to ensure effective and efficient access to care for older adults, including accessibility and comprehensiveness.^10,11 The CGA tackles important concerns impacting function, independence, and quality of life, such as chronic conditions and multimorbidity, changes to cognition, changes to mobility, frailty and polypharmacy.⁹ In the outpatient setting, the CGA has been shown to reduce functional decline and preserve health-related quality-of-life measures compared to usual care.^12,13 Pre-frail and frail older adults with multimorbidity and high healthcare-utilization particularly appear to benefit from outpatient geriatric assessment.¹⁴ Identifying older adults who may benefit from SGS to prevent functional decline and progression of frailty may not only prevent the need for expensive tertiary care but also help them maintain independence and preserve higher quality of life. However, there are barriers to accessing appropriate and timely care of this nature.¹⁵

There are only 0.57 geriatric medicine specialists per 10 000 older adults in Canada, which is less than half the estimated 1.25 per 10 000 required to serve the population.¹⁶ In 2022, the average wait time for any specialist was just shy of 7 months.¹⁷ This is more than double the 3 months a study has shown Canadians perceive as the maximum acceptable wait time for a specialist.¹⁸ Compared to seniors in 9 other developed countries, Canadian seniors were less likely to report seeing a specialist within 4 weeks, and a comparison among the same group found that Canadian seniors are less satisfied with the quality of care they receive.¹⁹ Given the challenges posed by ever increasing cost of healthcare, limited physician and healthcare resources and decreased patient satisfaction, it is critical to prioritize the optimization of currently available services and resources.

In response to waitlists, a Geriatric Care Hub (Geri-Hub) initiative was created. Geri-Hub is a centralized intake system established within the University Health Network (UHN) and Sinai Health Systems (SHS) to coordinate outpatient and community-based services for older adults, aiming to connect them with the most appropriate care in a timely manner. Geri-Hub aims to alleviate wait times for older adults seeking specialized geriatric services by implementing a centralized intake system. This system streamlines the referral process, allowing for more efficient and timely access to care. By coordinating outpatient (e.g. Geriatric Medicine Clinics, Home Visiting Geriatric Team, Geriatric Psychiatry Clinics, Falls Prevention Clinics, Geriatric Day hospital and other Rehabilitation Services) and community-based services (Social Work, Home Safety Assessments, Private Support Workers etc.) within the 2 hospital systems, Geri-Hub facilitates quicker assessments and referrals to appropriate healthcare professionals and services. Additionally, by offering a range of services such as geriatric medicine clinics, geriatric psychiatry clinics, rehabilitation programs, and community outreach teams, Geri-Hub provides comprehensive care options that can address the diverse needs of older adults. This integrated approach helps reduce the time spent navigating multiple healthcare providers and services, ultimately leading to shorter wait times and expedited access to specialized geriatric care. This timely access to care is crucial for addressing health issues proactively and preventing further deterioration in older adults’ health. It has helped referrers to send in one referral form with the patient’s issues and this information is used by the service coordinator who knows what services are available to identify what is needed to best address the patient’s needs.

The Consolidated Framework for Implementation Research (CFIR) framework has been widely used in evaluating novel geriatric care models (eg,^20-23). The CFIR identifies factors shaping implementation success. Its comprehensive domains encompass intervention, settings, individuals, and implementation process.²⁴ This study, guided by CFIR 2.0,²⁵ interviewed diverse healthcare professionals across 2 settings to outline factors impacting timely access to specialized geriatric services like multi-disciplinary rehabilitation. The twofold aim was: (1) comparing barriers and facilitators for Geri-Hub’s implementation using CFIR, (2) leveraging CFIR constructs to differentiate factors influencing Geri-Hub’s implementation strength, aiding in understanding how this referral system improves geriatric care.

Methods

This qualitative research is a component of a larger initiative known as the "Geri-Hub project”. This study occurred over a year. The overarching goal of the Geri-Hub quality improvement (QI) project was to conduct coordinated and interdisciplinary projects across multiple sites to assess and enhance the processes involved in referral to the Geriatric Care Hub (Geri-Hub) services with utilization of QI and implementation science methodologies to improve access to care.

This study protocol underwent institutional review and was deemed excluded from research ethics under Quality Improvement exemption. (Clinical & Organizational Ethics Department, Quality Improvement Review Committee, QIRC Approval/REB Exemption #23-676).

Local setting and problem

UHN and SHS are 2 healthcare organizations affiliated with the University of Toronto. They share a group of 15 geriatric medicine specialists working at either sites’ outpatient geriatric medicine clinics. UHN and SHS have established a Geriatric Care Hub (Geri-Hub) with the aim of coordinating outpatient and community-based services to connect older adults to the most appropriate care for their needs, in a timely manner. These services include geriatric medicine clinics, geriatric psychiatry clinics, several rehabilitation programs and community outreach teams to support independent living, which collectively receive 180 to 200 referrals per month and serve around 1000 unique older adults every year. One of the main objectives of Geri-Hub is to create a centralized intake system to facilitate the delivery of services. This system facilitates the referral process, ensuring that individuals are connected with the most appropriate care providers and services, including geriatric medicine clinics, geriatric psychiatry clinics, rehabilitation programs, and community outreach teams, in a timely manner. Geri-Hub employs a multidisciplinary team of healthcare professionals, including geriatricians, nurses, social workers, and therapists. This team collaborates to provide comprehensive assessments, personalized treatment plans, and ongoing support for older adults. Geri-Hub facilitates communication and collaboration among different healthcare providers and organizations involved in the care of older adults. This coordination ensures continuity of care and seamless transitions between different levels of healthcare services.

To refer to Geri-Hub a patient needs to reside in a catchment area or receive majority of their care at UHN or SHS. They need to have an issue, geriatric syndrome, which can be addressed by one of the many services provided through Geri-Hub. Prior to Geri-Hub referrers often referred patients to duplicate and multiple services across both systems. The two-hospital system did not have a way to communicate about incoming referrals and were unaware who was being seen at what hospitals. The system prior to Geri-Hub resulted in longer wait times and unequal distribution of services, with some patient receiving multiple assessments. Geri-Hub ensures a central intake of all referrals to 2 different hospitals geriatric services, reducing wait times and duplication of services.

Study design

To identify barriers and facilitators influencing the implementation of Geri-Hub, we engaged in a multi-method²⁶ qualitative descriptive approach^27,28 to delve into the process of improving the Geri-Hub referral process. We focused on geriatricians, administrators, and frequent referrers for open-ended surveys or one-on-one interviews.

Data collection

The qualitative study opted for a convenience sampling approach. Participants were invited to participate in the interviews via email. Given the qualitative nature of the study, the depth of the data were deemed satisfactory.²⁹ Consequently, following Malterud et al’s²⁹ model and recommendations, we infer that our sample provides adequate information power.

We did not conduct a power analysis to determine the sample size. Instead, we relied on convenience sampling and the principle of data saturation to guide the number of participants. Data saturation was reached when no new themes or insights were emerging from the interviews, indicating that additional data collection would likely yield redundant information. This approach is common in qualitative research and ensures that the sample size is sufficient to address the research questions comprehensively.²⁹

Questionnaires scripts for semi-structured interviews and surveys are available in Appendix A. These were not externally validated as the purpose was for Quality Improvement and therefore tailored to specific local context and stakeholders as per QI best practices. Scripts were initially pilot-tested with 2 respondents, 1 geriatrician, and 1 administrator, representing 11% and 25% of these stakeholder group respectively.

The semi-structured interview guide, detailed in Table 1, adhered to CFIR domains. In this study, these domains were defined as: Intervention characteristics covered the Geri-Hub referral system; Inner setting involved healthcare professionals and administrative support; Outer setting focused on patients referred through Geri-Hub; Characteristics of involved individuals referred to the multidisciplinary team’s traits; Process addressed Geri-Hub’s operational implementation.

Table 1.

Classification of CFIR 2.0 constructs as facilitators or barriers to implementation of GeriHub centralized referral system as perceived by informed stakeholders.

	Facilitator	Barrier	Both facilitator & barrier	Neither facilitator nor barrier	Absent
Innovation domain	−Relative advantage	− Intervention source	− Innovation evidence base	− Doing	− Innovation trialability
	−Innovation adaptability	− Innovation complexity	− Innovation design
	−Innovation adaptability	− Innovation cost
Outer setting domain		− Critical incidents	− Local conditions		− Financing
		− Local attitudes	− Partnerships and connections		− External pressure
		− Policies and laws
Inner setting domain	− Structural characteristics	− Mission alignment	− Communications		− Relative priority
	− Tension for change	− Available resources	− Culture
	− Compatibility	− Access to knowledge & information	− Relational connections
	− Incentive systems	− Access to knowledge & information	− Relational connections
Individuals domain	− High-level, mid-level and opinion leaders
	− Implementation facilitators and leads
	− Implementation team members and other implementation support
Implementation process domain	− Teaming	− Planning
	− Assessing needs	− Tailoring strategies and engaging
	− Assessing context
	− Reflecting and evaluating

Clinician-researchers trained in research interviewing (GLF, MD, male and LR, MD, female) conducted interviews lasting 25 to 60 minutes, recorded and transcribed. Zoom facilitated remote interviews, while Survey Monkey collected surveys. Some participants were known to the research team prior to the study. All participants knew of the purpose of the study. Non-physician participants received a $25 gift card for their involvement in this research. The researchers took memos during and after the interviews.

Data Analysis

Data analysis in this study involved a template analysis form of thematic analysis of interview transcripts and survey results to identify themes related to facilitators and barriers in the Geri-Hub referral process.³⁰ Template analysis emphasizes hierarchical coding and provides a structured approach to analyzing textual data while allowing flexibility to cater to the specific needs of the study.³¹ Prior to commencing the coding process, the research collectively reviewed and discussed the CFIR 2.0 coding definitions,²⁵ inclusion and exclusion criteria to ensure a shared understanding of the codes. Next 2 coders, KMK and GLF, deductively and inductively coded a subset of the data to determine the coding scheme. Thus, the CFIR constructs served as the basis for our coding framework (deductive coding), although we also included open-codes in the coding template that were derived from a subset of data. Three authors (KMK, GLF, and EF) independently applied the coding template to the transcripts and surveys. All coding processes were carried out using NVivo 12 software.³² The research team maintained regular meetings to ensure coding consistency and address any challenging constructs. Upon coding completion, the coding team convened several times to identify preliminary themes and achieve a consensus on final factors and themes.

The coded text related to the CFIR factors then underwent a rating process based on the methodology recommended by Damschroder and Lowery.³³ This rating involved a deliberative consensus process between the research team, assigning valence (facilitator or barrier) to each construct derived based on the coded text.³³ Constructs were classified as either exhibiting insufficient data for pattern discernment (absent comments by respondents) or strongly influencing implementation (respondents providing general assertions about the construct showcasing positivity), as done in other research.³⁴ KMK, GLF, and EF conducted independent analyses of the coded data. Subsequently, all authors engaged in ongoing discussions and reviews of the findings until a consensus was established regarding the ratings within the CFIR constructs.

Results

Nine geriatricians and 4 clinic administrators took part in semi-structured interviews. Interviews lasted approximately 60 minutes. Thirteen healthcare providers who were identified as frequent referrers of Geri-Hub completed surveys. In this section, we provide examples of findings that emerged from our analysis related to each construct of the CFIR, or note if it was absent. We summarize our findings as facilitators and/or barriers with quotations from our participants.

Innovation domain

The Geri-Hub was recognized as a valuable way to streamline the referral process for various services, yet its successful implementation relied heavily on technology adoption across multiple electronic health systems. This involved a robust strategy to navigate potential technological obstacles and ensure stringent data security and privacy measures crucial for electronic patient data exchange.

Intervention source (barrier)

Participants highlighted that as Geri-Hub is a technology-driven intervention, might face challenges in sourcing consistent technological support across different healthcare systems, affecting its reliability and consistency. Participants noted that the Geri-Hub QI study offered a critical exploration of the intricate challenges related to the seamless exchange of information and the coordination among the numerous healthcare professionals involved in the referral process. One geriatrician noted “It’s like no one talks to anyone, we need to just be able to talk to a bunch of [providers] at once.”—GeriatricianID3.

Innovation evidence-base (facilitator and barrier)

Participants noted that supporting the effectiveness of Geri-Hub might be influenced by barriers like incomplete referral forms and receiving referral information via multiple avenues, all impacting the quality of data generated by the intervention for triage. All stakeholder interviews revealed that the triage process was central to the success of the referral system. One nurse noted “If the referral forms are filled out and filled out properly, it works great because you kind of get a sense in terms of like the information that you get with the patient may need, and going over to chart. However, when it doesn’t work, the whole Geri-Hub won’t work”—AdministratorID1.

Relative advantage (facilitator)

Participants revealed the need for a balance between accommodating specific referral requests and ensuring that patients receive the most appropriate care. Geri-Hub was perceived as a facilitator in streamlining the process and reducing administrative burdens while considering these contrasting demands. Participants described wanting clear information about the process used for accepting or rejecting referrals and how these criteria influenced the quality-of-care patients receive. Specifically, participants noted that Geri-Hub should have the goal of providing clinic administrators and physicians with the tools and resources they need to access and communicate referral information effectively. By offering real-time access to comprehensive patient information to inform decision-making, geriatricians perceived that Geri-Hub could empower a range of healthcare providers to make informed, data-driven decisions, that in turn, lead to more effective and personalized care.

One participant shared “I hope that as admins, we are able to access any information about any referral and be able to tell people who call at what point of triage the referral is, if they’re accepted or rejected, just having that information accessible.”—AdministratorID3.

Innovation adaptability (facilitator)

Geri-Hub was perceived by participants as flexibly incorporating new processes (new referral pathways) over the period of implementation.

Innovation trialability (absent)

Participants did not specifically address Geri-Hub’s trialability; however, it’s worth noting that participants acknowledged that the current stage of Geri-Hub resembles a pilot phase in its initial implementation and desired that Geri-Hub be sustained for a longer period of time. One participant exclaimed: “I think it’s perfect and it has to continue. I don’t know that I can say right now that is can’t be better, but as I suspected keeping [Geri-Hub] running for longer will be better fair enough.”—GeriatricianID1.

Innovation complexity (barrier)

Participants noted that Geri-Hub’s success is influenced by multiple stakeholders, workflows and technological aspects across organizations. Thus, devising strategies that account for the nuances and intricacies inherent in implementing and sustaining a referral system across organizations with various processes was seen as a barrier.

Innovation design (facilitator and barrier)

Geri-Hub’s design quality includes its ability to streamline referrals, but extensive data collection demands could potentially impact its attractiveness and user-friendliness. Opinions differed on whether the redirection process was transparent and whether communicating to referring providers should be the role of clinic administrators. Notably, participants highlighted need for data management processes, documenting reasons for redirection and outcomes of referrals. One participant shared:

“I appreciate that [service coordinator] have a better understanding of the services and may be able to recommend something more appropriate. (However) if a patient has been referred to a specific program for a specific indication/reason, it would be ideal for that request to be accommodated without redirection unless an email/conversation was had with the referring physician indicating the reason for redirecting to a different service.”—ReferrerID3.

Innovation cost (barrier)

Concerns were raised about sustainability and spread of Geri-Hub due to limited financial support. Salary support for new triage coordinator role was acknowledged by team leaders. Some participants commented that the hospital systems should commit to funding projects such as Geri-Hub for long-term viability if proven success.

Outer setting domain

Critical incidents (barrier)

Participants highlighted various challenges and incidents that disrupt the effective implementation and delivery of the Geri-Hub innovation in the healthcare system. Specific incidents referred to be participants included a separate unchecked cache of referrals due to administrative turnover, and accumulation of referrals during COVID19-related service disruptions, however both preceded GeriHub implementation. Issues related to difficulties in triaging referrals, scheduling limitations leading to long wait times for urgent cases, inconsistencies in referral processes, and absence of consistent follow-up mechanisms post-assessment reflect critical incidents disrupting the seamless implementation and delivery of the Geri-Hub innovation. One participant shared “So there hasn’t been capacity to check in with the referring physician to clarify the reason for referral because there just is not time and the schedule is so limited.”—GeriatricianID1.

Local attitudes (barrier)

Participant interviews and surveys indicated beliefs around shared responsibility and the selection criteria for referred patients. One participant statement: “I always assume that if I’m referring to that program, they’re already involved with someone I know because we share care”—GeriatricianID3 reflected their belief in shared care and their belief in the program’s eligibility and possibly the worthiness of recipients being associated with a specific healthcare institution.

Participants spoke further regarding shared responsibility with apparent belief in the “necessity of stringent review processes to ensure the appropriateness of referrals”—GeriatricianID4. Reluctance to change these review processes stemmed from a perception that altering them could impact patient care negatively or disrupt established workflows.

Local conditions (facilitator and barrier)

Participants noted that the study offered a critical exploration of the challenges related to seamless exchange of information and coordination among numerous healthcare professionals involved in the referral process. Almost all participants noted that the current local conditions within Toronto’s healthcare system and Ontario more broadly, involves a fragmented infrastructure that serves as a significant barrier and a potential facilitator for implementing a new referral system. One geriatrician noted “One of the biggest issues we have is the fracturing, (. . .) like a fractured system, nobody knows what anyone else is doing.”—GeriatricianID3. This fragmentation was described as leading to inefficiencies and critical information gaps hindering continuity of care. The existing referral system used at Sinai Health had involved multiple separate systems for tracking referrals, supporting the need for the novel Geri-Hub process. Traditional referral systems were described as operating separately for various programs, such as falls prevention, geriatric psychiatry, and other clinical streams. The lack of centralization resulted in referrals being tracked across different platforms, leading to confusion and inefficiency. However, this fragmentation poses a challenge to Geri-Hub due to the inability to provide accurate and timely information to referring sources and consequently, may result in delays in the acceptance process.

Conversely, the anticipation of transitioning to the Geri-Hub system was met with optimism expecting that it would unify referral processes into a singular system, streamlining tracking, categorization, and communication. This transition was perceived as a potential solution to the current fragmentation. Therefore, while the current fragmented infrastructure acts as a barrier due to disparate systems and lack of centralization, the imminent transition to Geri-Hub presented an opportunity to address these challenges and streamline referral processes.

Partnerships and connections (facilitator and barrier)

Partnership and connections serve as both facilitators and barriers within the context of the proposed Geri-Hub referral system in geriatric medicine. Participants emphasized being well connected to numerous services within the Geri-Hub. However, participants also emphasized the pivotal role of effective tracking, communication, and the triage process within the context of their partnerships and other organizations in determining the success of such a system.

Participants questioned the extent to which clinic administrators can track the status of referrals and communicate information to patients and referring physicians. In discussing this concern, participants noted that effective tracking and communication would be essential for the referral process. One Geriatrician shared:

“I keep a very close eye like on my on my new referrals, because I like, I’m just like a bit [detail-oriented] like that. So, I wasn’t sure if someone else is also going to be tracking those things, and I don’t know if they can so then I have to track and then like, ask the admins if the information is probably coming”—GeriatricianID3.

Policies and laws (barrier)

Participants noted aspects related to policies and laws that act as barriers to the implementation of the Geri-Hub, particularly in the context of referrals and triaging processes in geriatric medicine. Both geriatricians and clinic administrators noted that the absence of defined policies within the usage of Geri-Hub to discern urgent from non-urgent cases, coupled with minimal clinical review of consults, posed challenges in prioritizing referrals. Moreover, some participants implied legal barriers related to inadequate EMR integration and communication among healthcare providers in discussing barriers concerning data sharing, privacy, and interoperability.

Financing (absent)

Participants did not discuss financing explicitly. However, one participant discussed how the intervention’s effect on shifting healthcare resources may be judged based on alignment with institutional goals (see Mission Alignment construct).

External pressure (absent)

Participant responses did not touch upon external pressures like societal pressure, market pressure, or performance-measurement pressure. Therefore, participant responses do not directly align with the External Pressure construct.

Inner setting domain

Structural characteristics (facilitator)

Participants noted that effective coordination in healthcare settings heavily relies on the easy access to essential data. In cases where accessibility is limited or complex, the potential for miscommunication and coordination breakdowns increases and thus, Geri-Hub was perceived as having the infrastructure to allow for the quick review of information streamlining of the availability of information to ensure that all healthcare professionals are well-informed and equipped to contribute to the patient’s care journey effectively. Numerous participants referred to Geri-Hub as “a unifying hub”—GeriatricianID5 where geriatric care professionals can easily access essential patient information, medical histories, ongoing treatments, and other critical data points.

Relational connections (facilitator and barrier)

By creating a consistent and standardized approach amongst professional colleagues, participants felt that Geri-Hub would reduce variability, minimize errors, and ensure that patients receive consistent care, regardless of the provider or institution they encounter. Moreover, by simplifying workflows and empowering service coordinators to support decision-making and handle inquiries, geriatricians expressed a sense of relief. They felt that the platform allowed them to redirect their focus from administrative tasks toward more attentive patient care.

However, some noted that healthcare professionals and staff may be accustomed to their current workflows for referrals and may resist adopting the process of Geri-Hub that may in turn hinder relational connectiveness. Overcoming this resistance was described as necessitating effective change management strategies, open communication, and strong leadership support. One participant expressed, “like everything it comes down to change management.”—GeriatricianID1.

Communications (facilitator and barrier)

Transcripts revealed the pivotal influence of communication on the implementation of Geri-Hub in healthcare, where communication was interpreted to act as both a facilitator and a barrier, profoundly shaping operational efficiency. Participants highlighted that open communication is needed to support Geri-Hub. One administrator shared: “I think there has to be. . . really good communication between the two sites. . . making sure that everyone is included in the communication.”—AdministratorID2.

Participants shared incidences of effective communication demonstrated its facilitative role, such as the emphasis on centralizing referral systems to allocate appropriate referrals and prevent redundant clinical work: “So everything being centralized so that appropriate referrals go to the right places. It avoids the duplication of services and communication.”—Geriatrician ID5. Conversely, some stories by participants underscored communication barriers encountered in the implementation process. Misinterpretation of roles, as exemplified by redundant triaging steps undertaken by some geriatricians pointed to the need for clearer communication and role definitions within the system. The same geriatrician stated “I find it kind of interesting that I need to sometimes speak with [Geriatrician Name] who has to determine if it’s appropriate. . . I feel like their time is valuable far more valuable than being a triager and me needing to constantly speak with them”—GeriatricianID5.

Culture (facilitator and barrier)

Participants described that their organization had a culture of collaboration between medical professionals and administrators. This culture supported the implementation of Geri-Hub as enhancing both geriatrician and clinic administrator roles with partnership in care. Clinic administrators were described as pivotal in the referral process, acting as a bridge between referring physicians, patients, and the internal healthcare system.

Tension for change (facilitator)

Participants expressed optimism and support for Geri-Hub implementation due to their observations of the current referral system’s fragmentation and inadequate referrals. They highlighted how these shortcomings fueled their belief in the potential of Geri-Hub to streamline processes and ensure appropriate referrals, fostering a sense of hope and encouragement for its successful implementation.

Compatibility (facilitator)

Participants spoke of how Geri-Hub integration aligned with existing priorities. In particular, clinic administrators expressed that Geri-Hub’s implementation might decrease referral volumes by redirecting patients to more suitable services within or outside the catchment area, allowing them to be more efficient with their job.

For sure this works with my workflow. There’s another thing as well where patients may be within the catchment but they’re closer to another clinic, so that’s the thing Geri-Hub can maybe help me do fast so, should we accept them here because they’re within catchment or should we redirect them to somewhere else where they’re a lot closer. That would make my life easier"—AdministratorID2.

Relative priority (absent)

No participants spoke about the relative priority Geri-Hub had in relation to other initiatives at their organization.

Incentive systems (facilitator)

Multiple participants acknowledged that they received emails or verbal conversations from leadership of thanks for their support of Geri-Hub and sharing their feedback throughout the implementation of Geri-Hub. This recognition was seen as a significant motivator for boosting involvement in supporting Geri-Hub throughout the quality improvement period.

Mission alignment (barrier)

Participants noted that they had to use the Geri-Hub platform. However, many questioned about where improvement in referrals fit within the broader objectives of the organizations. Moreover, participants noted that each organization may have different goals. One administrator noted:

“ The way our system works right now. . . every hospital has their own kind of program mission. . . that’s a huge disadvantage. In Toronto, rehab side is we. . . can really determine what our wait times are and so maybe that is the goal. . . No one says. . . ‘Oh, you guys have decreased the cost. . . Now we’ll provide you with some other services to make up for that. But maybe for others decreased wait is like not the goal for Geri-Hub.’” – AdministratorID2

Available resources (barrier)

Administrators, specifically, pointed out restricted availability of resources like dedicated time for learning more about Geri-Hub and the long term goal, personnel for aiding with onboarding, and inadequate training as obstacles hindering Geri-Hub. (Additional themes on funding resources were described separately under the Innovation Cost construct above.)

Access to knowledge & information (barrier)

Participants spoke about the lack of educational resources aimed at educating both healthcare professionals and patients about the limitations of the referral system, as well as the existing services. One administrative assistant shared “We need training like where we can actually run these groups and see these patients. And know what is happening by having a [education] session”—AdministratorID1.

Individuals domain

High-level, mid-level and opinion leaders (facilitator)

Participants described that the involvement and alignment of various leaders in advocating for the successful uptake of Geri-Hub and the commitment to delivering timely referrals to patient populations significantly facilitated the successful implementation. Hospital-level leaders, managers and staff were involved in the oversight, advocacy, and direct implementation of Geri-Hub and expressed a shared dedication to improving administrative processes at their organization. Involvement of experienced quality innovators with existing social capital (through a history of leading successful QI initiatives) as well as newer healthcare providers was seen as helpful in creating buy-in from team members diverse in experience and roles.

Implementation facilitators and leads (facilitator)

Implementation leaders and facilitators were overwhelmingly described in positive terms by participants. Participants frequently highlighted the significance of collaborating with key leads and facilitators, attributing to them the uptake of Geri-Hub by championing their use by frequently calling them “leaders.”

Implementation team members and other implementation support (facilitator)

Clinic administrators in champion and frontline implementation positions viewed their involvement in the Geri-Hub as helping to assert their importance in the referral process. This alignment between the culture of the organization, the workflow of clinic administrators and the organization’s drive to improve the referral process played a pivotal role in securing and sustaining implementation and use of Geri-Hub. One clinic administrator shared:

"So right now, I triage all. . . I do the admin portion of triage for all the referrals for the whole program and just took up Geri-Hub because I have to in my role. . . If we need extra info if they’re in our attachment or anything, I’m that role and I want to help.” – AdministratorID4

Geriatrician participants noted that their use of Geri-Hub was a potent force for motivating clinic administrative staff in accepting Geri-Hub.

Implementation and quality improvement process

Teaming (facilitator)

Numerous participants highlighted that the collaborative efforts of a multidisciplinary steering committee helped support Geri-Hub. Geri-Hub was conceptualized with various, organization and hospital leadership, providers, and staff leading to the execution of initiative. The prevailing sentiment resonates with a collective acknowledgment that “what you need to be effective” in healthcare innovation is the synergistic force generated by multidisciplinary teams.

Assessing needs (facilitator)

Throughout the interview transcripts, a recurring emphasis on the importance of Geri-Hub having met discerning needs emerged. Participants highlighted that by having alignment on the needs and patient population Geri-Hub would fill, they would support the uptake of Geri-Hub.

Multidisciplinary collaboration also facilitated the effective identification of needs pre-implementation of Geri-Hub as key stakeholders were able to collectively brainstorm the issues in current referral processes such as the need to “reduce paperwork, make reasons for acceptance or decline clear because of capacity issues due to compressed clinic hours and balancing new and existing patient care."—GeriatricianID9.

Assessing context (facilitator)

All participants expressed gratitude and support of the interviews and surveys being done as part of the quality improvement process. These assessments acted as facilitators by pinpointing crucial systemic bottlenecks and suggesting pathways for improvement.

Planning (barrier)

Participants expressed uncertainty about whom to direct questions about Geri-Hub to, as well as who to direct some referrals to, underlining the absence of defined pathways for transfers. Geriatrician participants underscored a lack of pre-emptive planning resulted in administrative issues, citing concerns about errors in appointment scheduling, administrative neglect for specific clinics, and discrepancies in handling new patient slots. This ambiguity in planning and scheduling was seen as causing a burden on healthcare professionals, resulting in a disjointed approach to handling referrals and in using Geri-Hub. One geriatrician shared:

“The challenge with that is, is that there just seems to be a lot of players involved, and, like, even myself when I am trying to refer a patient. . . I’m not necessarily sure who to make the referral to or what my role is suppose to be. I think the other thing is not always having like a clear idea of who exactly, like, what the referral criteria necessarily are. And not only for myself. . . but I guess for others. So I don’t know if any of that was preplanned” – GeriatricianID9

Further exacerbating this issue is the absence of clearly defined goals and measures for implementation success.

Tailoring strategies and engaging (barrier)

Participants outlined challenges in effectively tailoring strategies for the successful execution of the Geri-Hub, citing issues related to the current referral system’s complexity and lack of streamlined processes. The absence of personalized strategies amongst various organizations inhibits the efficient navigation of referrals, thereby hindering the implementation of this innovative hub.

Participants point out instances where stakeholders, including healthcare providers and trainees, exhibit limited knowledge about services offered by the Geri-Hub. This lack of awareness, coupled with insufficient engagement strategies, contributes to ineffective referrals and diminishes the potential impact of the Geri-Hub. A few participants noted the lack of patient and caregiver engagement as well as a barrier to Geri-Hub implementation.

Doing (neither facilitator nor barrier)

Participants described that the implementation of Geri-Hub occurred fairly quickly in one rapid implementation cycle. However, participants did not describe this as a barrier or facilitator.

Reflecting and evaluating (facilitator)

All participants described appreciating the opportunity to provide evaluative feedback and reflection throughout the quality improvement and implementation process. Participants felt that it allowed them to have their voices heard and for the program leaders to assess and act on implementation changes, as needed. They emphasized importance of checking in on balancing measures, such as changes in workload, to ensure that implemented processes are sustainable.

Discussion

The results of this study suggest that the Geri-Hub model offers readily identifiable benefits related to referring patients to various geriatric services. Participants spoke highly of the program, particularly how it would help streamline care coordination in the context of existing services. Administrators were enthusiastically engaged, and their instrumental role in triaging and decision-making was highlighted. However, feedback on the referral system suggests that there are resistances to adopting new processes, with need for seamless communication amongst providers and administrators across 2 different hospitals. Institutional inertia presents a formidable barrier to the adoption of new processes³⁵ within the referral system, as entrenched practices and protocols may resist change despite the introduction of innovative models like Geri-Hub. Staff reluctance to deviate from established workflows can impede the successful implementation of the Geri-Hub model, as healthcare professionals may be hesitant to embrace unfamiliar procedures or technologies, Participants in the focus group sessions clearly identified the utility of having a central technology for referrals, while also acknowledging frustrations with adopting technologies and need for safeguarding private information across care settings. Lastly, this study contributes to the broader field of implementation science and quality improvement by offering additional support that the CFIR can facilitate the effective delivery of novel referral processes and account for the rapid processes involved in quality improvement projects. Our approach involved the application of the CFIR to guide both data collection, improvement and analysis.

Engaging internal stakeholders and ensuring ongoing communication throughout implementation (CFIR: inner setting and individual domain) was crucial. Explicitly addressing the specific CFIR constructs highlighted as potential barriers by our stakeholders (Table 1) was crucial in troubleshooting implementation and creating greater buy-in from change resisters. Participants highlighted aspects such as ensuring adequate staff education on the system and processes to manage referrals across historically fragmented systems. Similar to our findings, the need for cross-sector and cross-institutional communication has been noted as one of the most documented barriers to timely referral of older adults in care.³⁶ Consistent communication is not only for ensuring that patients receive referrals to the care they require but also for maintaining the continuity of care amongst the various specialists, clinics, and facilities may be involved in a patient’s care journey.³⁷ As expressed by participants, without proper communication and coordination, there is a risk of fragmented care, duplication of efforts, and critical information gaps.³⁸ This inefficiency not only burdens older adults with additional appointments and costs but also places strain on healthcare resources, contributing to the rising costs of healthcare.³⁸ Difficulties in aligning communication and coordination practices across different healthcare providers and administrators pose a significant challenge to the seamless operation of Geri-Hub, as disjointed communication channels and conflicting priorities can impede the effective exchange of patient information and coordination of care.^39,40 In addressing these challenges, healthcare organizations and policymakers must invest in integrated systems and processes that facilitate seamless communication and coordination. This could involve implementation of electronic referral systems, standardized referral protocols, and tools for tracking the status of referrals by all members of the care team. These findings align with prior literature emphasizing the importance of transparent processes in referral systems, particularly in communicating redirection reasons to referring physicians and patients.^41-43 Our results suggest that these efforts require additional training and education for healthcare professionals to ensure they are proficient in using these standardized tools, aligned to their own approach to referral processes and patient needs.

Geri-Hub serves as a prime example of this approach, in that it employs tools like standardized referral protocols and a EMR system to ensure streamlined communication and coordination among various healthcare entities, reducing wait times for geriatric services and enhancing referral efficiency. However, while the adaptability of Geri-Hub was recognized, the study sheds light on the complexities arising from electronic data exchange, necessitating stringent data security measures and regulatory compliance alignment. Recent research exploring the implementation of referral systems across dual hospital settings with distinct EMRs echoed similar challenges.^42,44 In addition, the absence of explicit discussions surrounding the cost implications of implementing Geri-Hub underscores a critical gap in understanding the economic dimensions of such technological interventions, warranting further investigation in future studies on implementing novel referral systems. Healthcare organizations can explore interoperability solutions to facilitate seamless communication and data exchange between different EMR systems. Implementing standardized protocols and interfaces can help bridge the gap between disparate systems, enabling efficient referral processes across dual hospital settings.^45-47 The Quintuple Aim for health care improvement emphasizes the imperative to enhance outcomes associated with patient and population health, patient/caregiver experiences, provider experiences, and overall cost, while striving for health equity for socially marginalized populations including older adults. The interconnected nature of these elements in Canada signals the necessity for comprehensive assessments regarding the cost implications of technological interventions like Geri-Hub⁴⁸ (CFIR: Innovation Domain).

Another large facilitator identified was the fact that end-users of Geri-Hub had opportunities to provide feedback throughout the implementation process (CFIR: Process Domain), particularly through our interviews and surveys. Both geriatricians and administrator staff commented on the benefit of them sharing their opinions in making them feel valued. This aligned with the culture of support and collaboration, which is different from the fragmented healthcare system often described.⁴⁹ This finding speaks to the need to avoid medical hierarchy in program design and evaluation. Medical hierarchy refers to the hierarchical structure prevalent in healthcare systems where decision-making, responsibilities, and influence are often stratified based on professional roles, with physicians typically holding the highest authority followed by other healthcare professionals.⁵⁰ Thus, our study emphasizes the benefits of integrating diverse expertise and perspectives in program design.

Limitations

The limited size of the group of participants and the narrow geographical scope restrict the applicability of these findings to other settings.⁵¹ Our group is unique in that our geriatric services span 2 different hospitals not within the same network, housing a large group of 15 geriatricians. The questionnaires used in this study were not been externally validated as they were developed for quality improvement purposes and therefore related directly to specific local factors. Furthermore, this study exclusively delved into the perspectives of healthcare workers, and as such our analysis primarily considers system constraints that impact healthcare workers’ workflows. We do not have the valuable patient and caregiver experience. Patients and caregivers play a crucial role in the healthcare journey, and their perspectives can provide valuable insights into the effectiveness, accessibility, and acceptability of referral systems such as Geri-Hub. Future studies should aim to include patient and caregiver perspectives to capture their experiences, preferences, and any systemic barriers encountered in accessing specialized geriatric services. While they generally do not directly interact with this level of referral triage and management, their experiences can certainly directly influence the success of a novel model of service access, such as Geri-Hub. More qualitative research is necessary to gain a comprehensive understanding of how patients perceive referral systems, including any systemic barriers encountered and individual-level factors that affect ability to benefit from these services.

Conclusion

The increasing aging population in Canada poses a significant challenge to the healthcare system, demanding a more efficient referral systems to facilitate the multidisciplinary care of older adults. The Geri-Hub initiative, enhancing specialized geriatric service referrals, is a crucial step in this direction. Our study underscored vital implementation factors: seamless communication among healthcare stakeholders, refined triage methods ensuring care consistency, and addressing technological hurdles and change resistance. This highlighted the imperative for standardized, transparent processes ensuring appropriate patient care. While our insights are valuable, future research should encompass patient and caregiver viewpoints for a holistic understanding. Optimizing specialized geriatric referrals is pivotal for improved elder healthcare, advocating for timely access. Expanding centralized models like Geri-Hub across healthcare systems could help reduce wait times.

Supplemental Material

sj-docx-1-his-10.1177_11786329241274482 – Supplemental material for Evaluating the Barriers and Facilitators to Implementing a Novel Referral System for Outpatient Geriatric Services: The Geri-Hub Quality Improvement Initiative

Supplemental material, sj-docx-1-his-10.1177_11786329241274482 for Evaluating the Barriers and Facilitators to Implementing a Novel Referral System for Outpatient Geriatric Services: The Geri-Hub Quality Improvement Initiative by Guillaume Lim Fat, Kristina M Kokorelias, Erica Foronda, Bindhu Sadasivan and Lindy Romanovsky in Health Services Insights

Footnotes

Acknowledgements

We would like to extend our sincere gratitude to all the participants of this study. Their willingness to share their experiences and insights was invaluable and made this research possible. We are deeply appreciative of their time, effort, and contributions, which have significantly enriched our study.

Author Contributions

GLF, KMK, and LR conceived and designed the study. GLF and LR conducted the interviews and coded them with KMK and EF. All authors contributed to interpretation and prioritization of findings. GLF and KMK drafted the paper with LR. All authors contributed to writing the paper. All authors read and approved the final manuscript.

Declaration of conflicting interests:

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding:

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Sinai Health’s Healthy Aging and Geriatrics Program Research Fund. The funder had no role in the study preparation.

ORCID iD

Guillaume Lim Fat

Supplemental Material

Supplemental material for this article is available online.

References

Nuernberger

Atkinson

MacDonald

. Seniors in transition: Exploring pathways across the care continuum. Healthc Q. 2018;21:10-12.

Gibbard

. Meeting the care needs of Canada’s aging population. In: The Conference Board of Canada, Ottawa, ON, Canada; 2018:1-12.

Muratov

Lee

Holbrook

, et al. Incremental healthcare utilisation and costs among new senior high-cost users in Ontario, Canada: a retrospective matched cohort study. BMJ Open. 2019;9:e028637

de Oliveira

Cheng

Chan

, et al. High-cost patients and preventable spending: a population-based study. J Natl Compr Canc Netw. 2020;18:23-31.

Lau

. Correlates of Preventable Emergency Department Visits in Canada: Evidence From the Literature and the Canadian Community Health Survey. The University of Western Ontario; 2020.

Ohle

Perry

. Factors associated with choosing the emergency department as the primary access point to health care: a Canadian population cross-sectional study. Can J Emerg Med. 2017;19:271-276.

Kay

Morrison

Sidhu

Mahamud

. Coordinated Access to Specialized Geriatric Services: A Review of Evidence and Current Practice: Provincial Geriatrics Leadership Ontario. 2022. Accessed November 1, 2023. https://geriatricsontario.ca/wp-content/uploads/2022/02/Coordinated-Access-Final-Report-Version-March-22-2022.pdf

Cohen

Feussner

Weinberger

, et al. A controlled trial of inpatient and outpatient geriatric evaluation and management. New Engl J Med. 2002;346:905-912.

Bernabei

Venturiero

Tarsitani

Gambassi

. The comprehensive geriatric assessment: when, where, how. Crit Rev Oncol Hematol. 2000;33:45-56.

10.

Saxon

Gray

Oprescu

. Reducing geriatric outpatient waiting times: impact of an advanced health practitioner. Australas J Ageing. 2018;37:48-53.

11.

de Groot

Wattel

van Dam

, et al. Referral to geriatric rehabilitation: a scoping review of triage factors in acutely hospitalised older patients. Age Ageing. 2022;51:1-12. doi:10.1093/ageing/afac015

12.

Reuben

Frank

Hirsch

McGuigan

Maly

. A randomized clinical trial of outpatient comprehensive geriatric assessment coupled with an intervention to increase adherence to recommendations. J Am Geriatr Soc. 1999;47:269-276.

13.

Boult

Morishita

, et al. A randomized clinical trial of outpatient geriatric evaluation and management. J Am Geriatr Soc. 2001;49:351-359.

14.

Mazya

Garvin

Ekdahl

. Outpatient comprehensive geriatric assessment: effects on frailty and mortality in old people with multimorbidity and high health care utilization. Aging Clin Exp Res. 2019;31:519-525.

15.

Rockwood

Stadnyk

Carver

, et al. A clinimetric evaluation of specialized geriatric care for rural dwelling, frail older people. J Am Geriatr Soc. 2000;48:1080-1085.

16.

Basu

Cooper

Kay

, et al. Updated inventory and projected requirements for specialist physicians in geriatrics. Can Geriatr J. 2021;24:200-208.

17.

Moir

Barua

. Waiting your turn: wait times for health care in Canada, 2022 report. 2022.

18.

Liddy

Moroz

Affleck

, et al. How long are Canadians waiting to access specialty care?: retrospective study from a primary care perspective. Can Fam Physician. 2020;66:434-444.

19.

Canadian Institute for Health Information. How Canada Compares: Results From the Commonwealth Fund’s 2016 International Health Policy Survey of Adults in 11 Countries. Canadian Institute for Health Information; 2017.

20.

Peig

NNA

Ansryan

Mamais

, et al. Implementing safe™ care: evaluation of a geriatric model of care for real-world practice. Geriatr Nurs. 2021;42:167-172.

21.

Southerland

Hunold

Van Fossen

, et al. An implementation science approach to geriatric screening in an emergency department. J Am Geriatr Soc. 2022;70:178-187.

22.

Kokorelias

Grosse

Zhabokritsky

Walmsley

Sirisegaram

. Exploring implementation considerations for geriatric-HIV clinics: A secondary analysis from a scoping review on HIV models of geriatric care. HIV Med. 2024;25:38-59.

23.

Southerland

Gulker

Van Fossen

, et al. Implementation of geriatric screening in the emergency department using the consolidated framework for implementation research. Acad Emerg Med. 2023;30:1117-1128.

24.

Damschroder

Reardon

Lowery

. The consolidated framework for implementation research (CFIR). In: Nilsen

Birken

, eds. Handbook on Implementation Science. Edward Elgar Publishing; 2020:88-113.

25.

Damschroder

Reardon

Opra Widerquist

Lowery

. Conceptualizing outcomes for use with the consolidated framework for implementation research (CFIR): the CFIR Outcomes Addendum. Implement Sci. 2022;17:7-10.

26.

Roulston

. Preparing researchers to conduct interdisciplinary, multi-method qualitative research. Qual Rep. 2019;24:2259-2292.

27.

Lambert

. Qualitative descriptive research: an acceptable design. Pac Rim Int J Nurs Res. 2012;16:255-256.

28.

Colorafi

Evans

. Qualitative descriptive methods in health science research. HERD. 2016;9:16-25.

29.

Malterud

Siersma

Guassora

. Sample size in qualitative interview studies: guided by information power. Qual Health Res. 2016;26:1753-1760.

30.

Brooks

King

. Qualitative psychology in the real world: the utility of template analysis. In: British Psychological Society Annual Conference, 18–20 April 2012. Unpublished.

31.

Brooks

McCluskey

Turley

King

. The utility of template analysis in qualitative psychology research. Qual Res Psychol. 2015;12:202-222.

32.

Edhlund

McDougall

. NVivo 12 Essentials. Lulu.com; 2018.

33.

Damschroder

Lowery

. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implement Sci. 2013;8:1-17.

34.

Varsi

Ekstedt

Gammon

Ruland

. Using the consolidated framework for implementation research to identify barriers and facilitators for the implementation of an internet-based patient-provider communication service in five settings: a qualitative study. J Med Internet Res. 2015;17:e262.

35.

Genschel

. The dynamics of inertia: Institutional persistence and institutional change in telecommunications and health care. MPIfG Discussion Paper; 1995.

36.

Savoy

Khazvand

Mathew

, et al. Consultants’ and referrers’ perceived barriers to closing the cross-institutional referral loop, and perceived impact on clinical care. Int J Med Inform. 2023;180:e105265

37.

Andres

Spenceley

Cook

Wedel

Gelber

. Improving primary care continuity is about relationships. Can Fam Physician. 2016;62:116-NaN9, e57.

38.

Wust

Carayon

Werner

, et al. Older adult patients and care partners as knowledge brokers in fragmented health care. Hum Factors. 2024;66:713.

39.

Pourasghar

Kavakebi

Tabrizi

Mohammadi

. Barriers to communication and information exchange in patient transfer and its consequences. Glob J Health Sci. 2016;8:178.

40.

Rattray

Sico

Cox

, et al. Crossing the communication chasm: challenges and opportunities in transitions of care from the hospital to the primary care clinic. Jt Comm J Qual Patient Saf. 2017;43:127-137.

41.

Warren

Orr

. The potentials and challenges of electronic referrals in transforming healthcare. N Z Med J. 2014;127:111-118.

42.

Drewry

Yanguela

Khanna

, et al. A systematic review of electronic community resource referral systems. Am J Prev Med. 2023;65:1142-1152.

43.

Mohammed

Huebner

L-A

. Assessing patient satisfaction and experience with an electronic referral process. Qual Manag Health Care. 2020;29:20-29.

44.

Tuot

Leeds

Murphy

, et al. Facilitators and barriers to implementing electronic referral and/or consultation systems: a qualitative study of 16 health organizations. BMC Health Serv Res. 2015;15:568-610.

45.

Ndlovu

Mars

Scott

. Interoperability frameworks linking mHealth applications to electronic record systems. BMC Health Serv Res. 2021;21:459.

46.

Bhartiya

Mehrotra

. Challenges and recommendations to healthcare data exchange in an interoperable environment. Electron J Heal Inf. 2014;8:16.

47.

Iroju

Soriyan

Gambo

, et al. Interoperability in healthcare: benefits, challenges and resolutions. Int J Innov Appl Stud. 2013;3:262-270.

48.

Sikka

Morath

Leape

. The quadruple aim: care, health, cost and meaning in work. BMJ Qual Saf. 2015;24:608-610.

49.

Khalili

Hall

DeLuca

. Historical analysis of professionalism in western societies: implications for interprofessional education and collaborative practice. J Interprof Care. 2014;28:92-97.

50.

Liberatore

Nydick

. The analytic hierarchy process in medical and health care decision making: A literature review. Eur J Oper Res. 2008;189:194-207.

51.

Mayan

,ed. Essentials of Qualitative Inquiry. 2nd ed. Routledge; 2023.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB