Abstract
Evidence-based decision-making (EBDM)—using the best available evidence from multiple sources to make informed decisions—is critical to the successful functioning of any organization. Much of the literature on EBDM has focused on the decision-maker as an individual. Yet to be most efficient and effective, EBDM requires extensive communication with others. Our review of the body of research on EBDM shows that it is typical for several individuals and groups to contribute to evidence-based decisions. These varied participants provide the skills and knowledge needed to incorporate diverse types of evidence into the decision process. We have found that conversations and interactions through three distinct networks contribute to successful EBDM. These networks consist of decision-makers and their staff and either (a) researchers in a relevant field; (b) stakeholders in the community or organization; or (c) colleagues with similar responsibilities and challenges, which we call communities of practice. Building social connections with people in each of these networks enhances a person’s ability to make good decisions for an organization. It is therefore imperative that organizations engaging in EBDM have ongoing programs and policies geared toward creating and maintaining these three critical connections.
Let us tell you about Maya, who is fictional although representative of many people who have decision-making responsibilities in organizations. Maya is a rural community public health manager. When the first promising studies on COVID-19 vaccine effectiveness came out in 2020, she was tasked with designing a program that would lead to the vaccination of at least 80% of her community’s residents. Her challenges included finding optimal locations for COVID-19 vaccination sites and a safe and efficient way of delivering vaccines at those sites. Faced with a population that had limited access to both health care and the internet and that included demographic groups anticipated to be at least somewhat vaccine hesitant, Maya knew she needed to do more than just supply vaccines: She needed to understand the community’s concerns and any conditions affecting vaccine access and acceptance so she could respond constructively to those issues. At the same time, she needed to get up to speed on existing outreach and information campaigns and any details pertaining to their suitability for her community.
Maya did not want to rely solely on her own judgment to make these decisions. She wanted to build a program that was based on evidence. So she began a formal, stepwise process called evidence-based decision-making (EBDM) that involves the use of multiple data sources that have been vetted for quality. EBDM can be applied in a wide variety of contexts, from the criminal justice system to construction. It may be used by company managers, for example, to decide whether employee incentive programs or benefits such as paid time off are likely to improve morale, retention, or productivity. Some managers of grocery stores used EBDM to navigate the staffing shortages, supply chain problems, customer hesitancy, and safety concerns the COVID-19 pandemic caused.
To begin finding the information she needed to design an effective vaccination program, Maya dove into databases of relevant research, user statistics, and stakeholder surveys. But if Maya conducted her search for relevant data and evidence only on her computer and the internet, she would likely come up short. It is tough to be “evidence-based” on one’s own. Effective EBDM requires connecting with other people. In this article, we describe the kinds of collaborators who are likely to be most helpful and offer advice for how to work effectively with them in making evidence-based decisions.
Evidence for the Importance of Collaboration in EBDM
We determined that successful EBDM is a social endeavor after conducting a systematic review of articles on its implementation in organizations. We searched three databases (PubMed, Scopus, and ABI/INFORM) for “evidence-based decision making,” “evidence-based management,” “evidence-informed decision making,” or “evidence-informed management.” We found more than 5,000 articles published since 2000, which we narrowed down to 108 after excluding those that were not empirical or relevant to our implementation focus. The studies were conducted in settings as diverse as public health departments, hospitals, environmental quality facilities, and nongovernmental organizations (NGOs). 1 We evaluated the quality of the EBDM described in an article by the extent to which a decision-maker made use of four types of evidence—science (for example, research reports and review articles), organizational context (data relating to patients, clients, or the organization that can improve understanding of the circumstances in which a decision is to be implemented), stakeholder insights, and practitioner expertise (for example, a doctor or manager’s experience and judgment)—and undertook the six key phases of EBDM—ask, acquire, appraise, aggregate, apply, assess. 2 (See Figure 1 for a description of these steps).
Six phases of evidence-based decision-making
We extracted from these studies several factors that promote EBDM in organizational decisions, including an organizational climate emphasizing science over politics, educated decision-makers, and access to reader-friendly summaries and reviews. 2 But perhaps the most striking attribute of successful EBDM that emerged from our analysis was its collective nature. When EBDM is taught in professional settings or discussed in scholarly literature, the focus is typically on the actions of an individual decision-maker. But when we looked at the practice of EBDM in real organizations, what stood out most was the involvement of other people in the process of figuring out what evidence is needed, where to find it, and how to use it in the decision process.
The reasons to recruit others for EBDM are manifold. First, individuals typically do not have all the skills or resources needed to readily identify and access a full complement of evidence in a timely fashion. By consulting with others, however, individuals can learn how best to track down relevant evidence and make sense of it. In addition, conversations with others enable decision-makers to reduce biases in their thinking, unaware as human beings are of their own assumptions and false beliefs.3,4 A manager who assumes that only money can motivate improvements in employees’ behaviors and performance, for example, may learn about alternative strategies from colleagues who have had experience with the dysfunctional consequences that monetary incentives can have. Decision-makers also need others to help them critically appraise the trustworthiness of the evidence they access, evaluate its meaning, and assess its relevance to the problem they are trying to solve. Decision-makers benefit from others’ insights into how to best aggregate the evidence acquired and apply it in the decision process. And through interaction and feedback, decision-makers may discover key nuances in a problem or uncover a proposed solution’s drawbacks.
More specifically, we found that EBDM is enhanced by the interactions and conversations that occur in three networks, each consisting of the decision-maker (or the decision-maker and their staff) and either (a) researchers (internal or external to the organization), (b) stakeholders (people in an organization or community who are directly or indirectly affected by a decision), or (c) communities of practice (people who have job responsibilities similar to those of the decision-maker; see Table 1). In the next section, we describe these networks in detail. Creating and maintaining ongoing ties with people in these networks is key to an organization’s ability to engage in EBDM effectively.
Social networks for evidence-based decision-making
Successful evidence-based decision-making (EBDM) is a social endeavor that requires decision-makers (and their staff) to be a part of three types of networks. This table describes the benefits of each network, the EBDM phases in which it is most used, and strategies for creating it.
Key Networks for Successful EBDM
Researcher Networks
Researcher networks include academics who conduct rigorous studies and write research reviews at universities, governmental institutions (such as legislative bodies or the police), consulting firms, or NGOs (like the United Way or YMCA). These networks may be made up of the individual experts a decision-maker connects with on an ad hoc basis, people in organizations that have formally partnered with a decision-maker’s organization, or researchers for nonprofits dedicated to collecting and disseminating research (for example, the Institute for Work and Health or The Campbell Collaboration). A researcher network may even include experts within the decision-maker’s own organization.
Formal partnerships between a decision-maker’s organization and, say, a university or nonprofit think tank can facilitate EBDM by providing a built-in supply of individuals who can share relevant evidence and expertise, identifying studies that address a decision-maker’s specific needs and helping the decision-maker evaluate the quality of the evidence.5,6 Our review of studies suggests that EBDM is expedited when decision-makers tap researchers for pointer knowledge—insight into how best to find needed information—a term coined by the late Nobel Prize-winning polymath Herbert Simon. Pointer knowledge can lead a decision-maker to other experts or relevant publications.6–8
In some cases, a formal partnership may lead to a joint project, which can strengthen ties between decision-makers and subject-matter experts. This relationship, once established, may end up serving as an enduring natural bridge over which knowledge is transferred even after the joint project is completed. Moreover, strong ongoing relationships with researchers can, over time, boost a decision-maker’s skill and efficacy in appraising the quality of evidence so that the decision-maker can do so more often on their own. 9 For all these reasons, the researcher network is particularly useful in the ask, acquire, and appraise phases of EBDM, in which decision-makers and their staff seek to identify relevant evidence and evaluate its quality. In Maya’s case, to identify better scientific evidence on the best messaging approaches for different communities, Maya could email a researcher familiar with the scientific literature on community outreach whom she met at a conference. This expert may be able to direct Maya to useful studies or refer her to other knowledgeable academics, enabling Maya to quickly bring the most relevant information to bear on these decisions.
Stakeholder Networks
The stakeholders relevant to EBDM include people directly or indirectly affected by a decision. In Maya’s case, they are likely to include the community members Maya seeks to vaccinate as well as health care practitioners who would deliver the vaccines and local health officials who need to evaluate her plan. In other cases, stakeholders might include the staff or clients of an organization who will be affected by the decision. Research on EBDM has underscored the importance of obtaining data from stakeholders, such as information about their preferences, expectations, and values. 10
Stakeholders should be involved early in the EBDM process (in the Ask phase) because their feedback is particularly important in framing fundamental questions about the problems the decision-maker wants to solve. Stakeholder input is also important for aggregating evidence to make sense of the information from different sources, applying the evidence to a decision, and assessing the outcomes of the decision in a balanced and thoughtful way—that is, for the Aggregate, Apply, and Assess phases of EBDM. Our review suggests that to make a successful evidence-based decision, practitioners need to consult various stakeholders about their concerns, the outcomes that matter to them, their readiness for relevant technology, and the kinds of actions that are acceptable in their social circles. Maya, for example, could interview family doctors, nurses, or pharmacists to find out which groups of community members are most wary of vaccines and what obstacles need to be addressed by plans for vaccine delivery. 11 Moreover, in making decisions about vaccine messaging and delivery, Maya probably needs to consider what resources public health officials are able to allocate and which community leaders and local organizations may be in a position to convince people of the safety and benefits of the COVID-19 vaccine. To understand this context, Maya needs to speak with these stakeholders either individually or in groups (see the Building the Networks section). 12
Conversations with members of the community are particularly important for Maya’s decision, because their beliefs can sometimes clash with the scientific principles that tend to guide public health decisions. 13 For example, some communities see health matters as a family decision and not an individual one, and some see vaccination as governmental overreach. Involving stakeholders throughout the EBDM process increases the likelihood of bridging different belief systems, gaining acceptance of a decision, and increasing the effectiveness of any associated policy or program. 14
Communities of Practice Networks
As we have mentioned, successful EBDM implementations tend to involve linking decision-makers and their staff with a community of practice—that is, colleagues with similar responsibilities and challenges. Maya might, for example, benefit from having a network of rural public health managers who face challenges like hers. Other communities of practice might connect librarians, data retrieval specialists, statisticians, or program evaluators. Members of these communities can connect in ways that are informal and sporadic, such as when chatting at an industry dinner, interact online, or have a one-on-one get-together, or they can communicate when participating in regularly scheduled events, such as EBDM training sessions, continuing education presentations, or networking brown-bag lunches.
Members in these communities benefit from learning how others solve similar problems, from technical issues such as navigating a database to psychological hurdles like community resistance to a program.10,15 For example, Maya might connect with a public health manager who has had experience with vaccine hesitancy and who might be able to recommend strategies to overcome it and perhaps refer Maya to researchers who can provide more information.16–18 Colleagues who are also trained in EBDM are particularly valuable because they can help guide a decision-maker in that process, demonstrating how to frame the problem in an evidence-based way, recognize trustworthy information from multiple sources, and apply this information to decision-making.16,19,20
Building the Networks
Because of the importance of the three networks, organizations and individual decision-makers should devote resources to developing and strengthening them on an ongoing basis. Building these networks is as important to successful EBDM as is knowing how to search a research database.
Organizations or funding agencies can support researcher networks by funding and encouraging collaborative research projects involving both decision-makers and academics or by sponsoring joint workshops and seminars that bring them together. Organizations should also send decision-makers and their staff to relevant academic conferences, and decision-makers should introduce themselves to researchers during these events. In addition, organizations can hire knowledge brokers—people whose job it is to forge ties between their staff and academic researchers who work in fields relevant to the decisions staff need to make. 21
Strategies for building stakeholder networks center around promoting a dialogue between decision-makers and employees, clients, or members of a community. For example, decision-makers at an organization or agency can invite community leaders to planning meetings or hold public forums to provide information and get feedback from citizens. They can conduct focus groups to solicit feedback and opinions about a proposed program or policy. Decision-makers and their staff can also schedule conversations with community leaders to vet ideas. Mechanisms for getting ongoing feedback from stakeholders, such as regular community gatherings and focus groups, are important for decision-makers to establish if they are to remain responsive to stakeholder expectations, preferences, and values.
Communities of practice may coalesce around online communities dedicated to a field of practice, journal clubs, or informal networking among colleagues. Organizations can seed these communities by providing peer-learning and mentoring opportunities such as workshops or lunch-and-learn presentations on a topic of interest or focused on EBDM practices.8,10,22–24 At organization-wide events, such as presentations or brown-bag learning sessions, practitioners begin to recognize “who knows what” 25 and build cross-unit connections that facilitate EBDM. Organizations can also establish interdisciplinary task forces to complete projects that require them to gather and summarize evidence and make recommendations.5,26–28 The idea is to create a culture in which people invest in building connections and getting to know others in their field.
EBDM training sessions become a particularly useful way to build communities when participants follow up on the connections they make in these sessions. Providing training session participants with a contact list for all attendees facilitates additional get-togethers and creates a powerful lift for an organization’s EBDM capabilities. Professionals such as physicians and nurses are more likely to be persuaded by the ideas of members of their own profession than by those of people in other disciplines. 29 EBDM training is thus more effective when it includes several members of a profession from the same organization and follows up that training with efforts to foster the development of social networks and discussion groups among participants. 19
Investing in EBDM capacity can transform an organization’s decision-making culture. When people come together to practice EBDM, staff absorb the notion that decisions should be supported by evidence—in particular, multiple sources of evidence that have been vetted for quality. When such a culture exists, staff may routinely demand that claims be backed up with evidence, raising the chances that a decision will be carefully considered, thoughtfully administered, and result in a successful program that benefits stakeholders. What is more, when supervisors practice evidence-based management and are up front with staff about the rationale behind a decision or new policy, staff are more likely to trust that decision and leadership generally.
A Final Word
To develop a vaccine campaign, Maya spent several weeks studying the problem and exploring solutions with the help of her three networks. As she worked through the EBDM process, she laid out a logic model—a mapping of critical information about her proposed vaccine campaign and its intended effects (input → vaccine campaign → output). Maya’s model detailed the critical inputs to her vaccine program, including population characteristics; the needs of community subgroups; and available funds, facilities, and staffing. It outlined the core vaccination campaign, which comprised multiple reinforcing outreach activities, a well-defined vaccination-scheduling process, multiple sites for delivering shots in arms, and activities to support the community in the aftermath of vaccination. It also specified the campaign’s desired outputs: Near-term outcomes such as the percentage of the population served and longer term results such as the expected incidence of COVID-19 and its severity in the target population. Conversations with researchers helped Maya identify critical population information, and her interactions with stakeholders revealed subgroups with differing concerns. Working with public health officials and clinicians helped Maya detail the steps in the vaccination process and plan for follow-up activities. The logic model also drew Maya’s attention to additional questions to explore with her network contacts, including what supports the community might need after vaccination—information about how to manage side effects, for example.
Beyond promoting robust evidence-based decisions, ties linking decision-makers with researchers, stakeholders, and colleagues contribute to ongoing learning and an organization’s overall capacity for engaging in EBDM. And with each successful implementation of EBDM, that capacity grows as connections across the three networks strengthen in a feed-forward process. The upshot for leaders and decision-makers is clear: Start building connections now. Do not wait for the next crisis to take this critical step.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: H. J. Heinz II endowed chair held by the second author.