Seeing together: Lessons from the COVID-19 pandemic on understanding evidence

Abstract

Conversations about evidence have become much more personal and more divisive over the course of the COVID-19 pandemic. Using a metaphor to carefully consider all the different aspects of “seeing” evidence can remind us that assembling a complete picture of information on any topic is necessarily a communal effort, made more robust by actively seeking to learn about and mitigate our blind spots. An approach to evidence that is curious, humble, and seeks relationship and partnership with others can help us see more clearly and completely.

To work at this work alone is to fail. There is no help for it. Loneliness is its failure.

- Wendell Berry, “Healing” ¹

Introduction

As the Chief Medical Officer of Health for Alberta during the COVID-19 pandemic, reviewing and communicating about evidence was a daily part of my work. Over the past several years, I have seen how the pandemic exacerbated pre-existing trends of proliferation of misinformation (“when false information is shared, but no harm is meant” ² ), disinformation (“when false information is knowingly shared to cause harm” ² ), and mistrust of expertise.^2,3 In this context, how we define evidence and how and in what context it is created, interpreted, translated, and used matters more than ever. Polarizing debates about many of these things are common in public discourse, creating and reinforcing deep divides within our communities. The resulting lack of social cohesion is a public health problem. It is essential that we create environments that foster productive and respectful dialogue, seeking common ground and common definitions of evidence. This is particularly important when dealing with new challenges and wicked problems—those problems which lack agreement on both a definition and a solution. ⁴

Trying to “see” the larger picture of evidence gives us a simple metaphor to think about how we approach this topic. Although we tend to think of vision as a very independent and unbiased action, the reality is that what we see is impacted by such factors as the state and functioning of our nervous system, where we are located, and what lenses we use. Similar dynamics can be true of evidence, as outlined below.

Metaphor components

To start, it is an obvious fact that we see differently with one eye than we do with both eyes together. Binocular vision provides a more robust three-dimensional image. The scientific method is only one way of understanding our world and can be enriched by approaches that bring in new viewpoints such as Two-Eyed Seeing methodology. This involves “…learning to see from one eye with the strengths of Indigenous knowledges and ways of knowing, and from the other eye with the strengths of Western knowledges and ways of knowing, and to using both these eyes together, for the benefit of all,” ⁵ emphasizing the importance and power of “bringing together Indigenous and mainstream knowledges and ways of knowing.” ⁵ At the same time, expertise, experience, and methodological rigour are critical elements of seeking knowledge within any field, including acknowledging the rigour in Indigenous science. Knowing simultaneously that Western science is imperfect and Western science is valuable requires holding two truths—a task more difficult when dichotomous thinking seemed to be an increasingly common way of seeing the world through the pandemic.

A single study on a topic can also act as a monocular view—each individual study is less important than what the collected body of evidence shows. For example, in COVID-19, there were public debates about the use of ivermectin as a treatment modality based on individual studies showing benefit, but when the totality of evidence is brought together, it shows no evidence of benefit ⁶ with the possible exception of countries with a high prevalence of the parasite Strongyloides stercoralis, for which ivermectin is known to be an effective treatment, ⁷ in which case the medication may have been treating the parasite rather than the SARS-CoV-2 virus. Each study on a topic needs to be examined through multiple lenses within the context of cumulative evidence in the field.

Not only is it important to look at the totality of evidence together to answer a question, it is also important to be clear about what the question actually is, and if studies being pooled together asked the same question. In other words, are both “eyes” even looking at the same thing? Using “PICO” (Population, Intervention, Control, and Outcome), ⁸ evidence-based medicine’s acronym for crafting a research question, can systematize this. For example, the question: “Do masks work?” is unanswerable. Instead, a more precise question is needed, considering questions like: Within what population? What kind of face coverings? Is the intervention a policy, provision of masks, or something else? What control group is being used as a comparison? What is the outcome of interest? Is the outcome being measured at a population level or individual level? If lab testing is involved, are molecular, antigen, or serologic methods used? How are the potential harms of the intervention being identified and measured? A more answerable question might be: “Does a policy requiring mandatory universal use of medical masks in acute care settings reduce lab-confirmed COVID-19 outbreaks in inpatients compared with acute care settings with similar characteristics that do not have mandatory universal use?” Of course, well-designed and high-quality studies are difficult to implement during an emergency response, and there is also an ethical challenge of determining when the available evidence is considered to be so robust in an area as to make further studies inappropriate. As we recover from the pandemic, thinking about how our health systems can be structured and resourced to build in ongoing evaluation and research will enable our systems to be more responsive to future emerging health threats.

Even if both “eyes” are looking in the same direction, our ability to see clearly relies on our ability to focus—for example, I wear glasses so my vision is blurry without specific corrective lenses. Similarly, if the methods used to answer a particular question are not robust and appropriate, the results of that study will not give a clear picture of the phenomenon. All scientific research contains some uncertainty and vigorous debate about methods and outcomes is part of the scientific process, however, a key hallmark of the scientific method is reproducibility of results. ⁹ An example from the COVID-19 pandemic that shows the misleading impact of flawed methodology is the early report of hydroxychloroquine effectiveness for treatment, followed by many high-quality trials that showed no benefit.^10,11 Understanding the appropriateness and quality of methods to answer a particular question requires expertise, and robust, respectful discussion and debate.

Perhaps we can achieve binocular vision and have both “eyes” in clear focus on the same question, but another important consideration is where we are located. In a concrete sense, it is obvious we can only see part of a view from one location. In a metaphorical sense, it is also true that not only does my social location as a white, cis-gendered, able-bodied woman of northern European ancestry influence my perspective, but my disciplinary background in biomedical sciences also shapes my thinking. It should be acknowledged here that this metaphor itself contains an inherent bias, excluding people who are blind. Despite the Western scientific goal of seeking unbiased results, framing research questions, choosing methods, and interpretation of results are inherently influenced by the unconscious biases of those involved in the research. Seeking evidence for action includes learning about and acknowledging our personal and systemic biases, and active relationship and partnership building with diverse communities to understand their perspectives and interests. Two-Eyed Seeing methodology, mentioned above, is one example of how to do this with rigour. In addition, it is important to recognize that globally, the COVID-19 pandemic response disproportionately emphasized biomedical science. ¹² This reveals a need and opportunity for improved interdisciplinarity and integration of different ways of knowing going forward, creating permanent positions for those who can bring the richness of Indigenous wisdom, social sciences and other expertise to the work. No single person can “see” the totality of relevant evidence on a topic—we need to work with others to understand a fuller picture.

There is also relevant information beyond innate human ability to see. The ability of some animals and insects to see infrared or ultraviolet light reminds us of perspectives beyond the human. What evidence do we have of the outcomes of our decisions on our non-human relations? For example, we now know that the SARS-CoV-2 virus can affect many different animals, and considering this kind of evidence informs our responses. Our processes around evidence need to include this explicit consideration of broader issues.

Thinking about this metaphor of vision applied to evidence during the COVID-19 pandemic would not be complete without considering what happens when we are bombarded with high-volume, rapidly changing visual inputs. When this happens, we can experience inattentional blindness to content on which we are not specifically focusing. ¹³ In the past few years, we have experienced an infodemic (“an overabundance of information—some accurate and some not—occurring during an epidemic” ¹⁴ ), with the volume of available information far exceeding any individual’s capacity to absorb and process. ¹⁵ Working together across multiple fields of expertise, and with multiple people, is necessary to assemble a bigger picture. For example, in the response to COVID-19, the work of the Alberta Health Services Scientific Advisory Group was invaluable in compiling, analyzing, and synthesizing the current state of the evidence on a variety of important topics.

In addition to high volumes of information, spread of mis- and disinformation has caused challenges. From a visual metaphor perspective, anyone who has spent time watching shapes in the clouds knows from experience that human beings are hard-wired to see patterns and assign meaning to our sensory inputs. Pattern recognition does not, however, mean that there is actually a face in the clouds. Observing patterns and generating hypotheses is an important part of the scientific method, but the testing of hypotheses, and the rigour and appropriateness of the methods used, is essential to determining which hypotheses to reject. Unfortunately, through the COVID-19 pandemic, we saw highly active spread of false information, some of it with an intent to mislead. ¹⁶ We need to better understand how information travels and how it shapes behaviours, an area where social science has much to offer. The emerging field of infodemiology (“the science of distribution and determinants of information in an electronic medium, specifically the Internet, with the ultimate aim to inform public health and public policy” ¹⁷ ) offers another approach to better understand this. In addition, actions like building trusting relationships with key communities and forming partnerships across multiple sectors are important ways to enhance effective sharing of reliable evidence.^2,14,18,19 This can also enable a better understanding and application of what matters to communities in the generation, interpretation and sharing of evidence.

Our nervous system also influences our vision. When we feel acutely threatened, one “fight or flight” impact enhances narrow focus on what is in front of us, but reduces our peripheral vision. This seems to be mirrored in highly polarized public debate on certain COVID-19 topics. A belief about a polarizing topic has, in some contexts, become a marker for belonging to or exclusion from a group. Staying mindful of our own responses can help us make decisions about how and when to engage on a topic. Does a discussion trigger a feeling of anger? Have we ever used words like “stupid” to describe a person who expresses a different conclusion rather than addressing the conclusion? Awareness of how both we and those with whom we are engaged are feeling, helps us leverage openness and curiosity to broaden the field of vision. Labelling people or groups who have come to different conclusions does not enable productive conversations about evidence. It is important to be clear that this awareness of emotional states does not imply that all conclusions are equally founded on good evidence. As mentioned earlier, disciplinary expertise and methodological rigour will affect the quality of the evidence generated and synthesized in a particular discipline. Sometimes the evidence is emerging and not yet clear. We are best served by limiting “us vs. them” thinking in favour of respectful dialogue and curiosity about methods. An example with the potential for “us vs. them” rhetoric is a 2023 Cochrane review, ²⁰ of which the Editor-in-Chief of the Cochrane Library stated: “Many commentators have claimed that a recently-updated Cochrane Review shows that ‘masks don't work’, which is an inaccurate and misleading interpretation. It would be accurate to say that the review examined whether interventions to promote mask wearing help to slow the spread of respiratory viruses, and that the results were inconclusive.” ²¹ It should be noted that observational studies were excluded from this review, and other summaries of evidence can also inform the discussion on masking. ²² The use of face masks became a polarizing issue during the COVID-19 pandemic, which means that discussions about evidence on this topic can be very challenging. Conversations on polarizing topics such as this can be made more productive by intentionally staying curious.

Recommendations

So, how can leaders in healthcare incorporate some of these lessons in service of evidence-informed leadership? This can happen at both an individual and an organizational level. For leaders, I believe that curiosity, humility and relationship are vital to see the broadest possible view of relevant evidence. We can remain curious about the totality of evidence on a topic, the relevance of available evidence to the particular question at hand, and the methodology used to generate evidence. We can stay humble, seeking to better understand and then acknowledging our biases, knowing that we cannot see a complete picture of evidence by ourselves, and remembering that humans are only one part of a larger whole. We can create and strengthen relationships, refusing to engage in polarizing rhetoric and instead building bridges and finding common ground wherever possible. Doing this will alter the approach of leaders and also set an example and an expectation for others. Ensuring that leadership teams respectfully hold each other accountable to these goals can build capacity and support for individuals leading these changes. Some helpful resources include Brené Brown’s “Dare to Lead” ²³ and Joiner and Joseph’s “Leadership Agility.” ²⁴

From a systems perspective, organizationally supported partnerships are also essential. These partnerships have the most impact when founded on strong human to human relationships, and when they include an intention to learn from and with local communities and diverse populations. We shouldn’t wait for a crisis to set these partnerships up, but build sustainable processes and structures now that support long-term relationships and co-creation of programs, services and policy. To build effective, diverse partnerships and make sure our organizations can effectively improve the health of the populations they serve, we must also understand how dynamics such as systemic racism are operating and act to change these dynamics. ²⁵ Developing effective methods to counter misinformation and disinformation is another priority that needs dedicated effort. Actions such as building interdisciplinary evidence literacy in our teams, developing expertise in areas such as infodemiology, and improving sustainable infrastructure and partnerships for practice-informed research are crucial.

Conclusion

Seeing a more complete picture of evidence to inform our work in health systems requires us to seek diverse perspectives and expertise, to be mindful of the impact of our emotions and the emotions of others on what we perceive, and to build structures and processes that enable co-creation of evidence and hypothesis testing within our organizations. The better we are at doing all of this in our daily work, the better prepared we will be for any future crisis. We need each other to see most clearly and completely.