Abstract
This article explores innovative and novel research methods and adaptive approaches during the COVID-19 pandemic to examine visitor learning and proenvironmental behavior. We present a mixed methods study that used a methodological bricolage approach to field-based data collection. The pandemic limited our ability to carry out the original study design. Quickly pivoting, the study was adapted to an explanatory sequential design with a survey, an interpretive video, naturalistic observations, personal meaning maps, interviews and a new method: comprehension assessments. This resulted in data collection that maintained trustworthiness and rigor, while remaining flexible to changing protocols. This article contributes to the field of mixed methods research by demonstrating the application of methodological bricolage in visitor research during catastrophic social change.
The negative impacts of the COVID-19 virus are unparalleled in modern history, with consequences to our social, economic, and political systems (Gretzel et al., 2020). With lock-down and quarantine measures implemented globally in March 2020, the nature of human interactions changed immediately and dramatically. Within research contexts, this resulted in interrupted data collection as traditional forms of contact would put both researchers and participants at risk (Sy et al., 2020). Forced to find adaptive and innovative methodologies, scholars have found novel ways to conduct research to meet mandatory safety measures that are empathetic and reflective, recognizing the challenges faced during the pandemic. In a special issue call for novel mixed methods research during the COVID-19 pandemic, Fetters and Molina-Azorin (2020) explain as follows: The unprecedented COVID-19 pandemic calls for human ingenuity and innovation . . . The need for mixed methods research and the need for advances in innovation have never been greater. Researchers must seize this moment to use and create novel mixed methods methodologies for addressing the current crisis and to inform future cataclysmic and catastrophic social changes as well. (p. 285)
This article addresses the call for innovative and novel mixed methods research (Azzari & Baker, 2020; Dodds & Hess, 2020; Gretzel et al., 2020; Fell et al., 2020; Fetters & Molina-Azorin, 2020; Mars & Ball, 2016; McSweeney & Faust, 2019; Pauwels, 2010) through a discussion of how the pandemic required our research team to reevaluate and continuously adapt data collection plans of visitor learning and proenvironmental behavior change in a Canadian National Park between June and September 2020. This continual process of adaptation resulted in both novel and innovative approaches in mixed methods research. Innovative mixed methods research is a growing area of interest (Mayoh et al., 2012; Wheeldon, 2010), and specific to our interdisciplinary field, leisure scholars have increasingly advocated for methodological innovation (Johnson, 2014; Mars & Ball, 2016; McSweeney & Faust, 2019; Pauwels, 2010).
Study Overview: Becoming Bear Aware and Bison Wise
This study included quantitative and qualitative approaches and data integration to examine visitor learning and proenvironmental behavior change in Elk Island National Park, which is situated within the Beaver Hills Biosphere, a UNESCO Biosphere Reserve in Alberta, Canada. Interpretive programming (e.g., planned talks and guided experiential learning) for this project had originally been planned with the park interpretative team, but were cancelled due to the pandemic and the need to eliminate all in-person events. This global pandemic, combined with challenging weather conditions, limited our ability to use traditional forms of field data collection and to carry out the embedded experimental mixed methods design as planned. As a result, the study was transformed to an explanatory sequential design (Creswell & Creswell, 2018).
The adapted design began with quantitative data collection and analysis using an online survey, which featured an embedded video to replace the in-person interpretative programming. This was followed by an innovative comprehension assessment that adapted a form of naturalistic observations to collect both quantitative and qualitative data (See Appendix A). Near the end of the project, interpretation programs were permitted again (in limited capacity) and personal meaning maps and interviews were included as a final qualitative phase (all methods described in detail in the Methodology section). Participant recruitment strategies were also adapted. At the outset of the pandemic we planned to use a passive recruitment approach to reduce person-to-person contact and then modified this to an active recruitment approach when participation was low. We then combined these approaches and included social media recruitment to further increase participation. All data collection was contactless and mediated through use of a QR code on a postcard, which was passively handed out at the gate and left at campsites, or actively handed out to participants (by the Research Assistant [RA] using physical distancing strategies).
The purpose of this article is twofold: (1) to acknowledge that the pandemic has brought to light the need for adaptive frameworks and methodological bricoleurs; and (2) that this approach to research resulted in innovative and novel methods, which we argue enhanced the study through creativity and reflexivity while ensuring the safety of participants and the research team. Below we provide a review of literature, followed by an overview of the context, methodology, and COVID-19 methodological innovations, novel, and adaptive approaches. We define innovative methods as new methods that require an element of creativity inspired by the pandemic. Novel methods are simply new methods in the field, and we acknowledge that novelty and innovation are distinct in this context. Finally, we use the term adaptive approaches to describe elements of the study that required ongoing changes and reflection during the pandemic. We then present a reflective discussion that includes the benefits and limitations of the research process and methodology. The article concludes with recommendations for future mixed methods inquiries.
Literature Review
Mixed Methods Research in Leisure and Environmental Studies
Definitions of mixed methods research have evolved since its formal emergence in the 1980s (Creswell & Plano Clark, 2018; Johnson et al., 2007; Tashakkori & Creswell, 2007). This article follows the core characteristics defined by Creswell and Plano Clark (2018) that call for both quantitative and qualitative data to be collected, analyzed, and integrated in a logical research design framed by theory and philosophy. The integration of qualitative and quantitative methods and data allows us to study a complex research problem while harnessing the methodological strengths of different types of data collection and analysis (Creswell & Plano Clark, 2018).
In the fields of leisure, visitor, and environmental studies, these strengths can be used to address the intricacies of environmental education (Ardoin et al., 2013; Sosu et al., 2008), or interpretation of historic sites and parks (Farmer & Knapp, 2008; Schliephack et al., 2013). Novel contexts and situations call for innovative methodologies. Nontraditional research methods create work that is responsive to changing contexts or research gaps (McSweeney & Faust, 2019). Mixed methods have been employed to examine visitor learning at zoos (Bueddefeld & Van Winkle, 2017, 2018; Roe & McConney, 2014), investigate perceptions of sustainability (Santiago-Brown et al., 2015), and develop understandings about human–wildlife conflicts (Alexander & Draper, 2019; Wilson & Rose, 2019). Mixed methods research provides enriched answers to these complicated research questions.
Innovative, Novel, and Adaptive Approaches
Gretzel et al. (2020) argue that science should not only be bound by traditional analyses, “ . . . but should also aim at creating novel and original technical artefacts and frameworks with high practical relevance or high scientific impact (p. 198).” Researchers should immerse themselves in the context to develop an in-depth understanding of the phenomenon under study. They may also need to “think outside the typical research toolbox” and find innovative ways to approach research (Azzari & Baker, 2020, p. 103).
Within leisure studies, McSweeney and Faust (2019) describe becoming more adaptive and reflective through methodological bricolage. The word bricolage refers to a person “who makes use of the tools available to complete a task” (Kincheloe et al., 2011, p. 168). Bricolage is the combining of multiple perspectives and practices in order to improve the “rigor, breadth, complexity, richness and depth to an inquiry” (McSweeney & Faust, 2019, p. 343; Denzin & Lincoln, 1994). Researchers use accessible methods and empirical material and may incorporate new strategies as the research develops, or perhaps, as in this case, as the context changes. Bricolage allows mixed methods researchers to adapt different methods and data collection approaches in changing research contexts—such as a pandemic—and it allows the researcher to select the most appropriate method for each phase of the research (Sharp, 2019). By adding and creating new research tools, the researcher-as-bricoleur is reflexive and understands their work is shaped by themselves, the participants, and the setting (Denzin & Lincoln, 1994). Bricoleurs can be interpretive, narrative, theoretical, political, and methodological (Denzin, 2012; Denzin & Lincoln, 2008).
McSweeney and Faust (2019) and Pratt et al. (2020) focus on methodological bricolage. It is “an approach to research that seeks to be open to multiple perspectives, ways of knowing, power relations, and strives to adopt an interdisciplinary framework” (McSweeney & Faust, 2019, p. 343). It is contrasted with methodological templates because bricolage is a proactive approach to research and includes thinking about how to engage with the methods (Pratt et al., 2020). A methodological bricolage approach was followed in this study of visitor learning and behavior change in protected areas. The COVID-19 pandemic required a quick adaptation of research methods that would ensure rigor and produce high quality data for analysis.
Research Overview and Methodology
Context and Research Design
The Beaver Hills Biosphere includes several protected areas managed by different levels of government: the regionally-managed Strathcona Wilderness Centre, the provincially-managed (Alberta Parks) Miquelon Lake Provincial Park, and the federally-managed (Parks Canada) Elk Island National Park. Data collection was scheduled to take place in-person before and after interpretive experiences at each park where visitors would be provided with two forms of interpretation—one version would act as the experimental “treatment” intended to encourage proenvironmental behavior change. Reopening and safety strategies varied across the three governing bodies. Regional and provincial parks in Alberta began reopening in May with reduced capacity and limited group sizes for interpretive experiences to ensure physical distancing (Lawrence, 2020; Neustaeter, 2020). Parks Canada began a gradual reopening in June, and initially did not continue with in-person interpretive programs (Elk Island National Park, 2020; Gilligan, 2020). This created challenges for both the delivery of the learning experiences and the research methods. For this article, we focus on research conducted at Elk Island National Park because the park’s cancellation of in-person interpretive programs caused more substantial changes to our research project’s methods.
The overall goal of the project was to facilitate learning for behavior change regarding an environmental issue like wildlife safety. The principal investigator (PI) worked with Parks Canada interpretive staff to determine the highest priority items for the research. Human–wildlife conflict, specifically with black bears and bison, was identified as a major challenge due to a knowledge and awareness gap. For black bears, there was a lack of awareness about the increased prevalence of the animals in the park. A specific action (e.g., not stopping on roads, storing food properly) was needed to avoid habituation, which is a biological concept that describes wildlife becoming too comfortable with humans. For bison, visitors were not always aware of the distance they should maintain between themselves and the animal, nor how to judge this distance correctly. To address these safety issues, it was important visitors identify at least two ways to avoid habituating black bears in the park and learn and apply the thumb rule (to maintain safe distances from wildlife).
The original research plan consisted of an embedded experimental mixed methods study to evaluate visitor learning and behavior change in the Beaver Hills Biosphere, located in Alberta, Canada. Changes at the study site due to the COVID-19 pandemic created a need to modify and adapt the research design to align with physical distancing protocols. The study was changed to an explanatory sequential design (Creswell & Creswell, 2018) to include an online survey with an embedded interpretive video, a comprehension assessment and naturalistic observations, and, when COVID-19 restrictions relaxed and interpretation programming resumed, personal meaning maps. See Figure 1 for a flowchart visualization of the adaptive planning framework used for data collection during the pandemic.
Adaptive planning framework for data collection during COVID-19.
As part of the embedded experimental design, data collection at Elk Island National Park was planned to take place prior to and just after in-person interpretive programming. It was expected that weekend attendance to programming would be approximately 50 to 100 people. Quantitative data would be collected through surveys on tablets immediately before the interpretive program. Participants would be selected by random sampling and assigned to either a treatment or control group on alternating weeks. A subsample of approximately 20 to 30 visitors would participate in a personal meaning map and short interview immediately before and after the program in order to elicit a deeper understanding of the free choice learning experience (Bueddefeld & Van Winkle, 2017, 2018; Falk et al., 1998; Van Winkle & Falk, 2015). Personal meaning maps are similar to concept mapping, where a participant is asked to brainstorm what they know about a topic immediately before and after a learning experience (Falk et al., 1998; Van Winkle & Falk, 2015). This method is well established and has been rigorously tested as both a qualitative and quantitative method used to evaluate visitor learning (Falk & Dierking, 2000; Van Winkle & Falk, 2015). This embedded experimental mixed methods approach has been used successfully in several studies examining learning and behavior change in tourism and environmental education contexts (Bueddefeld & Van Winkle, 2017, 2018; Hughes, 2013). Since the purpose of this article is to discuss methodological innovations and adaptive approaches inspired by the pandemic, we do not discuss the details of this analysis nor the findings in this research in this article. However, Table 1 has been provided as a summary of the data sources and an overview of the analytical details.
Data Sources Table.
As with all studies that attempt to measure learning and behavior change, it is challenging to measure behavior change due to research capacity limitations. In particular, the time and cost of follow-up studies to assess postvisit behavior and the accuracy of self-reported measures are challenging (Hughes, 2011, 2013; Hughes et al., 2011). While intentions and attitudes can be reasonable alternative measures for when follow-up is not possible, in this particular context there was a known disconnect between knowledge and the needed application for behavior change (e.g., the “thumb rule”). Hence, the explanatory sequential design was adapted in order to capture additional data needed to understand not just what part of the interpretive intervention worked (quantitative), but what did not work and why (qualitative).
COVID-19 (Re)planning: Methodological Innovations, Novel, and Adaptive Approaches
In Alberta, Canada, the COVID-19 pandemic caused drastic social changes beginning in March 2020. Summers in Alberta are short, and so it was necessary to quickly adapt the research methods as data collection would need to be completed during warmer weather when parks see higher visitation (from June to August). Quickly pivoting to react to dynamic pandemic protocols, research plans became even more complex due to an unusually rainy and windy summer that resulted in highly variable visitor numbers and made field visits challenging.
A COVID-19 safety plan was implemented under direction from the University of Alberta research office. This was also approved through an ethics and safety protocol review by Elk Island National Park. The research assistant (RA) received safety training and was provided with personal protective equipment. Additional measures were taken, and the research tent was set up behind a picnic table to provide a physical barrier between participants and the RA. Each week, the PI and RA debriefed to discuss data collection and review safety protocols.
Given that all in-person interpretation was cancelled, the original mixed methods embedded experimental design would not be possible. An explanatory sequential design (Creswell & Plano Clark, 2018) was adopted and participants were recruited using a postcard distributed at the park. The research team used a novel interpretive video, embedded into the Qualtrics survey, followed by a comprehension assessment, as a way to capture pre-treatment and posttreatment quantitative data. Observations and knowledge testing data were recorded using the comprehension assessment. Quantitative data analysis was conducted immediately, but because of initial low recruitment (to be discussed later), analysis was ongoing throughout the study. In August, Parks Canada implemented a limited version of in-person programming and personal meaning maps were added as a qualitative data collection method. This final change allowed for the sequential design approach to be used, where the preliminary survey data could inform a qualitative analysis of the same interpretive messages in person. We discuss the novel and innovative methods and adaptive approaches below, but direct the reader’s attention to Table 2 for an evaluation of the quality of data collection methods. Within Table 2, we discuss the constructs of reliability, validity, richness, generalizability, and usability as related to the novel and innovative methods and adaptive sampling approaches.
Evaluation of the Quality of the Data Collection Methods.
Innovative Methods
Comprehension Assessments
Wildlife experts felt the “thumb rule” was often incorrectly applied by visitors, leading to closer encounters with bison (visitors should ensure their arm is fully extended and thumb is positioned upright, not sideways). The effectiveness of interpretation (now a video) was evaluated using a naturalistic observation process modeled from a previous study of mobile device use at festivals (Van Winkle et al., 2014; Van Winkle et al., 2016). An in-person comprehension assessment was developed as a direct measure of behavior change. Visitors were encouraged to visit a popular area of the park where the research team had a tent and table set up with materials to conduct the comprehension assessments after completing the survey. Successful completion of the task would result in obtaining a small incentive prize (approximate value was $5) as a thank you for their help with the research. All participants were also able to enter an online prize draw for one of two $100 gift cards. Using both an immediate incentive and a prize draw was an intentional effort to engage as many participants as possible.
As mentioned, the comprehension assessment was paired with naturalistic observation data recorded by the RA. The comprehension assessment and naturalistic observations provided additional insight and immediate feedback for participants and the researchers. For example, using the comprehension assessment approach, we could measure the efficacy of: (1) understanding the no-stopping rule for black bears, (2) being able to correctly identify at least two attractants, (3) correctly demonstrating the “thumb rule,” and (4) being able to correctly describe a safe distance at which to view bison (100 meters). As the RA was stationed at the comprehension assessment table, she was able to provide immediate feedback to participants and correct incorrect responses. This process allowed her to have casual conversations with participants and determine what aspects of the interpretation video were working, and what areas required further clarification. The comprehension assessment sheet also allowed her to ask questions in order to identify visitor motivations and record other relevant qualitative details (e.g., why she believed some participants did not demonstrate the correct responses). At the end of every day of data collection, the RA recorded her systematic naturalistic observations to provide further context and other relevant observations (e.g., viewing visitors teaching each other the “thumb rule” or practicing what they had learned).
Direct naturalistic observations were conducted by the RA during the comprehension assessment and throughout the day of data collection (e.g., including observations as she drove to the site past common areas to see bison). According to Kearns (2000), the purpose of observations is to provide additional context, complementary evidence, or to count particular phenomena. This research adapted naturalistic observations for comprehension assessment (quantitative) and additional context and complementary evidence (qualitative). Note, additional field notes were maintained throughout the research study as relating to the weather, current COVID-19 safety protocols and other details relevant to the study but not directly related to the data collection or analysis.
Novel Methods
Embedded Interpretive Video
In response to the COVID-19 pandemic, a novel approach, in the form of an interpretive video, was created to replace the on-site interpretation. The interpretive video was filmed by the Elk Island interpretive team and embedded into a single survey to test pretreatment and posttreatment measures of learning and behavior change. The use of a video within a survey as a replacement for the in-person interpretation was a novel approach to deliver environmental education materials and was intended as the experiment’s intervention. To our knowledge, no prior environmental education research examines the use of an interpretive video in lieu of in person interpretation. This also allowed for a paired-samples t-test with an immediate pre–post evaluation of the interpretation, which is otherwise very challenging to do with visitor research where the amount of time spent collecting on-site data must be limited (which also eliminated the need for a control group, as this allowed for the evaluation of all participant’s preinterpretation understanding and knowledge).
The PI advised a dialogic-based narrative approach (Williams et al., 2018) to video production that would incorporate elements of community-based social marketing (McKenzie-Mohr, 2000) and self-serving bias theory (Van Winkle & MacKay, 2008). This approach highlights both barriers and benefits relating to behavior change in visitors. The 5-minute video (https://youtu.be/N3HnV2797Ro) became the new form of interpretation and was intended to teach visitors about wildlife safety in the park. Conducted immediately after viewing, the survey provided researchers and their practitioner partners with invaluable insights into the efficacy of the video and an information transmission tool. This after-intervention assessment of interpretation efficacy is not uncommon in the environmental education field (e.g., Kim et al., 2011; Ren & Folta, 2016), but was novel in its use of a video embedded in the survey tool.
The video was uploaded to YouTube using a hidden link and embedded in a quantitative survey on the Qualtrics platform. This new design was contactless and participants could safely access the video and survey using a single link or Quick Response (QR) code. Visitor knowledge about bear and bison safety were measured prevideo and postvideo viewing within the same online survey, an advantage to the novel approach. An additional challenge was to measure behavior change and the application of visitor learning while maintaining 6 feet of physical distancing and without in-person interpretation.
Adaptations to Sampling Strategies
In addition to the innovative and novel methods, the PI created three adaptive mixed methods plans for data collection based on best- and worst-case scenarios regarding response rates and health and safety protocols. Park visitation patterns were notably different from prior years and contact needed to be eliminated between the research assistants and the participants. It was also acknowledged that safety protocols may change over the summer, and methods would need to be easily altered. Table 3 provides an overview of the adaptive research approaches in sequential order along with a brief summary of benefits and limitations of each approach.
Summary of Adaptive Research Approaches With Benefits and Limitations.
Approach 1: Passive Recruitment Strategy
The first approach (see Figure 1 and Table 3) consisted of a passive recruitment strategy (see Velott et al., 2008) where visitors were invited to participate in an online survey (quantitative) with the use of a postcard which included a brief description, a link, and QR code. Dodds and Hess (2020) note that one benefit of the pandemic on research methodologies is the improved familiarity with technology and digital platforms. Anecdotally, the research team observed that participants seemed to be familiar with how to scan QR codes on their mobile devices as their use of these technologies increased during the pandemic. This allowed the quantitative data collection to be contactless.
The interpretive video was embedded within the survey in Qualtrics and this allowed for prequestions and postquestions about awareness, knowledge, and human–wildlife interaction behaviors. The postcard was distributed at the park entry gate to all visitors along with maps and other park and safety information provided to visitors on a random selection of days of the week that coincided with the park staff wildlife guardians’ working days. Postcards were also distributed by the wildlife guardians as they interacted with visitors. The first approach was implemented and pretested during the first week of data collection (after the park reopened) and 250 postcards were handed out to visitors. Of those 250 postcards, only 25 surveys were completed (10% response rate) and 2 families visited the on-site research assistant to complete the comprehension assessment aspect of the study. With this very low response rate, it was decided that a more dynamic recruitment process would be needed.
Approach 2: Active Recruitment Strategy
The second approach (see Figure 1 and Table 3) was an active recruitment strategy (see Velott et al., 2008). An active approach requires a direct and personal invitation to explain and participate in the study. In this research, an RA approached groups of visitors at the park (maintaining a safe distance of at least 6 feet) and personally invited them to participate in the study and encouraged participants to immediately open the QR code with their smartphones whenever possible. She also indicated where the assessment tent was set up (near a main restroom) for participation prizes and informed visitors how long she would be on-site. On data collection days, postcards were similarly distributed to all campsite visitors and placed on all vehicles with a visible annual pass (since those visitors would not have stopped at the front gate for a day pass and already received a postcard about the study). To ensure continuity of messaging, data collection took place on days the wildlife guardians were on site (Friday-Monday). While response rates improved with this approach, overall responses still remained low (approximately 20% to 30%).
Approach 3: Combined Passive and Active Recruitment Strategies With Social Media Distribution
The third approach incorporated the two prior approaches with an additional social media distribution (see Figure 1 and Table 3). The information about the survey was posted on social media channels for Elk Island National Park, the Biosphere (which includes the three parks: Elk Island National Park, Strathcona Wilderness Centre, and Miquelon Lake Provincial Park) and in the Beaver Hills Biosphere Newsletter. This approach garnered some additional responses. It should be noted that the social media followings of the parks and the Biosphere were limited, with less than 500 followers. Also, with this method of distribution a specific response rate was not available for calculation (as the total reach is unknown with sharing and resharing across platforms). Certainly, the number of responses peaked after sharing on social media platforms, but the inability to calculate a response rate is a limitation of this approach (see Table 3 for further details). The social media posts included details about when the research team would be on-site and invited participants to come in person to participate in the comprehension assessment for the additional prizes.
Interpretation and Personal Meaning Maps
By August, a limited version of in-person interpretation was allowed again in Elk Island National Park. A roving interpreter provided interpretation to individuals or single families. The interpreter worked with the research team to integrate and incorporate key teachings from the research. With this change, personal meaning maps and interviews could be conducted. The RA visited the park on three predetermined dates with the Parks Canada interpreter and invited visitors to participate in the study by completing a pre-talk personal meaning map, engaging with the interpreter, and then completing a post-talk personal meaning map. The RA completed an abbreviated version of an on-site interview and invited participants to leave their phone number and email address in order to participate in a short follow-up interview 1 to 2 months later.
Throughout August, 19 visitors completed personal meaning maps and 9 of those participants completed follow-up interviews in September. Response rates for the personal meaning maps were 95%, where only one participant declined participation. This final adaptation brought the research design back to a variation of the original mixed methods embedded experimental design—where the personal meaning map and interview data were collected concurrently with the quantitative data from the survey and the comprehension assessment in order to compare knowledge and behavior change outcomes of visitors. This version of interpretation still posed some restrictions, and so instead of a control and treatment group in an embedded experimental design, we chose to compare and integrate data from the interpretive survey video (quantitative) and the in-person interpretation (qualitative). The study included elements of the original embedded experimental mixed methods design as well as the adapted explanatory sequential design. Quantitative data would assist with determining which interpretive approaches were most effective and qualitative data would provide insight into why and what elements of the interpretation were effective or not.
Data Analysis
Through mixed methods interpretation, we examine both quantitative results and qualitative findings to address our research questions (Creswell & Plano Clark, 2018). Quantitative data from the survey were analyzed using SPSS. Simultaneously, qualitative and quantitative data from the naturalistic observations (qualitative), the comprehension assessments (both quantitative and qualitative), the personal meaning map data (qualitative), and the follow-up interviews (qualitative) were analyzed using an inductive approach in NVivo. Attribute data collected from the comprehension assessment and the interviews were used for additional deductive analysis in NVivo.
As our data collection design evolved, we were forced to adapt our approach and our data analysis plan. Our explanatory sequential research design evolved as some elements of our original embedded experimental design were reintegrated as person-to-person interpretation was permitted in the park late in the field data collection season (see Table 2 for details). This resulted in a data analysis approach that collected and analyzed the majority of the quantitative and qualitative data simultaneously as per the original embedded experimental design. However, the follow-up interviews maintained the explanatory sequential design, where quantitative data were examined first and then qualitative data were employed to further investigate interesting findings in depth and bridge the results (Creswell & Plano Clark, 2018). Rather than thinking of the multiple forms of data as triangulation, we employed the concept of crystallization which “seeks to produce thick, complex interpretations” examining the data through a myriad of prisms and perspectives (Denzin, 2012, p. 84). Crystallization of the data consisted of analyzing visitor learning and behavior change through surveys, comprehension assessments, personal meaning maps, interviews, and through reflexivity as facilitated by the naturalistic observations and field notes of the research team. As this project is rooted in transforming human relationships with wildlife, reflecting on the data through crystallization aligns with our stance as methodological bricoleurs (Denzin, 2012).
Discussion
While innovative and adaptive methods have never been more integral to research and data collection as during the COVID-19 pandemic, research must remain rigorous with attention directed toward the benefits and limitations of different methods (Dodds & Hess, 2020; Fell et al., 2020; Fetters & Molina-Azorin, 2020; Gretzel et al., 2020). As with any toolbox, selecting the appropriate tool is important and depends on the research questions and the study context (Azzari & Baker, 2020). This study of visitor learning and behavior change during a global pandemic required the research team to make quick decisions about both the quantitative and qualitative data collection phases, as well as changes to sampling and recruitment procedures. The appropriate tools for this mixed methods study included the use of a survey instrument and comprehension assessments as the quantitative phase, with personal meaning maps, interviews, comprehension assessments, naturalistic observations as the qualitative phase. As the pandemic progressed and restrictions and physical distancing measures were put in place, the methods were adapted to be contactless while still collecting the desired data.
The use of technology and nonpersonal interpretation videos to present materials is relatively novel in the field of interpretation and environmental education, and the comprehension assessment was innovative in this research. The use of QR codes will likely grow and become more important for research during the COVID-19 pandemic because they can be integrated with a variety of methods and can collect large amounts of data while limiting personal interactions (Faggiano & Carugo, 2020). However, when adapting a method for a particular purpose, researchers must assess the reliability and validity of the methods used to measure a particular phenomenon - ensuring measurement consistency and intention (Creswell & Plano Clark, 2018; Rose & Johnson, 2020). As noted by Rose and Johnson (2020), tending to the overall trustworthiness of the research will address issues of reliability and validity. Trustworthiness applies to the rigor of the research design and results, which is determined by careful collection and analysis of both types of data, followed by integration and mixing of the qualitative and quantitative results (Harrison et al., 2020). We discuss quality constructs of each of our methods in Table 3.
Benefits and Limitations of the Research Approaches
The active recruitment approach was more successful than the passive recruitment approach, though the PI was initially concerned about maintaining physical distancing. Inclement weather made data collection even more challenging because of lower visitation and less visitor movement on poor weather days. Table 3 summarizes the benefits and limitations of the different research approaches.
Adapting to the Pandemic and Methodological Bricolage
As the pandemic disrupted research plans, the team became researchers-as-bricoleurs, understanding that the work would be shaped by the dynamic setting, the participants, and also themselves (Denzin & Lincoln, 1994). The pandemic dramatically impacted the research context because in-person interpretation at Elk Island National Park was no longer possible and physical distancing measures affected data collection. The participants themselves would also shape the data collection process. Visitation set record breaking numbers on weekends when it was not raining at the park because people wanted to engage in outdoor activities and safely meet with friends and family outside. At the same time, it was observed that many of these groups were not interested in contributing to the research. This was thought to be partially attributed to the concept of pandemic fatigue, discussed in more detail later.
Methodological bricolage pushed the research team to be proactive and think carefully about how to collect data safely, and how to adapt methods for COVID-19 pandemic protocols. Furthermore, this approach provided the opportunity to make choices and adapt methods as the research context changed (Pratt et al., 2020). As on-site interpretation programs were halted, the PI and park staff quickly responded by filming a short educational video about wildlife safety. Needing a safe way to show the video, it was embedded into the computer-assisted survey, making the quantitative phase of the research contactless. Promoting reflexivity, bricolage also allowed the research design and process to emerge creatively—inspiring the PI to create the comprehension assessments as a direct measure of visitor behavior. As noted by Denzin (2012), bricoleurs create “complex, quiltlike bricolage, a reflexive collage or montage, a set of fluid, interconnected images and representations” (p. 85). When Parks Canada announced a return to a limited variation of roving interpretive programming, our research team adapted the personal meaning maps to maintain physical distancing protocols and gather qualitative data about visitor learning.
Pandemic Fatigue
Our response rates from the active recruitment approach were lower than anticipated based on prior visitor research (Bueddefeld & Van Winkle, 2017, 2018). During the weekly debriefing meetings and in the observational notes, it was felt that some visitors had what is referred to as pandemic fatigue, which has been observed in research and workplaces and relates to a lack of interest or ability to engage (Patel et al., 2020; Zerbe, 2020). In this context, the research team believed that pandemic fatigue was exhibited in park visitors, and people appeared less interested in participating in research than prior years and less likely to read postcards or on-site signage about the study. The RA noted in her observations and in debriefing meetings that visitors seemed less likely to read additional materials due to the increase in COVID-specific signage. While the exact effect of this is unknown, it was agreed on by the research team that pandemic fatigue certainly played a role in the abnormally low response rate—especially considering two forms of incentives (immediate reward and a prize draw) were available to all participants.
Patel et al. (2020) compare pandemic fatigue with research fatigue, where participants lose interest in research engagement. They note that during a disaster there is often an overabundance of research addressing the current problem (the pandemic, in this case) with participants pursued by multiple researchers and projects (Patel et al., 2020). Though our study was not about the COVID-19 pandemic, we feel participants were overwhelmed with pandemic-specific messaging and perhaps unmotivated by a feeling of despair. Patel et al. (2020) include “seeing no change” (p. 449) as a factor in research fatigue, and we wonder if negative and overwhelming emotions directed toward the current pandemic might influence potential participants to avoid research engagement. Linking this back to the concept of bricolage, participant pandemic fatigue shaped the course of the research as the research team continued to adapt recruitment approaches to garner more participation. This also changed the intended sample, because social media recruitment may have reached out to people not at the park on research days.
Methodological Benefits and Limitations
There are several methodological benefits and limitations to our adaptive mixed methods study, which are summarized in Tables 3 and 4. Regarding the survey, not everyone has access to a mobile device and cell service at Elk Island National Park is not always reliable. Additionally, while it could be noted that an outcome of the pandemic is improved digital literacy (e.g., the use of QR codes and online surveys), it is also possible that by using these technologies we create an overrepresentation of particular groups of the population (Fell et al., 2020; Sy et al., 2020). Due to the pandemic, we had limited ability to distribute the survey and could not provide paper or tablet options. Participants unable to use their personal mobile device were given information about how to access the survey at home, but were unable to take the comprehension assessment. This limited our ability to conduct on-site comprehension assessments and some naturalistic observations.
Methodological Benefits and Limitations for Participants and Researchers.
Source. Table adapted from Dodds and Hess (2020).
Contribution to the Field of Mixed Methods Research
There is a growing body of literature that addresses a need for creative mixed methods research (Azzari & Baker, 2020; Dodds & Hess, 2020; Fell et al., 2020; Fetters & Molina-Azorin, 2020; Gretzel et al., 2020; Mars & Ball, 2016; McSweeney & Faust, 2019; Pauwels, 2010). Research was conducted during a global pandemic and this study is an example of a design that was quickly modified to follow public health protocols while not just continuing data collection, but using the situation as a moment of innovation. Future scholars can use this as a guide when faced with similar challenges that require changes to planned research designs. It also serves as an example of leveraging technology to replace an in-person experience (the interpretive programming became a video) and creating new ways of assessing knowledge when traditional methods are no longer appropriate. In relation to methods, this article describes an innovative and novel method of facilitating interpretation and measuring visitor learning and behavior change in the midst of a pandemic. The comprehension assessment, in particular, demonstrates a creative solution to a pandemic problem, and offers future researchers and park interpreters a means of directly evaluating visitor understanding about wildlife safety. Furthermore, the article adds to the literature on methodological bricolage and brings attention to its importance in mixed methods work, as well as its importance during challenging times.
Recommendations for Future Mixed Methods Inquiries
We have made the argument for the support of interdisciplinary and bricolage informed research, especially during the COVID-19 pandemic; however, we simultaneously acknowledge the challenges this creates. Even without needing to be continually adaptive due to the pandemic, interdisciplinary research always requires collaborators to “ . . . devote substantial time and effort to cultivating relationships, establishing a shared language, and devising a common point of view from dissimilar stances” (Wen et al., 2020, p. 2). In this case, the research partners from Elk Island National Park were integral in their willingness to communicate and support the research with ongoing adaptations during an already challenging time.
Future research can benefit from our approach to innovative, novel, and adaptive research during the pandemic in several ways. First, researchers reading this article are reminded of the need to create a research plan that can be modified and adapted quickly to changing contexts. Being open to new research formats and different data collection methods can enhance our ability to be adaptive. Second, the innovative, novel, and adaptive methods discussed here can be modified and used by scholars from a wide variety of disciplines. These added research tools aid in a methodological bricolage approach. Furthermore, thinking outside of the usual research toolbox assists with the overall research design process (Azzari & Baker, 2020). Third, a review of our research approach and methodological benefits and limitations can aid researchers in preparing and adapting their methods to the specific challenges of field-based research design and recruitment.
Conclusion
Within the past decade, “leisure scholars have called for the use of new and innovative research methods that are designed to engage participants in the research process” (Johnson, 2014, p. 321). This becomes even more essential during a global pandemic where research methods need to be modified to reflect health and safety protocols. This article provides an outline of the adaptive planning framework that we used in order to successfully respond to ongoing and rapidly changing protocols to ensure the health and safety of the research team and study participants. Our argument for adaptive approaches combined with innovative and novel methods is strengthened by acknowledging the constructs of reliability, validity, richness, generalizability, and usability. We note some issues with the social media distribution approach in the discussion, however, additional survey completions by respondents who follow social media related to Elk Island National Park were effective for crystallizing the data. Additionally, this approach was a safe way to collect data during the pandemic.
The combination of on-site survey (active) recruitment and an in-person comprehension assessment improved overall reliability and validity by providing immediate measures of understanding and behavior. This also contributed to richness by crystallizing the data and it allowed the RA to provide participants with immediate feedback about their assessment. Overall, the comprehension assessment was easy to apply, enjoyable for the participants, and provided rich quantitative and qualitative data. We note this method as a strength to our mixed methods study.
In conclusion, while the COVID-19 pandemic presented unprecedented challenges to research and data collection, it has also inspired innovation and created space for adaptive frameworks that were not previously considered. In this study, our research team used innovative and novel data collection methods and adaptive approaches, including QR codes, videos, comprehension assessments, and social media for innovative dissemination and evaluation strategies. Even as the COVID-19 pandemic subsides, future research will be permanently altered as we have a precedent to use these innovations and adaptive methods. We believe this is the future of applied research, and mixed methods in particular—where innovation is embraced and a methodological bricolage becomes the new standard.
Supplemental Material
sj-pdf-1-mmr-10.1177_15586898211019496 – Supplemental material for Methodological Bricolage and COVID-19: An Illustration From Innovative, Novel, and Adaptive Environmental Behavior Change Research
Supplemental material, sj-pdf-1-mmr-10.1177_15586898211019496 for Methodological Bricolage and COVID-19: An Illustration From Innovative, Novel, and Adaptive Environmental Behavior Change Research by Jill Bueddefeld, Michelle Murphy, Julie Ostrem and Elizabeth Halpenny in Journal of Mixed Methods Research
Footnotes
Acknowledgements
The authors would like to thank the research partners from Parks Canada at Elk Island National Park. In particular, thank you to Cameron Johnson, Heather Doyle, and Ramona Maraj, for your expertise and flexibility with this project.
Author Contributions
Principal investigator and primary author.
Second author and research assistant.
Third author and research assistant.
Fourth author and postdoctoral fellow supervisor.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Mitacs and the Land Stewardship Centre of Canada in partnership with the Beaver Hills Biosphere (FR42787).
Ethics
Ethics approval obtained from the University of Alberta (PRO00096944).
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
