Abstract
Aim:
To propose a methodological framework for conceptualizing, planning, and implementing an environmental scan (ES) in health services delivery research (HSDR).
Background:
An ES is a methodological approach employed to examine a range of practices, policies, issues, programs, technologies, trends, and opportunities from a variety of data sources to inform program or policy development. Despite the wide use of ESs in health care to inform decision-making, a lack of methodological guidance exists to support researchers in planning and conducting an ES in HSDR.
Methods:
Adapting McMeekin et al’s process for developing methodological frameworks, we identified literature that described approaches to planning and conducting ESs in addition to exemplar articles that featured ESs in HSDR. We integrated original research findings and synthesized data from all sources to generate an evidence-informed methodological framework.
Results:
We developed RADAR-ES that consists of 5 phases and is informed by 4 guiding principles: (1)
Conclusion:
RADAR-ES will provide comprehensive guidance for researchers and health services stakeholders who plan and conduct ESs in HSDR.
Keywords
Introduction
An environmental scan (ES) has been described as a type of inquiry and process to identify, collect, and synthesize information to understand services, issues, trends, and other aspects of service delivery.1,2 The concept of ES is derived from the term environmental scanning that was defined by Choo as “the acquisition and information about events, trends and relationships in an organization’s external environment. . .” (p. 21) 3 and by Rowel et al 4 as a process “to assess internal strengths and challenges and external opportunities and threats” (p. 1). Originating in the disciplines of business and information science, ESs have been adopted and widely used in many other fields, including health care.2 -5 Despite the increasing use of ESs in health care, there is a lack of clarity on the methodological process and how it differs from other methodologies. Moreover, there is a dearth of sources available to guide researchers in planning and conducting this specific type of inquiry. In this paper, we build upon our previous work and extant literature to propose an evidence-informed 5-phase methodological framework, RADAR-ES, for conceptualizing, planning, and implementing an ES in health services delivery research (HSDR).
The genesis for developing a methodological framework for ESs in HSDR arose from challenges we experienced in identifying a comprehensive model or theoretical approach to inform an ES that was part of an earlier multi-component study. 6 Our goal was to ensure a high degree of consistency and quality in our research; however, as we discovered through a scoping review, literature outlining approaches to ESs and environmental scanning were inconsistent, fragmented, and incomplete.1,2 The lack of guidance available to support researchers in planning and implementing ESs has been corroborated by other authors, who either relied on methods from other primary studies or adapted other frameworks to inform their work, such as methodological approaches used for scoping reviews.7,8
Building on our scoping review, 2 a survey was conducted of stakeholders to better understand their perspectives on and experiences with using ESs in HSDR. 9 In this study, 42% of stakeholders indicated they had challenges finding methodological guidance for designing, conducting, and reporting an ES and 75% of them reported a need for best practices, standards, or guidelines to plan ES. As well, stakeholders noted a lack of a formal definition for ES and identified inconsistent use of terminology to describe ESs, highlighting confusion as to whether ESs differ from other methodological approaches. 9
A Delphi study of panelists with expertise using ESs in HSDR was also conducted to further advance the development of a methodological framework for ESs and to seek consensus on elements of a definition and components specific to ESs conducted in HSDR.
10
Panelists in the Delphi study included researchers, healthcare practitioners, policy makers and others from academic, health, and government sectors who had expertise in conducting ESs in HSDR.
10
Over 76% of study panelists rated the importance of a consensus definition for ESs as high and almost 81% were in high agreement that a definition of ES would clarify its conceptual meaning and provide guidance to participants. Eighty-four percent of panelists indicated high agreement that ES is a type of methodology and, while only 68% were in high agreement of ESs being a distinct methodology, the remaining 32% were in medium agreement.
10
From the Delphi study we refined our original definition of ES in HSDR to:
a methodology used to examine a wide range of healthcare services, practices, policies, issues, programs, technologies, trends, and opportunities through the collection, synthesis, and analysis of existing and potentially new data from a variety of sources to help inform decision-making in shaping responses to current challenges and future health service delivery needs.
10
As of 1 May 2025, a total of 142 unique literature sources by authors representing 20 different countries had cited our scoping review and research articles. Of this literature, 48 sources (see Appendix A) were specifically related to HSDR where an ES was explicitly described as either the central focus or component in 39 primary studies and 6 study protocols (n = 6). The remaining 3 sources represented methodological, theoretical, and program development perspectives related to ESs in HSDR. References to our previous works outside of HSDR were mainly presented in primary studies in other contexts, but also in methodological or theoretical papers11 -15 and book chapters.16,17 As well, 7 citations of our previous work were in graduate student theses and dissertations.
References to our published works were used to help define or describe ESs, rationalize the use of an ES approach, and/or help delineate the process of an ES.7,18 -22 Several articles had adopted and/or adapted methodological approaches used for other types of inquiries,7,8,23 -25 such as using Arksey and O’Malley’s 26 methodological framework for scoping reviews or mixed methods designs.23,24,27 We established that a gap exists in guidance for designing, implementing, and reporting ESs in HSDR through our previous research1,2,9 and this has since been identified by other authors,14,15 thus supporting the need for development of a methodological framework for ESs in HSDR.
The Merriam-Webster online dictionary defines methodology as “a body of methods, rules, and postulates employed by a discipline: a particular procedure or set of procedures.” 28 McMeekin et al 29 refer to this definition in mapping of methodological frameworks and provide a working definition for methodology as “a tool to guide the developer through a sequence of steps to complete a procedure” (p. 2). In relation to research, Singh et al 30 states a methodological framework “identifies the principles, rules, and procedures that guide the process through which knowledge is acquired” (p. 36). We define methodological framework as a structure that outlines a sequence of principles, methods, and procedures that informs the research process. Methodological frameworks improve consistency in conducting research activities through standardized approaches and enhance the robustness, quality, and trustworthiness of findings.29 -31
Method for Developing the Methodological Framework
McMeekin et al 29 proposed a 3-phase approach to developing a methodological framework that includes identifying evidence to inform the framework, developing the framework, and evaluating and refining the framework. In this paper, we focus on the first 2 phases of this approach to develop an evidence-informed methodological framework for ESs in HSDR. For the purposes of this paper, we regard HSDR as research that explores services or initiatives provided by the healthcare sector and/or healthcare providers to support planning and/or implementation of direct delivery of services to specific client groups (ie, individuals, family, community, target populations). This encompasses a range of foci (eg, services, practices, policies, etc.) across various a wide array of care types (eg, primary care, acute care, health promotion, prevention services, etc.).
Phase 1: Evidence to Inform the Methodological Framework
For this phase, McMeekin et al 29 outlined 5 steps in developing a methodological framework: (a) identifying existing frameworks, guidance, and methodology; (b) conducting purposeful literature searches; (c) drawing on the experiences of individuals with expertise in the field; (d) using qualitative research to generate new data; and (e) collaborating and consulting For the first step in this phase, we identified foundational sources that explicated theoretical and methodological underpinnings for ESs through background literature searches and our scoping review.1,2 This included primary sources that described methods and procedures used in conducting ESs and we present a summary of these sources in Table 1. Additional primary sources of studies and descriptions of methodological approaches are included in Appendix A.
Sources Referenced for Methodologic Approaches to ESs.
As part of the second step in this phase, our scoping review identified and mapped examples of the use of ESs in HSDR that described methods, procedures, and ways of conceptualizing ESs. 2 We augmented this scoping review with targeted literature searches and the articles in Appendix A to identify additional examples that described approaches to ESs in HSDR. From these searches we identified a sample of peer-reviewed articles that illustrate a range of methodological considerations for conceptualizing, designing, and implementing ESs (see Table 2). To promote a degree of variation in this sample we selected articles to reflect a range of countries, health service delivery contexts, types of data collected, and methods or procedures used in the ES.
Sample Articles of Primary Studies and Protocols Identified as Being or Having ESs in HSDR.
For steps 3 and 4 of this phase, we explored the perspectives and experiences of a range of participants (eg, researchers, clinicians, policy makers, etc.) who had conducted ESs in HSDR in a stakeholder survey 9 and invited additional input specific to a methodological framework for ESs from experts through a Delphi study. 10 The final step of collaboration and consultation for framework development was facilitated through connections with experts as a result of the studies and through the interdisciplinary, multi-university nature of our team membership in this project.1,2,9,10
Phase 2: Developing the Methodological Framework
In this second phase of framework development, McMeekin et al 29 emphasized an iterative process that involves: (a) data extraction; (b) analysis of data to identify and synthesize categories; and (c) a process to develop and evolve the methodological framework through a consensus. We developed an early conceptualization of a methodological framework and its elements in a face-to-face 2-day working session in June 2022 and this evolved as we collected additional data.1,2,9,10 We extracted and compared data from foundational sources and other literature to help delineate phases and steps in the methodological framework. Decisions on the phases and stages of our framework was achieved through immersion in the data, consensus during team meetings, and online collaboration spanning 3 years.
Methodological Framework for an ES
In presenting our proposed methodological framework for conceptualizing, planning, and implementing an ES in HSDR, our goal was to develop a framework based on available evidence but also supported by theoretical and philosophical underpinnings. Participants in the Delphi study were in high agreement that a methodological framework should be guided by theoretical concepts (81%) and grounded in principles that guide the research (79%). 10 In this section we present our philosophical and theoretical perspectives on ESs in HSDR, guiding principles for ESs, and RADAR-ES, our proposed methodological framework for ESs in HSDR.
Philosophical and Theoretical Perspectives on ESs
We situate our perspective from a research philosophy grounded in both pragmatism and constructivism. Pragmatism emphasizes practical outcomes and the use of multiple methods to solve real-world problems in a practical way.52 -55 Tenets of pragmatism include determining the appropriate methods for the research question, use of multiple research approaches, integrating a range of data sources, involvement of people with expertise in the design and execution of research, and applicability findings to other situations.52 -57 We believe that pragmatism is one defining attribute of ES since the primary purpose of an ES is to describe the current landscape of services and to explore gaps and trends as a means to inform decision-making through practical and flexible methods of data collection. We believe that constructivism is another defining characteristic of ESs given a project team’s (the researchers) involvement with collection, analysis, and synthesis of data, particularly when engagement occurs, particularly with input from experts and stakeholders. Constructivism holds that research is a subjective and socially constructed enterprise where the researcher(s) are the main instrument in all aspects of the research process and where co-construction of new insights and knowledge generation is a result of social processes.30,55,58 -60
In conceptualizing this methodological framework, we distinguish an ES as a distinct methodological approach to generate knowledge in that it is meant to assess internal and external environments for changes (eg, social, economic, technological, political, etc.) related to an issue or phenomenon to inform decision-making by examining the current landscape of services or programs.4,34,35,37 -40 This is different from the purpose of a scoping review that is meant to “determine the scope or coverage of a body of literature” (p. 2) 61 and to map the concepts, sources, types of evidence, and knowledge gaps from the literature on a particular topic of focus.26,61 To this end, we believe ESs that solely draw on peer-reviewed and gray literature sources would be more appropriately framed in the context of a literature or other systematic review for which methodological conventions (ie, principles, methods, and procedures) are already well-established and validated.26,61 Similarly, ESs should not be framed as solely being either a qualitative or quantitative study where longstanding methodological traditions (eg, qualitative descriptive, quantitative descriptive, etc.) exist to inform those research processes.
Based on our research and stakeholder input, we believe another important defining attribute of an ES is it being a multi-method inquiry comprised of 2 or more different data sources and, at least, 2 different methodological approaches for data collection where results are synthesized to address a phenomenon of focus.2,7,9,10,20,32 Given that a significant amount of data is collected through web searches where gaps in available information and from gray literature may be identified, the use of multiple data sources and methods can help these gaps and promote a more robust and rigorous inquiry.7,13,20,32,62 In alignment with this perspective, an ES may have elements of quantitative and/or qualitative research; however, ESs should not be regarded as a type of mixed-method study. A main goal of mixed-method research is to bring together data points from 2 or more different research methodologies for the purposes of explanation and methodological approaches for this type of research have been well-defined and validated.56,57 Mixed-methods research differs from the main goal of an ES where the explicit focus is assessing current practices and trends to inform decision-making.
Guiding Principles of ES Framework
Underpinning our proposed methodological framework for ESs in HSDR are 4 guiding principles that thread through the phases and steps of the ES. These guiding principles take into consideration: (1) the context in which the ES occurs; (2) timelines for the project; (3) engagement of knowledge users in ES process; and (4) elements that promote ethics, transparency, and rigor. In addition to underpinning the framework, the principles intersect and overlap with each other to varying degrees and in an iterative cycle across ES processes.
Ecological Context of ES
When conceptualizing and working through the phases of an ES, we draw upon an adaptation of Bronfenbrenner’s ecological systems to help anticipate and account for contextual factors that may impact the work of the project team.63,64 The Bronfenbrenner ecological theory is a framework originally based on human (child) development that emphasizes the interdependency and interplay among the various components of the ecosystem and how these relationships influence the person.63,64 This theory was initially developed within the developmental psychology field but has since been applied in other domains including education-related research and public health contexts.65 -67
The ecological systems levels we have reflected in our methodological framework are micro, meso, macro, and chronological based on Bronfenbrenner’s theory and other applications of the ecological model. We define the levels as follows:
a. Micro: encompasses factors and interactions within the immediate sphere of the project team that impacts planning and implementing ESs. Examples include identifying skill sets needed of team members (Phase #2)50,68,69 and team approaches to data analysis (Phase #4).11,22,70
b. Meso: these include influencing factors within the immediate settings and environments that the project team operates in, usually the place of employment or agency initiating the ES, and ought to be considered when planning and implementing an ES. Examples include having dedicated time and resources to conduct an ES (Phase #2) 23 and determining the requirement for ethics (Phase #3).27,43,44
c. Macro: encompasses those factors, often social, political, philosophical, and regulatory/legal in nature that influences the research team’s engagement in an ES and transcends the sphere of influence of the local institutional setting. Examples would be jurisdictional (eg, national, state, or provincial) freedom of information and protection of privacy laws that restrict access to information (Phase #3) and knowledge dissemination external to the project team’s organization (see Phase #5).20,41,50
d. Chronological: accounts for the timing and contextual events that may impact any phase or step during the span of an ES project; this is not synonymous with the principle of ES Project Timeline but, rather, accounts for events happening while the ES is underway. An example would be the occurrence of a world event, such as the COVID-19 pandemic, that changed the way health services were delivered, restricted a team’s ability to access information, an/or resulted in changes to project funding.
Project Timelines
One critical element to an effective and rigorous ES is the timeframe in which the project is conducted; this needs to be considered at conceptualization of the ES project as it may influence the design, implementation, and reporting processes.5,9,15 Project timelines may be dictated at the outset by chronological events within the ecological context of an ES, such as urgency and timeliness related to the phenomenon of interest (eg, COVID-19 and Monkey Pox outbreaks) or, in the case of research, project funding opportunities.11,35,40,68 As with most research studies, ES project timelines also need to account for time required to engage knowledge users, collaborate with experts, collect data (eg, searches and participant interviews), conduct analysis, and write up the findings.11,40
Many studies have reported dates delineating the start and end of data collection for the ES and, in the case of literature reviews, identified dates associated with executing literature searches that help provide a chronological context of the ES.20,21,27,42,47,69 An ES has been described as a “snapshot” in time that captures a specific period in which a phenomenon occurs (eg, issue, trend, policy, etc.) and the temporal dimension in which the ES is conducted; this influences both methodological processes (eg, what can be achieved) and contextualization of findings.4,9,21,48 As we discuss presently, reporting time periods of searches and data collection enhances transparency of the ES process and supports dimensions of credibility and transferability of findings.
Knowledge Users
In an ES process there will often be a requirement to involve various stakeholders who may contribute expertise related to the research process, perspectives on HSDR, and/or information that is formally collected as data. We differentiate stakeholders into 3 distinct categories, recognizing that there may be an overlap depending on the type of design adopted for the ES project: project team members; participants; and knowledge users. Project team members are those individuals who play an active role in conceptualizing, leading, managing, and/or supporting an ES.32,33,39,40 Participants are those individuals or groups who are invited to provide information in the form of qualitative or quantitative data in either an informal (eg, “word-of-mouth”) 5 or a formal research capacity (eg, interviews, focus groups, surveys, etc.).4,13,14,32,33,38,40 We discuss project team members and participants more fully in Phases 2 and 3 of the methodological framework, respectively.
For the purposes of our methodological framework, we have adopted the definition of knowledge user from the Canadian Institute of Health Research (CIHR):
. . .an individual who is likely to be able to use research results to make informed decisions about health policies, programs and/or practices. A knowledge user’s level of engagement in the research process may vary in intensity and complexity depending on the nature of the research and on his/her information needs.
71
We regard knowledge users as having an investment or interest in the issue or phenomenon of focus for the ES and who may be an important source of contextual information related to the ES project. For instance, knowledge users may identify a health services delivery opportunity (Phase 1) or be potential gatekeepers in accessing and collecting data (Phase 3). In alignment with CIHR’s definition, a knowledge user “. . .can be, but is not limited to, a practitioner, a policy maker, an educator, a decision maker, a health care administrator, a community leader or an individual. . .” 71 whose contributions are both actively sought out by the project team and span the duration of the ES project.
Contributions of knowledge users may occur in various roles or take a range of forms, such as through advisory committees or expert panels, collaborations, consultations, and so forth.4,13,22,33,40,50 Knowledge users may potentially be integrated as part of the project team to assist with key decisions throughout the phases and steps of an ES project thus enhancing the robustness and utility of the findings.21,22 Incorporation and engagement of knowledge users ought to be considered early in the conceptualization of an ES and consistently maintained throughout the phases of an ES project.39,40
Ethical Comportment, Transparency, and Rigor
The fourth guiding principle that underpins our methodological framework reflects the integrated elements of an ethical approach to ES processes, facilitation of transparency, and promoting rigor when planning and implementing each phase and step of an ES.
Ethical Comportment
Ethics approval is an expectation in academic institutions when conducting research involving human subjects and a statement related to ethics is generally a condition for publication in reputable peer-reviewed journals. In addition, professional health disciplines must adhere to codes of ethical conduct and to maintain ethical comportment when interacting with clients and populations, particularly persons who are structurally disadvantaged (eg, individuals experiencing homeless, dealing with addictions, disability, etc.). 72 Ethical comportment is defined as a “pre-reflective, socially embedded practical knowledge that is rational,. . .is lived and embodied in practices” (p. 77) that transcends merely abiding by formal ethics requirements to the etiquette of outward performance and intentions of the researcher. 72 Our position is that ethical comportment is good practice and a way of being that is intentionally and mindfully enacted by all project team members throughout the ES project and not simply attending to the technical aspects of humans’ research ethics approval. This is a particularly crucial element when engaging knowledge users and participants where relational and social interactions transpire.
Transparency
There was high level of agreement (75%) in the Delphi study that a methodological framework would promote transparency in both the process and results of an ES. 10 Transparency in research is the open sharing and reporting of methods, procedures, and findings throughout the research process and is essential to supporting accuracy and other elements of rigor (eg, credibility and confirmability) that we presently discuss.73 -78 Transparency is also necessary for reproducibility in empirical research; that is, the ability for others to replicate the research.73-75,77 Specific reporting guides have been established and are required by many journals to assess transparency and quality of article submissions, such as Standard for Reporting Qualitative Research (SRQR) for qualitative research74 -77; however, there currently are no guidelines for transparency specific to ESs. Given the qualitative nature of ES inquiries and frequent inclusion of systematic literature reviews as a data source, we suggest that project teams adapt appropriate guidelines, such as SRQR 76 and/or the applicable Preferred Reporting Items for Systematic reviews and Meta-Analysis (PRISMA)77,79 to guide reporting elements of an ES from the outset of developing the designing the project.
Rigor
Rigor is well-recognized as essential in research and knowledge generation to ensure the quality, reliability, validity, and trustworthiness of findings; it is realized through attention to the planning, design, and execution of research processes in a study. 30 A high level of agreement (81%) was reported in the Delphi study that a methodological framework for ESs would enhance rigor. 10 However, few foundational sources or other methods-based papers for ESs (eg, Author et al11,32 being examples) explicitly describe rigor as a consideration in ES processes. In reviewing the primary studies listed in Appendix A, rigor was not explicitly described in the methods section of articles; however, some authors reported elements of rigor associated with their ES, such as using a protocol and terminology associated with rigor (eg, validation, triangulation, etc.).21,22,41,42,44,50 Interestingly, 6 articles in Appendix A and other sample articles (Table 2) identified issues of rigor when describing limitations of their ES, such as generalizability or transferability due to lack of representation in a participant sample and collection methods.20,43,45,47,51,80
Our view is that an ES, as a research methodology, should adhere to established conventions of rigor to both facilitate and demonstrate quality in knowledge generation. Given that many ESs collect, analyze, and ultimately report qualitative data, the dimensions of rigor associated with naturalistic inquiry and qualitative research can be appropriately applied to the planning and implementation of an ES project. Well-established and traditional dimensions of rigor that stem from Lincoln and Guba’s original work are authenticity, confirmability, credibility, dependability, and transferability; cumulatively, these criteria reflect the trustworthiness of both naturalistic and constructivist inquiry in qualitative research.30,55,59,78,81 -83 We advocate that ethical concordance also be considered a dimension of rigor that integrates ethical comportment and transparency to enhance the overall quality of an ES. 30 We provide a definition for each dimension of rigor as it applies to an ES in Table 3.
Definitions for Dimensions of Rigor in Environmental Scans.
RADAR-ES: A 5-Phase Methodological Framework
Our proposed methodological framework for ESs in HSDR, RADAR-ES, is delineated into 5 phases: (1)
Phases of RADAR-ES process.
Phase 1: Recognizing the Issue
The first phase of the ES process is mainly concerned with conceptualizing the project to delineate a clear focus of inquiry and contextualize the scope within which the inquiry takes place. The starting point, as illustrated in Figure 2, is identifying the issue related to health services delivery that precipitates an iterative process of determining the purpose or aim of the ES, the objectives to be achieved and the framing of a research question. From our experience, there is an interplay between the steps in this phase as clarity is sought on the phenomenon of interest through immersion in the extant literature, initial exploration of similar health services delivery models, and engagement with prospective knowledge users.
Phase 1- recognizing the issue.
Identify the Issue
The impetus for change in health services delivery may arise from a broad range of ecological system influences that present an issue or a problem that needs to be addressed and may take the form of a gap in knowledge, challenge, opportunity, or concern.5,32,37,39 These issues or problems may be identified through: (a) previous work (eg, literature reviews and research);22,43,45,49 (b) recognition of need to improve direct service delivery (eg, enhance access, support clinical practice, etc.);7,18,41,46,85 (c) emerging situational events (eg, COVID-19 pandemic, Monkey Pox epidemic, etc.);20,21,68 (d) opportunities for collaborations with knowledge users (eg, program development);13,21 (e) identification of innovations or trends (eg, use of technology);7,11,68 and (f) funding for research;45,86 -88 Often these issues are external to the project team and overlap ecological contexts,37,39,40 such as with COVID-19 that was a specific event in time (chrono level) and had consequences related to health services delivery (meso level) and was also influenced by government mandated laws for distancing and isolation (macro level).20,21
Explicitly defining the issue or opportunity will be beneficial in delineating the focus of inquiry and the contexts to be explored as this will help inform the steps that follow and the scope of the ES project. For instance, clearly identifying the core focus (ie, service, practice, policy, etc.) and the setting(s) involved can provide a foundation to establish the purpose or aim, and to also offer a more homogenous parameter for the project. This may necessitate further targeted searches of the literature and consultation of knowledge users or other sources of expertise.
Determine the Purpose/Aim
Congruent with research convention, a clear purpose or aim should be stated to answer the “why” an ES is being conducted and to outline the intention.15,32,33,39,40 The rationales for having a clear purpose or aim are to focus the line of inquiry, anchor the ES process, and keep the project team on track throughout the process.39,40 Most often, primary studies reporting ESs articulated either a purpose or an aim; occasionally, however, both were provided (eg, Asheber et al 18 and Neville et al 49 ). The purpose or aim may be focused on a singular outcome41,44,49,85 or there may be more than 1 motivation, such as seeking to both understand an issue and look for gaps, challenges, or innovations related to that issue.7,18,21,46,51 In some instances, the purpose or aim was directly linked to either objectives of the ES or specific questions associated with the inquiry.18,22
Develop Objectives
There is often an overlap of purpose/aim with objectives; however, the intent of having objectives is to more discretely delineate the tangible outcomes of the ES project and they may also be used as indicators to measure completion. There can be several objectives for an ES outlining a range of outcomes, from identifying and describing specific elements of an issue to mapping resources to exploring perspectives with participants.20,22,27,41,42,89 Objectives should clearly align with the issue and the purpose/aim to further focus the overall project.
Frame a Research Question
Similar to conventional research approaches, 1 or more research questions ought to be developed for an ES project.5,32,39 The purpose of a research question(s) is to help ensure ES activities are targeted and effective.5,32 Research questions can also inform choices for research design, such as identifying data sources and data collection methods.5,30,39,55 Atuluru et al 68 illustrate how questions can be linked to specific data sources and methods for data collections, and additional examples of research questions are evident in the works of Asheber et al, 18 Brickley et al, 19 Deshpande et al, 86 Holdsworth et al, 45 Kimber, 90 and Porterfield et al. 50
When developing a research question, it is beneficial to use a mnemonic such as PICo (Participants, Phenomenon of Interest, and Context) or PCC (Population, Concept, and Context) to frame the question; this helps in identifying variables associated with the project (ie, for searches), defining key concepts, establishing inclusion criteria for the ES design in Phase #3.30,91,92 Determining variables and key concepts of focus in any research is essential for clarity for purposes of determining search terms (literature or web), inclusion and exclusion criteria, and analysis of data.30,41,57 Explicit definitions of concepts and constructs have been cited in numerous examples of ES studies, often supported by evidence from referenced sources.7,42,45,46,86
Phase 2: Assessing Factors for ES
The second phase of the ES process is focused on identifying and considering all factors that could either impede or facilitate the ES in anticipation of decisions that would inform the design of the ES project. Challenges and barriers reported in relation to ESs include factors such as time constraints, workload, resources, funding, information source accessibility, and expertise needed for this work.5,9,15,32 From our collective experience, identifying some of these factors would begin during Phase 1 and change during the project timeline, thus illustrating the iterative and mutable nature of this phase (Figure 3). The steps within this phase are also an iterative process as preparations for the ES are made.
Phase 2: assessing factors for ES.
Select Project Team
Having a dedicated project team to provide leadership, expertise, and support throughout the project is instrumental to planning and executing an ES.32,40 Considering the level of personal expertise in conducting an ES was rated high agreement by 94% of respondents in our Delphi study. 10 The project team should have a designated lead to coordinate and “. . .champion the entire environmental scan process from development to dissemination” (p. 2). 40 Project team members can provide additional insights to better understand an issue and provide refinement to the project purpose/aim and objectives in Phase 1; they can also offer input and guide decisions related to substantive areas of knowledge (eg, specific knowledge of health services), research methodologies, and development of a workplan for the ES.15,32,39,40
Selection of project team members should be considered based on both the needs of the project and the necessary knowledge and skills required. 32 The size, scope, and timelines of an ES project will often be a factor in the amount of human resource support required to ensure tasks of data collection, data extraction, and other administrative support are met. Searching and screening sources, especially if a systematic approach to literature review is done, often requires a minimum of 2 or 3 people.18,45 However, expertise and skill sets specific to literature searches, data analysis, evidence synthesis, and related research processes are essential to ensure thoroughness and rigor of an ES.11,32,49 For instance, librarians are often included in the project team to assist with identifying key search terms for both literature search strategies and web searches.18,27,62 Knowledge users may also be included as a formal part of the project team to provide perspectives on of health services delivery, particularly from the viewpoint of a collaborator or consumer; this can be achieved in several ways through individual contributions or in groups, such as on an advisory, education, or steering committee.22,25,50,69
Identify Available Resources
The capacity to undertake an ES is often contingent on availability of resources and organizational support, especially in non-research contexts where recognition by senior management for the need of an ES may be necessary.5,9,32,40 This requires identification of resources needed to effectively undertake an ES project, such as estimating human resource (eg, research assistants) needs, funding, and infrastructure (eg, technology) to support the initiative.5,9,32,40 Resource requirements were not explicitly noted as a consideration in the primary studies and protocols we reviewed; however, references to size of teams, implied time required for executing procedures (eg, data collection and analysis), and use of technology for ESs (eg, data management platforms, web searches, etc.) was evident. Furthermore, impact of resources on an ES was suggested as a limitation in some studies and these included labor-intensive tasks associated with the process (eg, screening large amounts of data sources), geographical locations (eg, travel and coordination challenges), and time constraints in relation to resources.7,22,23,68 From our experience, identification of available resources is necessary to balance projected timelines and the necessity to apply for research funding is similar to any other research project. We noted from our review that ESs in health services delivery often receive funding.12,20,23,24,68 and, while it is often difficult to determine precise estimates for human resources and material costs, considering a budget may help establish feasibility of the project and is often a requirement for funding applications.
Determine Feasibility and Mitigate Challenges
In this step, the project team considers the various factors that may impact the feasibility of the ES project. As previously noted, sufficient resources (ie, funding and personnel) and timelines may play a part in determining the ability to conduct the ES, but it may also influence decisions on the scope and size of the project. 32 For instance, urgency of an issue as in the case of the Monkey Pox pandemic resulted in a “rapid ES” due to a compressed timeline; it was noted by the authors that this possibly resulted in missed data points and the inability to include consumer engagement data resources, 68 thus impacting elements of rigor. Other challenges to feasibility noted by authors were access to information due to complexity of healthcare systems and geographical location, language barriers (eg, limitation of team to English), and/or lack of published information on very specific topics (eg, hearing care).7,18,20,21,23,43 Specific examples of lack of published information included constraints of websites where little information is posted online (eg, health service agencies) and posted information is not current.21,22,25,62
Early planning and discussions to get an overall picture of a proposed ES are instrumental for an effective and efficient project.39,40 From our experience as researchers, this is a worthwhile investment of time to identify potential facilitators, barriers, and mitigation strategies to deal with any anticipated challenges. It is also a step in pre-planning ahead of designing the ES project (Phase #3). We recommend that the project team use a pragmatic step-wise approach to: (a) clearly delineate the scope (size and scale) of the ES project; (b) use the steps from Phase #1 to identify likely data sources and collection strategies; (c) list facilitators or barriers to each element of the project (eg, funding, human resources, expertise, etc.); and (d) consider actions that leverage facilitators and decrease barriers to enhance the effectiveness of the ES project.
As noted by Wilburn et al, 40 “The scope and magnitude of the project needs to be within the organization’s capacity” (p. 2). While this will overlap with factors such as timelines and resources, the scope will be contingent on the aim and/or purpose (why) the ES is being conducted. For instance, a stand-alone ES intended to address bariatric services within aged residential care or tobacco cessation interventions for youth may need to be more robust and exhaustive than an ES used to gather foundational information in a multi-component study.41,44 Similarly, an ES done by a graduate student for a master’s thesis is not likely to be the same scope as one done for a PhD dissertation or by an organization to propose a change in health service delivery. Identification of data sources and data collection strategies should align with the research question; although these steps are delineated in Phase #3, anticipation of time, resources, and access to information needs to be considered in early planning. For instance, identifying a limitation in available information (eg, understudied phenomenon) or challenges locating and accessing data early in the process might be established by conducting preliminary searches in early planning of the project.7,18,20,43
From the scope and identified data sources and data collection strategies, the project team can list each element involved and then brainstorm facilitators and any barriers, actual or potential, that may affect each element. Using a simple technique, such as Lewin’s Force Field Analysis, the facilitators (driving forces) and barriers (retaining forces) for each element can be identified and evaluated. 93 From this analysis, the project team can brainstorm mitigation strategies to enhance facilitating factors and/or decrease anticipated barriers; complete elimination of barriers may not be possible, or necessary but alternative solutions may be identified. For instance, if preliminary searches for literature yielded few results, expanding the number of bibliographic databases and other sources of gray literature in the search strategy might facilitate an increase in findings.7,88 At the same time, augmenting a web-based literature search with alternative sources of data (eg, access to documents through stakeholders, data from participants, etc.) may be considered as a means to decrease or work around a barrier related to availability of information.20,62
Phase #3: Developing an ES Protocol
In this phase of the RADAR-ES process, developing an ES protocol builds on the work and information gained in Phases #1 and #2. Methods and procedures are delineated for each step of the ES similar to conventions for traditional research to account for types of data to be collected, approaches to data collection, how data will be managed, and the processes through which each of these steps occur (see Figure 4).9,14,32,33,39,40 Intentional and mindful planning at this stage serves to enhance the rigor of the project, particularly in relation to dependability and credibility.11,32 It can also engage the knowledge, skills, expertise, and insights of both project team members and knowledge users to augment the robustness of the ES and enrich the final product of the study.13,32,40
Phase 3: developing an ES protocol.
The steps in Phase 3 are iterative in nature as the design is developed and may need to be revisited and/or revised once the project begins. Analogous to the emerging design principle inherent to qualitative research, early findings may reveal new data sources, require alternative data collection techniques, and/or prompt refinement of the research question.30,55,57,94 This aligns with a need for flexibility in ES design that has been identified in methodological references for ESs and by stakeholders with experience in conducting ESs.9,10,39 From our experience and that described by other authors, development of a formal workplan or protocol is recommended as a means to: (a) clearly delineate steps and the sequence of procedures in the project; (b) articulate roles and responsibilities of project team members and knowledge users; (c) set incremental goals for the project; and, (d) map out project timelines.32,39,40,49 From a practical perspective, we have found that a formal protocol at the outset of an ES project is helpful as a communication tool within the project team and, later (Phase #5), for knowledge translation (eg, methods section of reports and articles); a protocol is often also required for applications for funding, submissions to ethics boards, and proposals for graduate student research. Similar to other primary research initiatives, protocols can be published in advance of implementing the ES project to share the background and methods, and to receive peer feedback.20,24,41,49
When conceptualizing an ES design there are 3 important considerations. First, if the ES is part of a larger research project, the relationship of the scan and its findings to the design and findings of the overall undertaking must be established. In some cases, an ES may be a preliminary stage that is foundational to a larger research project12,19,41,43,89,90 and in other situations it may be a concurrent study to augment findings of the larger project.47,50,51,85 -87,95 An ES may also be incorporated into a mixed-methods design where findings from the ES are integrated and/or triangulated with findings from other study components;6,20,25 this particularly needs intentional planning informed by conventions of mixed-methods approaches to produce relevant and meaningful evidence and ensure that dimensions of rigor are upheld.56,57 Regardless of the relationship to the larger research project, design elements of the ES should factor in sequencing of study components, approaches to analysis, and both methods and procedures inherent to the research design.
A second consideration for an ES project design is whether a conceptual or theoretical framework would be used to inform elements of the design. In conventional research, conceptual or theoretical frameworks may be used to guide articulation of the focus (eg, background), data analysis, presentation of findings, and other aspects of the study.30,55 Use of frameworks and models have been described in relation to ES designs for these purposes, such as for practice change, chronic care, behavior change theory, access to care, supportive care, social determinants of health, and social ecology.7,20,23,41,43,50,51,87
Lastly, initial decisions on inclusion and exclusion criteria should be established by the project team to provide parameters for the ES and ensure relevance when selecting potential data sources. Such criteria would be made in relation to the research question and study purpose to establish whether internal and/or external data sources would be sought, and the eligibility of prospective informants, organizations, types of services or technologies, and other contextual elements to be considered for the ES.5,34,36 -38,40 As examples, eligibility criteria were developed by Atuluru et al 68 to select organizations using online platforms messaging for Monkey Pox, Castro et al 7 to identify appropriate respite care services, 7 and Kimber to determine appropriate training and education materials on family violence for health professionals. 90 These initial decisions on inclusion and exclusion criteria often help inform development of search strategies for literature and online web searches used as data sources and may be further refined when data collection starts in Phase #4.
Determine Data Sources
Similar to traditional research conventions, determination of relevant data sources and data collection methods can usually be derived from the research question and/or objectives for an ES.41,90 For instance, Kimber posed the research question “what online, publicly accessible training materials are available, and which aim to increase the knowledge, attitudes, skills and behaviors of Canadian physicians and social workers related to recognizing and responding to family violence in their practice encounters?” (p. 5). 90 From this research question, Kimber determined that generation of a list of organizations with involvement in family violence/education, Google searches for training materials, and hand searching of specific websites would yield appropriate data for the ES. 90
Data sources may take on many forms (see Table 4) and can broadly be categorized as internal (meso level) or external (macro level) to one’s organization.5,14,15,37,38 Further, data sources may be non-person (eg, websites, records, etc.) or person (eg, interviews and observations); this is critical to identify and plan for since application for institutional ethics review may need to be considered. As we previously noted, we regard an ES as a multi-method inquiry comprised of 2 or more different data sources and, at least, 2 different methodological approaches for data collection. To that end, choices of data sources may include a mix of internal and external and/or person and non-person sources. In Table 4 we present a range of potential non-person and person data sources that may fall into either internal or external domains.
Examples of Specific Data Sources Used in ESs.
The most common data sources for ESs are peer-reviewed and gray literature derived through literature searches and web-based information obtained through Google searches.2,32 To enhance the rigor of the ES process and the quality and robustness of findings, a structured and systematic search strategy should be developed. For literature searches, scoping review strategies described by Arksey and O’Malley (2005) and Joanna Briggs Institute provide a process for identifying and screening literature based on an established methodological framework,26,91 and elements of these frameworks have been used and/or adapted for ESs.7,48,49 Development of effective literature search strategies that include selection of search terms, key words, appropriate bibliographic databases, and parameters (eg, year range, language, etc.) are instrumental to a rigorous literature review, and benefits from inclusion of an experienced librarian in the project team.7,18,27,45,69,88,90 Online search strategies for locating websites, documents, and other data sources can be developed in much the same way for Google and other online search engines.22,27,49,69,88,90 However, there are limitations to use of Boolean operators (“AND” and “OR”) and search term combinations for online search engines and, often, the volume of information may require restriction to number of web page results reviewed (eg, first 100 results or first 10 web pages in Google).22,49,69,88,90
When considering the requirement for institutional ethics review, a general rule is that any information derived from human participants requires approval from an ethics board or panel; this particularly applies when findings are to be presented publicly through knowledge translation activities and may apply to both non-person and person data sources. 30 It is important to be mindful of the trade-off of quality and value of information if strictly selecting non-person data sources compared to the potential richness of data from person sources. 38 As evidenced in published ESs, many authors report an explicit stance on whether ethics approval was needed or not by their institutional review boards (IRB).7,12,20,23,41,43,51 From our experience, there can be variation in how IRBs understand and/or perceive ESs. In keeping with ethical comportment and respect for due diligence, we recommend that the project team be explicit in detailing data sources and collection strategies in their protocol and recommend seeking guidance on their protocol from their IRB in advance of data collection.
Identify Data Collection Methods
For each data source that is included in the ES an appropriate data collection method must be determined and delineated for the design.4,13,14,32,33,38 Given the broad range of options for both non-person and person data sources (Table 4), the project team should draw on established research conventions that best fit collection from each data source to help promote rigor and robustness of the ES process. For instance, as previously mentioned, use of a recognized scoping review framework is beneficial in determining data sources, but they also outline the steps for retrieving, screening, and selecting specific sources of information for inclusion for the ES.26,91 Aligned with this systematic approach, strategies such as assigning at least 2 reviewers to independently screen data during the retrieval and selection of both literature and web search materials have been used in ESs,21,47,49 thereby enhancing the rigor of the process.
One benefit to collecting information from non-person data sources is that it tends to be unobtrusive, in that it can mainly be achieved through online web searches using search engines (eg, Google), access to public websites, and other published electronic sources.37,38 However, it may sometimes be necessary to approach organizations for permission to access internal documents not in the public domain.20,38 In contrast, collecting data from person sources (eg, interviews, focus groups, and observations) tends to be regarded as being more intrusive to organizations requiring their permission, access, and time; depending on the circumstance and use of data collected, access to person data sources will need ethics approval.4,34,39
We recommend the use of applicable and established methodological approaches for collection of qualitative and quantitative data from person sources to inform the design of an ES; this will serve to enhance the rigor of the ES process, particularly credibility and auditability.30,60,78,81 In general, qualitative data gained through literature (peer-reviewed and gray) and website searches is collected using extraction criteria and tools (eg, templates) that the project team develops.7,20,21,44 Data through interviews, focus groups, and short-answer questions in surveys for ESs appear to be mainly descriptive in nature and data collection through interviews often involved a semi-structured guide or standardized script common to qualitative research methods.4,23,40,43,44,47 Few articles delineating ES methods referred to specific qualitative approaches or explicitly identified sources that informed the approach used 20 ; this has implications for assessing rigor, particularly dependability, of this aspect of an ES. Given the qualitative descriptive nature inherent to many ES designs, drawing on methods described by Sandelowski,96 -98 Creswell,57,94 Patton, 99 or similar foundational sources may be beneficial in outlining approaches to data collection.
Use of quantitative data through surveys was reported in 35% of ESs included in our scoping review 30 ; however, methodological references on ESs do not address the specifics of collecting quantitative data from participants.4,32 -34,39 Traditional conventions associated with quantitative research emphasize rigor in development of survey instruments, recommending that instruments with established validity and reliability be used. 30 As evidenced in Table 2 and our scoping review, 2 ESs explore a broad range of topics in a variety of contexts ESs that may mean an established or suitable instrument may be available. However, more often it appears that custom designed instruments have been developed and used for data collection in many ESs.41,42,51 To enhance rigor and preciseness of instruments used in ESs, we recommend that the project team membership include someone with quantitative research experience to help develop and test custom designed instruments; this expertise will also be beneficial for analysis in Phase #4. Instruments should be pilot tested in advance of data collection to ensure clear wording, accuracy in answering the research question, and consistency.30,51,57
Establish Data Management System
When considering data sources and overall project management, decisions must be made on how best to store and manage data while also ensuring access to team members. Further, consideration should be given for security of confidential information collected from participants in accordance with IRB requirements and thinking ahead to data analysis. 30 Depending on the scope of the ES project, the types of data sources used, and other elements of design, it is often necessary to have more than 1 type of software for data management, particularly if both qualitative and quantitative data are used. Reference management systems, such as Zotero™, may be used for organizing and screening data from bibliographic databases. 22 Textual data is commonly extracted and organized in tables using software, such as Microsoft Access, Excel spreadsheets, or Word documents;41,49,50 however, other advanced software applications, such as Covidence™ and NVivo™ may also be used.20,22 Covidence™ is an online platform that supports both a systematic review of literature and data extraction by multiple team members. NVivo™ can be used for data extraction from many data source formats for literature, interview transcripts, and other media. 22 Excel may also be used for capturing quantitative data and render to simple statistics, while Qualtrics™ and SPSS™ are examples that can be used for managing and analyzing more detailed data, such as from aggregated data and surveys.21,44,47,68
Phase #4: Assembling the Data
The fourth phase of the ES process is concerned with bringing the data together through analysis and synthesis, 2 processes that often interconnect in an iterative way, to render findings from the scan’s activities (see Figure 5). As findings evolve, the project team may elect to seek input from knowledge users and other stakeholders to verify and, potentially, expand on the results from the ES.
Phase #4: assembling the data.
Data Analysis
For each data collection strategy employed in the ES, an associated and appropriate analytical technique needs to be determined.13,32 Some of the key purposes of data analysis in ESs are to identify patterns, compare and contrast data points, establish relationships, and identify gaps. 14 None of the foundational sources for ES methods we reviewed and few studies reporting ESs have delineated explicit steps in data analysis or referenced established resources for to support their approach to analysis;7,18,21,22,43 -45,51 yet detailed and supported approaches to analytical methods are required to promote credibility and dependability of findings.60,78 Use of a framework in the analytical process was demonstrated by Richard et al 51 for mapping out access to care and Yu et al 80 for development of risk-stratified treatment recommendations.
Content and/or thematic analysis are common approaches described by authors of ES projects for qualitative sources of primary data (ie, interviews, literature, documents, and textual information from websites).7,12,18,23,32,43 -45 Specific approaches to content and/or thematic analysis we identified in ES literature included processes delineated by Braun and Clarke, 18 Elo and Kyngas,7,19,43,44 Hsieh and Shannon,22,44 and Schreirer. 21 Content analysis is a basic level of analysis that can be used for describing and quantifying phenomena related to a particular topic of focus.30,56,57 Often derived by coding content at a granular level, content analysis can be applied to any amount of qualitative data and is particularly helpful to categorize data from short answer responses in a survey and for deductive analysis when using concepts from an a priori framework.7,30,45,51,94,100,101 Coding that starts from content analysis can then be organized to create higher level categories and themes through various degrees of interpretation.30,56,57,94,100 -102 Braun and Clarke’s six-step approach to thematic analysis is well-established across several disciplines and is particularly useful for those new to qualitative data analysis and begins with initial coding.100,102 Similar to conventions of qualitative analysis, dimensions of rigor for ESs can be promoted through use of established analytical processes and strategies such as using transcribed interviews, immersing in the data, developing codes and themes through team processes, and developing a coding matrix that were demonstrated in several articles.12,18,22,43 -45,51
Quantitative data may be used as a primary data source in ESs and is often presented as descriptive or epidemiological statistics (eg, prevalence rates) in relation to the topic of focus.21 -23,32,44,46,47,51,68 Descriptive statistics reflected data from surveys, rating forms (eg, evaluation of observed data), measurements (eg, body mass indexes; frequency of events, anthropometric parameters, etc.), and characteristics related to population and context (eg, participants, geographical regions, practice setting, etc.).7,22,44,47,51,68,80 Forms of descriptive statistics reported in ESs included ranges, frequencies, means, standard deviations, and Chi-square tests.21,22,25,42,44,45,47 -49,51,69 Inter-rater agreement has also been reported as part of screening and observation processes.32,68
Depending upon the design, triangulation may be part of the analytical process for ESs whereby data from different sources may be brought together to strengthen and/or verify findings. 40 Triangulation of data from multiple data sources may be particularly helpful when there is sparsity in available information, an intent to augment one source of data with another, or to corroborate findings to illuminate a phenomenon in question.14,30,40,55,57 Examples of triangulation used in ESs were attaining a comprehensive overview of practice in development of a framework, combining qualitative and quantitative data in an explanatory design, and testing one source of data to another.12,44
Data Synthesis
This step of Phase #4 entails bringing together the results of data analysis to formulate and present findings of the ES and, ultimately, create new knowledge.13,32,39,40 As noted by Shahid and Turin, it is helpful to keep in mind the purpose of the scan when deciding which information to focus on; this is supported by Wilburn et al 40 who state that synthesis of results should be “meaningful conclusions as they relate to the focus area” (p. 4). Two specific strategies used in data synthesis for ESs include evidence mapping and development of an analytical matrix to map and group findings together as it relates to the research question and/or purpose.11,20,45 Creation of visual summaries, figures (eg, graphs and charts), and tables to organize and summarize findings were used to visualize results and, later, for presentation in knowledge translation activities.7,12,20 -22,25,43,44,48,51,62,68,80,87,90
Verification
Verification, often used synonymously with the concept validation, is an approach to promoting rigor in qualitative research and has been aligned with dimensions of validity and reliability associated with quantitative research.59,60,81 Morse et al 60 defined verification as “. . .the process of checking, confirming, making sure, and being certain. . .” (p. 17) through strategies such as ensuring methodological coherence, and other authors have described strategies such as member-checking, inviting stakeholder feedback, and peer-review to increase credibility and confirmability of findings.33,59,78 Member-checking has been reported for ESs to verify or validate data and findings with knowledge users, such as key stakeholders and organizations.7,42,44,62 Utilization of advisory committees to solicit feedback and input throughout the ES process has also been described.22,87 Triangulation, already noted as a strategy for data analysis, can also be utilized as part of the verification process and enhance credibility.40,81,94,103
Phase #5: Reporting the Results
The last phase of RADAR-ES consists of writing up and presenting the results and conclusions from the ES; however, planning for what and how information will be disseminated should be considered early in the planning phase.5,13,15,32,33,39,40 Determination for formality of reporting results and extent of knowledge dissemination will relate to the purpose and aim for the ES but must also be appropriate to the intended audience and knowledge users (eg, decision-makers, policymakers, funders, etc.).5,13,32,33,39,40 Timelines may play a part in deciding how information is to be presented, particularly if there is an urgency to share the results. 13 The 3 steps to reporting the results are writing, reporting, and knowledge translation (see Figure 6).
Phase #5: reporting the results.
Writing
In some situations, the format for drafting a report of results and conclusions may be predetermined, as in the case of specific templates of funding organizations, knowledge users (eg, government agencies), and other stakeholders.32,33,40 It may be that a technical format is required or is more suitable for the intended audience22,90 or that a goal is for publication in a peer-reviewed journal.,12,21,23,47,50 From our experience, knowing what written format in advance is beneficial for both consistency and efficiency, since preparation of tables or schematics as part of analysis can often be used to prepare a report, paper, or other form of knowledge translation. Furthermore, much of the protocol background and methods can readily be revised and adapted as part of the writing process.
With respect to formatting and style of writing, we have found it helpful to draw on examples from published literature sources (eg, samples in Table 2) to provide ideas on how best to present findings, insights, discussions or summaries, recommendations, and conclusions. Key elements to be mindful of are ensuring reports are customized to the audience to be concise, yet comprehensive, while considering both easy understanding and use of visual representations for information (eg, graphs, tables, schematics, etc.).5,32,39 As noted in Phase 4, creation of visual summaries for analysis and synthesis can be utilized for reporting and knowledge translation.
Reporting
To date there has been a lack of consistency in what information is reported out for an ES or the degree of detail required for a fulsome report. Predetermined templates and processes for ESs, such as those outlined by CADTH, 33 may provide guidance for reporting and reputable peer-reviewed journals generally outline requisite criteria for studies; however, in many cases what can be reported may be limited by word count restrictions. In the absence of a standardized guideline, some authors have adopted reporting guideline checklists, such as PRISMA (eg, systematic reviews) to support reporting findings for ESs.7,18,69 Currently, there is no established approach to appraising the quality of ESs. We advocate that the principles of ethical comportment, transparency, and rigor be adopted to both demonstrate application of established research conventions and quality in the implementation and presentation of ES processes and findings. This would necessitate providing as much explicit detail and supporting evidence, as practical, for the reader to assess the rationale, methods, results, and recommendations or conclusions. As we previously noted, some authors have opted to publish protocols in advance of their ES project that may help reduce word count when publishing results by providing background information and a more fulsome description of the methods and procedures. We believe that submitting the protocol to a reputable peer-reviewed journal will enhance rigor by adding credibility and transparency to the ES project and potentially benefit the project team through receipt of additional feedback and expertise.
Knowledge Translation
Knowledge translation is defined as the process of making research findings accessible by whomever are the knowledge users, whether people or organizations, and is generally concerned with how findings and recommendations are disseminated. Reports and presentations to immediate knowledge users, such as managers, stakeholder partners, and policymakers are common knowledge translation activities for ESs at the organization and government levels,32,33,40 while academic papers and conference presentations are frequently employed by researchers to reach of broader audience of knowledge users.13,40,41 As with knowledge translation for conventional research, other strategies have been used for disseminating information from ESs, such as use of narrative summaries, infographics, postings on online platforms, and sharing through networks.11,20,33,41,50 From our perspective, robust and rigorous ESs in HSDR can yield very valuable information that, if available and accessible to a broad range of knowledge users, can provide new knowledge and insights for change. To that end, drawing on innovative knowledge translation strategies to reach a broad audience should be considered from the onset of planning since some activities can be integrated throughout the ES process.
Conclusion
Through use of foundational sources for ESs, examples of ESs for HSDR in the literature, and original research exploring the use of ESs in healthcare contexts, we have developed and presented RADAR-ES, a 5-phase methodological framework to guide the planning and implementation of ESs (see Table 5). This framework is underpinned by 4 guiding principles to enhance the quality and robustness of an ES that includes the ecological context of the ES; project timelines; knowledge users; and ethical comportment, transparency, and rigor. In keeping with McMeekin et al’s third phase for developing a methodological framework, next steps will be to evaluate and refine RADAR-ES by receiving input from researchers and other knowledge users who utilize this framework for ESs in HSDR. RADAR-ES may serve as a foundation for development of tools to guide researchers in reporting and appraising the quality of ESs.
Summary of RADAR-ES Phases, Steps, and Recommended Actions.
We believe RADAR-ES will provide comprehensive guidance to researchers and other knowledge users in planning, implementing, and reporting ESs in HSDR. Although RADAR-ES advances ESs as a methodology and fills an important gap in the context of HSDR, we believe that it could potentially be applied and/or adapted to other disciplines and substantive areas of research. Given the apparent need for a methodological framework to both guide planning and implementing ESs in HSDR, RADAR-ES is a starting point for promoting consistency and quality in the use of ESs for knowledge development in health care.
Footnotes
Appendix
Guiding Questions for Planning an Environmental Scan.
| Principle | Prompt questions | |
|---|---|---|
| Ecological context of ES | Why is an ES the most appropriate method of inquiry for the project or study? What are the micro, meso, macro, and chronological factors (uncertainties, issues, and resources) that may affect ES completion and relevance? Are you able to manage any identified risks? |
|
| Project timelines | What are the estimated project timelines? What is the urgency of the ES? What is the schedule for ES completion (eg, start, tasks, milestones, end)? How will the timeline affect the rigor employed in the ES development? |
|
| Knowledge users | Do you have the key project team members, participants, and knowledge users involved in the study? How will these stakeholders be engaged throughout the ES planning, implementation, analysis, reporting, and knowledge translation? |
|
| Ethical comportment, transparency, and rigor | What ethical principles will guide the ES procedures and processes? Have ethical comportment and power dynamics been considered, if working with priority populations or those identified as vulnerable? What steps will be taken to ensure the methodological rigor of the ES protocol? How will transparency be addressed (eg, a description of the approach to rigor, research methods, analysis, and reporting of the procedures) throughout the ES process? Are there existing guidelines to drive the ES planning and report writing to enhance rigor and transparency? How will all the recommended elements of rigor (authenticity, confirmability, credibility, transferability, ethical concordance, and dependability) be embedded and reported in the ES? |
|
| Phase | Step | Prompt questions |
| Recognizing the issue | Identify the issue | What is the problem, issue, or opportunity being addressed? How clear is the information gap that needs to be addressed? How will stakeholders or knowledge users be engaged in this phase to better understand the importance of the ES and the likely impact? Have initial targeted searches been completed to understand the problem, issue, or opportunity and inform the ES search strategy, sources, and methods? Who is the target audience for the gathered information? |
| Determine the aim/purpose | How clear and concise is the ES aim or purpose statement? |
|
| Develop objectives | What are the objectives that need to be met to satisfy the ES goals? |
|
| Frame a research question | How clear and concise is the ES research question? |
|
| Assessing factors for ES | Select project team | |
| Identify available resources | What are the financial resources available to support the ES? |
|
| Determine feasibility and mitigate challenges | How clear is the team’s understanding of why the information is needed (eg, enhance knowledge, inform decision-making)? |
|
| Developing an ES protocol | Determine data sources | What existing and/or new data are required to answer the research question? Is it a combination of both? |
| Identify data collection methods | What will be the purpose of the information (eg, to explore or understand phenomena/experiences with a qualitive approach; describe facts/statistics, using surveys with a quantitative approach; or both with either multiple or mixed methods if a single-phase ES is part of a mixed methods study)? |
|
| Establish data management system | How will data be recorded, managed, and kept secure? |
|
| Assembling the data | Data analysis | How will a data analysis strategy be developed based on established methodological approaches? |
| Data synthesis | What methods will be used for data synthesis? |
|
| Verification | To what extent will the gathered information be relevant and sufficient to answer the ES research question? |
|
| Reporting the results | Writing | Who will take the lead in writing the ES? |
| Reporting | Will the findings be presented in an appropriate format for the ES target audience? |
|
| Knowledge translation | What is the plan for knowledge translation and dissemination of the ES results? |
|
Acknowledgements
We gratefully acknowledge Dr. Katherine Kelly, Research Scientist, Centre for Health and Community Research (University of Prince Edward Island) for her earlier contributions to the foundational work for RADAR-ES and for reviewing this submission. We also thank Olaide Fadare, College of Nursing (University of Manitoba) for her work as research assistant early in this project.
Ethical Considerations
As this was not a research study, no ethical approval was required for this project.
Consent to Participate
Not applicable.
Author Contributions
DAN, RA, PC, TK, and KB all contributed to the conceptualization and development of this methodological framework. Data extraction was conducted by DAN, LK, and PC, and all authors participated in analysis and synthesis of data related to the framework. DAN was lead writer with PC and KB contributing content, and all authors assisted in the development of appendices and final edits.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by an operating grant from the New Brunswick Health Research Foundation (NBHRF), now known as Research NB.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability Statement
Not applicable.