Abstract
Background
Health guidelines are important resources for informing decisions made by health practitioners, patients, caregivers, policymakers, and other interest-holders. Interest-holder engagement in health guideline development can improve the relevance of recommendations, promote considerations of equity, and support implementation. To facilitate efficient and effective health guideline development, there is a need for a comprehensive, global, evidence-based guidance for interest-holder engagement in guideline development. This paper synthesises the barriers and facilitators to interest-holder engagement in guideline development. Several systematic reviews have examined the topic of interest-holder engagement in guideline development, but few have explicitly focused on barriers and facilitators to engagement and the majority of these were related to patients. With increased recognition of the value of broad interest-holder engagement, understanding the barriers and facilitators to interest-holder engagement is key to developing a relevant and inclusive health guideline.
Objectives
The objective of this review is to identify and synthesise the reported evidence on barriers and facilitators to interest-holder engagement in health guideline development. We address this objective through two research questions: 1. What are the reported barriers to interest-holder engagement in health guideline development across the 18 topics of the GIN-McMaster checklist? 2. What are the reported facilitators to interest-holder engagement in health guideline development across the 18 topics of the GIN-McMaster checklist?
Search methods
We searched five major electronic databases (MEDLINE (OVID), Cumulative Index to Nursing & Allied Health Literature (CINAHL; EBSCO), EMBASE (OVID), PsycInfo (OVID), Scopus, and Sociological Abstracts. We also conducted an extensive grey literature search using the websites of agencies who actively engage interest-holders in research and/or guideline-producing agencies, such as PCORI, WHO and GIN. We searched from database inception up to the 26th September, 2022. Backward and forward citation tracking was performed on included articles to identify other eligible studies.
Selection criteria
We included primary research studies which qualitatively reported on the barriers or facilitators to interest-holder engagement in health guideline development. This included qualitative or mixed method research studies using methods such as interviews, focus groups, or surveys to collect participant experiences; case studies of existing programmes; and process evaluation studies. We excluded non-empirical publications including commentaries and editorials. We excluded publications with incomplete data, including conference abstracts and protocols. We defined interest-holders as “any individual or group who is responsible for or affected by health- and healthcare-related decisions”. We identified 10 types of interest-holders whose input can enhance the relevance and uptake of guidelines. We included studies that reported on the barriers or facilitators to engagement of one or several of these interest-holder groups at any step of the guideline development process.
Data collection and analysis
All identified citations from electronic databases were imported into Covidence for screening and selection. Documents identified through our grey literature search were managed and screened using an Excel spreadsheet. A two-part study selection process was used for all identified citations: (1) a title and abstract review and (2) full-text review. At each stage, teams of two review authors independently assessed all potential studies in duplicate using a priori inclusion and exclusion criteria. Data was extracted from each included article in duplicate and independently. We extracted information about study characteristics and methods. Additionally, we extracted qualitatively reported barriers and facilitators and conducted a framework analysis. We selected the Theoretical Domains Framework (TDF) for extracting our barriers and facilitator findings. Developed through a multidisciplinary consensus approach and subsequent validation, TDF consolidates overlapping behavioural theories into 14 domains encompassing 84 theoretical constructs, and provides a theoretical lens through which to view the cognitive, affective, social and environmental influences on behaviour and implementation. Two review authors conducted the analysis.
Main results
Our qualitative evidence synthesis identified a total of 51 findings, encompassing 23 barriers and 28 facilitators to interest-holder engagement in health guideline development. While the large majority of reported evidence is about patient engagement, many findings are applicable to all interest-holder groups and should be considered by guideline developers who aim to apply a multi-interest-holder engagement approach. Many organisations that produce guidelines lack the resources to support interest-holder engagement. By implementing an organisational interest-holder engagement model, guideline developers can ensure that they have the necessary human and financial resources in place. The interest-holders which are recruited to a guideline project should be knowledgeable and experienced, and guideline developers should consider diverse perspectives, roles, and personal characteristics. Using a network approach for identification and recruitment can help identify appropriate and committed interest-holders. To prevent tokenistic engagement, guideline developers can promote the meaningful engagement of interest-holders by providing practical, technical and emotional support. This includes providing training to empower interest-holders and improve self-efficacy and confidence in guideline development skills. Training should focus on evidence-based methods such as systematic reviews and evidence-to-decision frameworks. Engagement throughout all stages of guideline development supports more meaningful engagement compared to occasional involvement. A skilled moderator/chair can manage group dynamics and support the contributions of all those involved in guideline development. They can also offer explanations and promote plain language as-needed. Finally, many interest-holders have limited time to contribute to guidelines, and developers should establish flexible processes and timelines, and consider interest-holders preferences for in-person and online engagement activities.
Authors’ conclusions
Identifying and analysing barriers and facilitators to interest-holder engagement is important in order to optimise health guideline development. Knowing facilitators to engagement can support future interest-holder engagement work. More research is needed on the barriers that do not have potential facilitator solutions, to understand how to improve upon known challenges. This review calls for the development of training courses and tools to support and facilitate involvement of all interest-holders, both public and professionals. In addition, it calls for further research to be done with a focus on the dynamics of guideline development, perceived interactions within the group and their impact on decisions. Work should also be done to look at the implementation of guidelines and the influence of interest-holder engagement on uptake.
Plain Language Summary
Keywords
Background
The Problem, Condition or Issue
Health guidelines are systematically developed evidence-based statements which assist providers, recipients and other interest-holders in making informed decisions about appropriate health interventions (World Health Organization, 2014). Guidelines play a crucial role in the delivery of evidence-based medicine by providing a systematic examination of evidence conducted through a comprehensive search of the literature, critical appraisal of the quality of the evidence and interpretation of the findings in light of patients’ preferences and societal values (Guyatt et al., 2008). These guidelines assist health practitioners, patients, caregivers, policymakers, and other interest-holders to make informed decisions about health practice, public health and health policy. It is increasingly recognised that the engagement of multiple interest-holders in guideline development improves recommendation relevancy, uptake, implementation and sustainability (Esmail et al., 2015; Gagliardi & Brouwers, 2012; Moulding et al., 1999).
Interest-Holder Categories and Their Descriptions (Petkovic et al., 2023)
Major guideline developers, including the World Health Organization (WHO), the Canadian Institute for Health Research (CIHR) and the National Institute for Health and Care Excellence (NICE), as well as the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) Working Group recommend the inclusion of interest-holder groups in the guideline development and implementation process (Akl et al., 2017; Canadian Institutes of Health Research, 2014; National Institute for Health and Care Excellence (NICE), 2014; World Health Organization, 2014). They recommend this involvement to improve considerations of equity (Akl et al., 2017) and to ensure that the final guideline materials are useful and understandable for its end-users (World Health Organization, 2014). However, it is not clear how best to engage interest-holders or at what stage of the guideline development cycle. There is a need for the development of comprehensive, global, evidence-based guidance for interest-holder engagement in health guideline development that brings together the evidence (literature and interest-holder expertise) amassed to date.
The MuSE Consortium was established in 2015 to advance methods and approaches used in interest-holder engaged health research. The consortium is focused on engagement in health research and practice – including evidence synthesis and guidelines -- and includes more than 140 researchers, trainees and interest-holders from over 20 countries (Petkovic et al., 2020). As part of the consortium’s work, a working group was formed to develop global guidance on interest-holder engagement in health guideline development. The working group set out to conduct four concurrent systematic reviews to summarise the evidence on: (1) existing guidance for interest-holder engagement in guideline development, (2) barriers and facilitators to interest-holder engagement in guideline development, (3) managing conflicts of interest in interest-holder engagement in guideline development, and (4) measuring the impact of interest-holder engagement in guideline development. This review is the second in this series.
Description of the Phenomena of Interest
Topics Included in the GIN-McMaster Checklist for Guideline Development
Further, we define
In the context of this manuscript, Diagram of the Guideline Development Process. “The Steps and Involvement of Various Members of the Guideline Development Group are Interrelated and Not Necessarily Sequential. The Guideline Panel and Supporting Groups (e.g., Methodologist, Health Economist, Systematic Review Team and Secretariat for Administrative Support) Work Collaboratively, Informed Through Consumer and Interest-Holder Involvement. They Typically Report to an Oversight Committee or Board Overseeing the Process. For Example, While Deciding How to Involve Interest-Holders Early for Priority Setting and Topic Selection, the Guideline Group Must Also Consider How Developing Formal Relationships With the Interest-Holders Will Enable Effective Dissemination and Implementation to Support Uptake of the Guideline. Furthermore, Considerations for Organisation, Planning and Training Encompass the Entire Guideline development Project, and Steps Such as Documenting the Methodology Used and the Decisions Made, as Well as Considering Conflicts of Interest, Occur Throughout the Process” (Schünemann et al., 2014)
Specific mechanisms through which interest-holder engagement may influence guideline developmentBox 1
Logic Model of the Effects of Interest-Holder Engagement in Guideline Development
Interest-holder engagement is a complex process, and successful engagement requires both individual and organisational adoption. Organisational cultures and relationships act as a backdrop to these interest-holder engagement activities (Oliver et al., 2018). There are a multitude of individual- and organisational-level factors that can affect a interest-holder’s ability to engage in the guideline development process, or a guideline developer’s ability to effectively engage them.
Why It Is Important to do This Review
With increased recognition of the value of interest-holder engagement in guideline development, there is a need for expert opinion and evidence-based guidance on when and how to engage interest-holders. This review of barriers and facilitators to interest-holder engagement in guideline development will help fill this gap. It will build on existing reviews of barriers and facilitators, synthesize the current evidence-base, and examine a broad range of interest-holder groups.
To date, several systematic reviews have been conducted on the topic of interest-holder engagement in guideline development, but few are explicitly focused on barriers and facilitators to engagement. Oxman and colleagues (2006) conducted a series of reviews to advise WHO to ensure that health care recommendations are informed by the best available research evidence (Oxman et al., 2006). Relevant reviews in this series included group composition (Fretheim et al., 2006a), group processes (Fretheim et al., 2006b), and consumer involvement (Schünemann et al., 2006). However, these reviews were not systematic nor exhaustive, and prioritized existing syntheses over primary empirical studies. Relatedly, Oliver and colleagues synthesized theoretical literature to learn about the structure, processes, and environment of committees generally, which has applications for health guideline development (Oliver et al., 2018). Given the recent promotion of co-production in health and the rapid pace of involvement of diverse people in all types of research, these reviews are likely out of date and require updating to better capture this evolving landscape.
Much of the existing barrier and facilitator syntheses focus specifically on patients. Légaré and colleagues reviewed published and unpublished literature regarding the effectiveness of patient and public engagement programs in clinical practice guidelines and described the barriers and facilitators to these programs (Légaré et al., 2011). Barriers included recruiting challenges, lack of representation of certain patient and public groups, and participants’ lack of familiarity with medical and scientific terminology. Facilitators included the provision of training and support. Similarly, Grant and colleagues conducted a rapid review and qualitative evidence synthesis to make inferences about how feasible online methods for patient engagement would be during clinical practice guideline development (Grant et al., 2018). The authors highlighted important considerations for patient and caregiver participation related to skills, time and resources. Notably, both of these reviews only considered patient and public involvement and did not consider the engagement of other interest-holder groups such as healthcare providers or policymakers, for example. Recently, Selva and colleagues reviewed guidance documents for developing clinical guidelines to assess how these documents address the incorporation of patients’ views (Selva et al., 2017). They identified the need for additional guidance in this area.
Cluzeau and colleagues conducted a literature review and workshop to answer six questions related to interest-holder engagement in guideline development, including identification of the potential barriers and facilitators to integrating interest-holder involvement (Cluzeau et al., 2012; Kelson et al., 2012; Kunz et al., 2012). They defined interest-holders as “all those who have a legitimate interest in a guideline. They include healthcare professionals, patients and caregivers, public and private funding bodies, managers, employers, and manufacturers”. They noted that integrating different, sometimes competing, interest-holder perspectives can be challenging. Other barriers included issues with potential bias among interest-holder views, and the costly nature of the process. In contrast, they found that educating interest-holders and ensuring effective communication enabled effective interest-holder engagement.
In sum, there is a lack of systematic reviews that have synthesised barriers and facilitators to multi-interest-holder engagement in health guideline development, and our review aims to address this gap in the literature. The results of this review will be used to inform the development of guidance for interest-holder engagement in guideline development and implementation.
Objectives
The objective of this review is to identify and synthesise the existing evidence on barriers and facilitators to interest-holder engagement in health guideline development. We will address this objective through two research questions: 1. What are the barriers to interest-holder engagement in health guideline development across the 18 topics of the GIN-McMaster checklist? 2. What are the facilitators to interest-holder engagement in health guideline development across the 18 topics of the GIN-McMaster checklist?
The results of this review will be used by the MuSE Consortium to develop an Engagement Extension to the GIN-McMaster Checklist for guideline development (Petkovic et al., 2025). The findings of this review will assist organisations who develop healthcare, public health, and health policy guidelines to involve multiple interest-holders in the guideline development process to ensure the development of relevant, high quality, and transparent guidelines.
Methods
Reflexivity and Positionality: Interest-Holder Engagement in the Conduct of This Review
This review was developed and conducted by members of the MuSE Consortium, a group of individuals interested in advancing the methodology of interest-holder engagement in research and health care. The views of the authorship team reflect decades of work in the areas of health research, community engagement and clinical practice guideline development methodology. The protocol for this review was developed by an interest-holder research team that engaged investigators, trainees, providers of care, policymakers, peer review editors and patient representatives (Magwood et al., 2022). All members of the authorship team were provided with equal opportunities to be engaged in selection of studies, data extraction, critical appraisal, analysis and writing. The review authors acknowledged that their own perspectives on interest-holder engagement could influence the design and execution of this review, as well as the way the results shaped those perspectives. At the outset, all the authors agreed that some level of interest-holder engagement in health guideline development was beneficial and desirable. However, they acknowledged challenges in balancing pragmatism and the technical nature of guideline development with the aspirational and theoretical benefits of engagement. To mitigate these biases, the authorship team engaged in regular discussions with the broader MuSE Consortium to contextualize our findings within the GIN-McMaster Checklist for Guideline Development.
Criteria for Including and Excluding Studies
Types of Studies
We included primary research studies which qualitatively reported on the barriers or facilitators to interest-holder engagement in health guideline development. This included qualitative or mixed method research studies using methods such as interviews, focus groups, or surveys to collect participant experiences; case studies of existing programmes; and process evaluation studies.
We included published guidelines that reported lessons learned only if these statements were derived from empirical qualitative methods and the details of these methods are available in a published report. Mixed method and intervention studies (randomised or non-randomised designs) were eligible only if they qualitatively report barriers and facilitators to engagement, independent of their quantitative findings.
We excluded non-empirical publications including commentaries and editorials. We excluded publications with incomplete data, including conference abstracts and protocols. Finally, we excluded existing evidence syntheses (e.g., systematic reviews, meta-ethnographies, scoping reviews) as it would not be possible to extract the information relevant to our review, and reviewed their reference lists to identify any potentially eligible primary research articles.
Types of Participants
The population of interest is interest-holders in health guideline development. We define “interest-holders” as groups with legitimate interests in the health issue under consideration. The interests arise and draw their legitimacy from the fact that people from these groups are responsible for or affected by health-related decisions that can be informed by research evidence (Akl et al., 2024). Building on previous work, we have identified 10 types of interest-holders whose input can enhance the relevance and uptake of guidelines (Concannon et al., 2012, 2018; Tugwell, 2006). We included studies which reported on the barriers or facilitators to engagement of one or several of these interest-holder groups at any step of the guideline development process. For the purposes of this review, have grouped them as follows: Patients, caregivers, and patient advocates Public Providers of health care Payers/purchasers of health services Payers of research Policy makers Program managers Product makers Principal investigators and their research teams, and Peer-review editors/publishers
We recognise that within each of these categories, there is diversity and heterogeneity in types of interest-holders. For example, we recognise that patients and caregivers have differing characteristics and experiences, and their engagement brings diverse value to guideline development. The categories outlined above are for operational purposes and are based on prior research (Concannon et al., 2012, 2018; Tugwell, 2006). Studies which consider a single group of interest-holders or multiple groups of interest-holders were eligible for inclusion. If a study included multiple interest-holder groups, findings were coded for each interest-holder group separately.
Phenomena of Interest
Eligible studies must have described interest-holder engagement in any of the topics of the GIN- McMaster Checklist for Guideline Development (Schünemann et al., 2014). Eligible studies must have reported primary qualitative data in their results on barriers or facilitators to interest-holder engagement for at least one topic of the checklist: 1. Organization, budget, planning and training 2. Priority-setting 3. Guideline group membership 4. Establishing guideline group processes 5. Identifying target audience and topic selection 6. Consumer and stakeholder involvement 7. Conflict of interest considerations 8. Question generation 9. Considering importance of outcomes and interventions, values, preferences, and utilities 10. Deciding what evidence to include and searching for evidence 11. Summarizing evidence and considering additional information 12. Judging quality, strength or certainty of body of evidence 13. Developing recommendations and determining their strength 14. Wording of recommendations and of considerations about implementation, feasibility and equity 15. Reporting and peer review 16. Dissemination and implementation 17. Evaluation and use 18. Updating
Findings (Qualitative Data) of Interest
The primary findings of interest were any barriers and facilitators to interest-holder engagement in health guideline development. Barriers were defined as any variable or condition that impedes interest-holder engagement in health guideline development. Facilitators were defined as any variable or condition that promotes interest-holder engagement in health guideline development. To be eligible, these findings could be reported as first or second order constructs related to interest-holders’ and study team perceptions and experiences of barriers and facilitators to their engagement in guideline development. First-order constructs refer to original narratives (e.g., participant quotes and observations) in the reviewed articles that relate to a particular topic. The interpretations of these narratives by the authors of the articles (e.g., themes, author explanations or recommendations) are considered second-order constructs, while the synthesis of the review team are third-order constructs (Figure 3). In this review, both first order and second order constructs were considered eligible data for inclusion, extraction and analysis and could be located in an articles’ method, results, discussion or conclusion sections of the report. First, second and Third Order-Constructs of Qualitative Evidence
Setting
We did not place any restrictions on geographic settings. Studies focused on health guideline development from any country in the world were eligible for inclusion.
Language
No language restrictions were applied to our search, however all identified included studies were published in English.
Publication Date
No publication date restrictions were applied.
Search Methods for Identification of Studies
This review is part of a series of four reviews conducted by the MuSE working group on interest-holder engagement in guideline development. One comprehensive search strategy was developed and peer-reviewed in consultation with a medical librarian in accordance with PRESS guidelines (McGowan et al., 2016). A second medical librarian reviewed the search strategy.
Electronic Databases
The search combined index terms and keywords, combining concepts of interest-holder engagement (e.g., “patient participation”, “consumer involvement”) and guidelines (e.g., “clinical practice guideline”). Complete search strategies for all databases are available in Appendices 1-6.
We searched the following databases from database inception up to the 26th of September, 2022: MEDLINE (OVID) (Appendix 1) Cumulative Index to Nursing & Allied Health Literature (CINAHL; EBSCO) (Appendix 2) EMBASE (OVID) (Appendix 3) PsycInfo (OVID) (Appendix 4) Scopus (Appendix 5) Sociological Abstracts (Appendix 6)
Searching Other Resources
We conducted an extensive grey literature search using the websites of agencies who actively engage interest-holders in research and/or guideline-producing agencies. We searched the following resources between June 24 and July 29, 2020: Agency for Healthcare Research and Quality’s (AHRQ) Canadian Institutes of Health Research (CIHR) Strategy for Patient-Oriented Research (SPOR) National Institute for Health and care Research (NIHR) Be Part of Research Guidelines International Network (G-I-N) INVOLVE National Institute for Health and Care Excellence (NICE) Patient-Centered Outcomes Research Institute (PCORI) American Academy of Paediatrics Australia’s National Health Medical Research Council (NHMRC) World Health Organization (WHO), including Latin American and Caribbean Health Sciences Literature (LILACS)
We solicited suggestions for additional grey literature sources from the members of the MuSE Consortium. Backward and forward citation tracking was performed on included articles using the Citation Chaser tool (Haddaway et al., 2021) to identify further eligible studies. We also reviewed the reference lists of relevant reviews to identify eligible primary studies for inclusion.
Selection of Studies
All identified citations from electronic databases were imported into Covidence software for screening and selection. Documents identified through our grey literature search were managed and screened using an Excel spreadsheet. A two-part study selection process was used for all identified citations: (1) a title and abstract review and (2) full-text review. The selection process was piloted by senior review authors (O.M., J.P., J.K., L.L., A.R., P.A.) and a moderate to very good level of agreement (Cohen’s Kappa range 0.43-0.87) was achieved before all studies were assessed for eligibility, independently and in duplicate. At the title and abstract stage, reviewers assessed whether the article described health guideline development. If yes, articles were retained for full-text review. Reviewers noted that it was often difficult to assess whether the article focused on interest-holder engagement from the abstract alone. At the full-text review stage, reviewers confirmed that the topic of the article was health guideline development, and further assessed for two inclusion criteria; namely that (a) the article described the engagement of at least one interest-holder group in guideline development, and (b) that the article described barriers or facilitators to engaging interest-holders. We resolved any disagreements through discussion during weekly meetings. We produced a PRISMA flow diagram which reports the number of studies included and excluded at each stage. Reasons for exclusion have been provided for all studies assessed at full-text.
Details of Study Coding Categories
Data were extracted by two review authors independently and in duplicate according to a standardised data extraction form. Disagreements were discussed and resolved by a third reviewer. The data extraction form was pilot tested by two review authors and coding categories were clarified iteratively, as needed. Data were extracted on basic study characteristics and methods including: Study aim/objective Study setting (i.e. location where the guideline development took place, and geographical location) Type of guideline (e.g., clinical practice, public health, or health system guideline, specific condition(s) addressed) Population and participant characteristics, including categorisation according to interest-holder group and equity-relevant characteristics according to PROGRESS-Plus (O’Neill et al., 2014). PROGRESS-Plus is an acronym referring to Place of Residence, Race/ethnicity, Occupation, Gender, Religion/culture, Education, Socio-economic status, Social capital/networks) and other important factors which impact on health equity (e.g. disability, sexual orientation, age). Recruitment methods Data collection methods Analysis methods
In addition, we extracted information on study: Limitations Recommendations Conclusions Funding Conflict of Interest
Data Extraction Categories From the Theoretical Domains Framework (Atkins et al., 2017)
We extracted interest-holders’ (first-order constructs) and author perceptions and experiences (second-order constructs) regarding the barriers and facilitators to engagement in health guideline development from the results and discussion sections, including participant quotations from interviews or focus groups, excerpts or quotations from documentary analysis, narrative descriptive summaries, author hypotheses, explanations and recommendations, themes and sub-themes. Discrepancies in the data extraction process were resolved through discussion.
Analysis
Our review included studies employing a qualitative approach to inquiry. We used the ‘best fit’ framework method as a systematic and flexible approach to analysing the qualitative data (Booth & Carroll, 2015; Carroll et al., 2013; Gale et al., 2013). Framework-based synthesis, using the ‘best fit’ method, is a highly pragmatic and useful strategy for analysing a range of questions and is supported by guideline development literature (Dixon-Woods, 2011; Flemming et al., 2019). Framework analysis is a five-stage process that includes familiarisation with the data, identifying a thematic framework, indexing (applying the framework), charting and mapping, and interpretation (Ritchie & Spencer, 1994).
Data were analysed deductively, using the TDF domains to generate a framework of preliminary themes. Whilst the domains are purposefully designed to be broad groupings of the possible factors to influence behaviour, the intent is to explore the important domains in further detail. Two review authors (O.M. and A. Raymond) examined the data that were extracted under each TDF domain. The two authors identified and came to agreement on recurring concepts to develop the initial themes. Disagreements were resolved through discussion. When necessary, the authors returned to the full texts of the included studies to verify the perspective and context of the extracted data. After developing the initial thematic findings, all themes were circulated to the authorship team for comment. In a series of iterative drafts, the themes were refined and expanded upon. An illustrative quote was selected for each theme and the authorship team decided to organize themes using tabular and narrative approaches.
All identified key findings were mapped to the 18 topics of guideline development process outlined by the GIN-McMaster checklist (Schünemann et al., 2014). Through team discussions, we opted to remove topic 6 (Consumer and interest-holder engagement), as this project moves engagement from a stand-alone topic to a cross-cutting consideration for guideline developers to consider throughout each step of guideline development. This change is reflected in our published GIN-McMaster Guideline Development Checklist extension for engagement (Petkovic et al., 2025). An initial categorization was proposed by the lead author (O.M.) and then iteratively revised by the authorship team. We present this information as a narrative and series of tables, indicating the barriers and facilitators that exist for each interest-holder group and for each step of guideline development. The findings of this mapping exercise will be combined with the results of the other reviews in this series (Khabsa et al., 2022; Lytvyn et al., 2024; Petkovic et al., 2022) to develop international guidance on interest-holder engagement in guideline development.
Critical Appraisal
Two review authors assessed the methodological rigour of all included studies independently and in duplicate. We assessed the quality of qualitative studies and the qualitative components of mixed method studies using the Critical Appraisal Skills Programme (CASP) qualitative appraisal research tool (Singh, 2013). Discrepancies in the critical appraisal process were resolved by a third reviewer.
Confidence in Findings
We conducted GRADE-CERQual assessments to determine the confidence in the qualitative findings. Assessments were conducted by one review author (P.C.) and reviewed by a second author (O.M.). GRADE-CERQual is an assessment of the extent to which the review finding is a reasonable representation of the phenomenon of interest, and results in four levels based on the findings relevance, methodological limitations, coherence, and adequacy of the data (Lewin et al., 2018): High confidence in the evidence: It is highly likely that the review finding is a reasonable representation of the phenomenon of interest. Moderate confidence in the evidence: It is likely that the review finding is a reasonable representation of the phenomenon of interest. Low confidence in the evidence: It is possible that the review finding is a reasonable representation of the phenomenon of interest Very low confidence in the evidence: It is not clear whether the review finding is a reasonable representation of the phenomenon of interest.
GRADE-CERQual assessments for each finding are available in Appendix 7.
Results
Results of the Search
Our systematic search identified 55,364 citations. After removal of duplicates, we screened 31,505 articles by title and abstract. Of these, 731 were retrieved for full-text review.
Additionally, our citation analysis and grey literature search identified 83 reports which warranted full-text review. After review of all articles against a priori inclusion criteria, we included a total of 34 studies published in 40 reports. Thirty-eight reports were identified through database searching and 2 reports were identified via other methods (Figure 4). PRISMA Flow Diagram
Included Studies
Characteristics of Included Studies
Place of residence: All included studies except for one (Kredo et al., 2018) were conducted in high-income countries: Canada (n = 7), UK (n = 7), USA (n = 5), Netherlands (n = 4), Finland (n = 2), Europe more broadly (n = 3) and Australia (n = 1). Four studies collected data internationally, but were managed by a project team based out of Canada (n = 3) or Australia (n = 1). Kredo et al. (2018) was conducted in South Africa, an upper-middle-income county with significant disparities between residents. In this study, one participant explained how South African provinces have vastly different resources, cultures and infrastructure, and that a failure to accommodate these diversities could have dire consequences (Kredo et al., 2018). Notably, no evidence was identified on the barriers or facilitators to engaging interest-holders originating from low-income countries.
One study recruited patient representatives with traumatic brain injuries from a community-based association, and highlighted that this association was chosen because it encompassed rural and urban areas (Lamontagne et al., 2021). Of the 16 patients included in the study, 9 (56.3%) lived alone and 7 (43.4%) lived with close relatives. Only one other study reported on the living situation of their participants; which took place within the psychiatric ward of a veterans hospital and recruited in-patients who resided on the ward (Lanza & Ericsson, 2000).
Race/ethnicity/ancestry/language/culture: Six studies reported on the race of their participants: 4/6 reported that the majority of their sample identified as white (range 87.5-100%) (C. Armstrong et al., 2019; M. J. Armstrong et al., 2020; Harding et al., 2010; Lanza & Ericsson, 2000). One study reported the majority of their sample was African-American (67%, n = 10) (M. J. Armstrong et al., 2017), and in another study one consumer self-identified as belonging to a minority ethnic group (Synnot et al., 2022). In most studies, the ability to communicate in English was a requirement for study participation. One study was conducted among French-speaking individuals living in a majority Francophone context (Lamontagne et al., 2021).
Occupation: Participants of the included studies covered 9/10 MuSE interest-holder groups, including: (i) Patients, caregivers, and patient advocates, (ii) Public, (iii) Providers of healthcare, (iv) Payers of health services, (v) program managers, (vi) policymakers, (vii) Product makers, viii) peer review journal editors and (viii) Principal investigators and their research teams. No studies included payers of research. One study reported a median of 9 years (range 1-26 years) of professional experience in the guideline topic area (fertility care) (Den Breejen et al., 2014), whereas another study reported that employment status among patient representatives varied (Köpke et al., 2018).
Gender and sex: Thirteen studies reported on the gender or sex of their participants. In 7/13 studies, the samples were described as majority female or women (range 56-87%) (C. Armstrong et al., 2019; M. J. Armstrong et al., 2017; Biggane et al., 2022; Brouwers et al., 2017; Köpke et al., 2018; Lindsay et al., 2020; Synnot et al., 2022). Three studies reported their samples as the majority being males (M. J. Armstrong et al., 2020; Lamontagne et al., 2021; Lanza & Ericsson, 2000). In three additional studies, the samples had equal representation from males and females (Chartres et al., 2020; Den Breejen et al., 2014; Harding et al., 2010). Notably, no studies reported on diverse gender identities.
Religion: Not reported.
Education: While infrequently reported, three studies described the level of education of their participants. In their study with cancer patients, cancer survivors, and their families/caregivers, Brouwers et al. described that 85% (n = 34) had post-secondary degrees or diplomas (Brouwers et al., 2017). Similarly, Köpke et al. reported that 57% of patients and 64% of caregivers had tertiary education (Köpke et al., 2018). In contrast, Lanza & Ericsson. reported that most in-patient residents had either high school or college education levels (Lanza & Ericsson, 2000).
Socio-economic status: Not reported.
Social capital: Not reported.
+ Age: One study explicitly aimed to develop a tool for guideline developers to take children’s perspectives into account and facilitate children’s participation in the process of guideline development (Schalkers et al., 2017). However, no children were included as participants in the study and data was collected from professionals involved in guideline development. An additional 8 studies reported on the age of their participants, varying in age from mid 20s to late 70s. In 4 studies, the majority of the sample was described as “middle aged” (40–60 yrs) (Den Breejen et al., 2014; Köpke et al., 2018; Lamontagne et al., 2021; Lanza & Ericsson, 2000), and in 3 studies the majority of the sample was described as “older” (60+ years) (M. J. Armstrong et al., 2017; Brouwers et al., 2017; Synnot et al., 2022). One study reported the ages lay members (mid 20s - late 60s) and health professionals (30–65 years) separately (Biggane et al., 2022).
+ Disability: Two studies aimed to engage people living with disability. In their study engaging patients with dementia, Armstrong et al. (2020) recruited four patient representatives. One of these was an individual with mild cognitive impairment/early Alzheimer’s disease, who was able to fully participate alongside their spouse. In another study, Lamontagne recruited patients with traumatic brain injury, and 10/16 (62.5%) of patient representatives had a severe traumatic brain injury (Lamontagne et al., 2021).
No other dimensions of “PLUS” (personal characteristics associated with disadvantage) were reported in the included studies.
Excluded Studies
We excluded 693 citations after full-text review. Studies were excluded for the following reasons (Figure 4): Did not report on interest-holder engagement (n = 202); not related to guideline development (n = 158); wrong study design (e.g., editorials, commentaries) (n = 102); reports of single guidelines [e.g., which described the development process but did not empirically explore barriers/facilitators to engagement] (n = 102); duplicates (n = 16); topics not related to health (n = 2), and ‘other’ (e.g., poster abstracts) (n = 17). Additionally, 94 citations were excluded because they do not report barriers or facilitators to engaging interest-holders in guideline development, but were considered for inclusion in other reviews in this series (Khabsa et al., 2022; Lytvyn et al., 2024; Petkovic et al., 2022).
Methodological Quality of Included Studies
We critically appraised studies using the CASP checklist for qualitative studies. A summary of the critical appraisal is presented in Figures 5 and 6. All studies, except for one (Carter et al., 1995), reported clear objectives. Generally, the methods selected to collect data were appropriate to collect the subjective experiences of participants (e.g., interviews, focus groups, surveys), but the strategies to recruit these participants were not always explicit or relied on convenience/purposive samples. Therefore, it is possible that the perspectives collected may not be generalizable to all interest-holder groups or all guidelines. In some retrospective case studies (e.g. (Chalmers et al., 2017)), the data is derived from guideline developer perspectives only (e.g., presented as “lessons learned”). The most common weakness across all studies was a lack of reflexivity/positionality by the study authors. Often, researchers collected data during or after guideline development, and the study authors overlapped with the guideline secretariat (guideline developer) - therefore the authors collecting data may have had prior relationships with the participants, and the authors did not discuss their potential influence on participant responses. In many studies, the reporting of their analytic methods and approval of an ethics review board were unclear. Methodological Quality of all Included Studies Summary of Methodological Quality
CASP Questions: 1. Was there a clear statement of the aims of the research? 2. Is a qualitative methodology appropriate? 3. Was the research design appropriate to address the aims of the research? 4. Was the recruitment strategy appropriate to the aims of the research? 5. Was the data collected in a way that addressed the research issue? 6. Has the relationship between researchers and participants been adequately considered? 7. Have ethical issues been taken into consideration? 8. Was the data analysis sufficiently rigorous? 9. Is there a clear statement of findings? 10. How valuable is the research?
Synthesis of Findings
Barriers to Interest-Holder Engagement in Guideline Development
Facilitators to Interest-Holder Engagement in Guideline Development
Organization, Budget, Planning and Training
Barriers
A Lack of Human and Financial Resources at the Organisational Level Limited the Collaborative Development of Recommendations (CERQual: Moderate Confidence, n = 6 Studies)
Resource constraints were a general concern across several studies, which acted as a barrier to engaging patients, caregivers, patient organisations, and providers of care. Three studies specifically recognized the high level of human resources necessary to coordinate interest-holder engagement efforts (Den Breejen et al., 2014; Ilott et al., 2006; van Wersch & Eccles, 2001), citing the need for dedicated paid staff or coordinator positions to handle the logistics of consultations, which was particularly challenging for smaller organisations (Ilott et al., 2006). Specific engagement methods, such as surveys, interviews or focus groups, were identified as being desirable to capture the patient perspective in guidelines (M. J. Armstrong et al., 2020; Brouwers et al., 2017), but not all organisations would have the resources for such work (Hämeen-Anttila et al., 2016).
Guideline Development Organisations Have Unclear Goals Related to Patient and Public Engagement (CERQual: Very Low Confidence, n = 2 Studies)
Guideline developers and organisations often lacked clear goals about patient and public engagement, being described as “largely implicit or articulated in vague terms […] which makes it difficult to assess their success or failure” (Boivin et al., 2010). Patients who are engaged in guideline development may have different understandings of the goal of their involvement; in one study patients believed that their involvement was a mechanism to change their own healthcare and disclose personal health problems, which derailed the guideline development group from its true purpose (Lanza & Ericsson, 2000).
Guideline Developers Deliberately Exclude Patients From Guideline Development (CERQual: Low Confidence, n = 3 Studies)
Guideline developers may choose not to engage patients because they feel like the clinical practice guideline topic is too complex to require patient input (van de Bovenkamp & Zuiderent‐Jerak, 2013) or because they feel that patient contributions are too general for the question the specific guideline is trying to answer (Brouwers et al., 2018). Patient organisations do not always see the added value of participation either, as they may think that professionals already have the knowledge needed to develop a good guideline, and patients provide little added-value (van de Bovenkamp & Zuiderent-Jerak, 2013). In one study, guideline developers intentionally had no formal patient or public involvement to prevent tokenism (Selby et al., 2017).
Facilitators
Guideline Development Organisations Should Implement Formal Interest-Holder Engagement Models Into Their Everyday Operations (CERQual: Moderate confidence, n = 6 Studies)
To facilitate the engagement of providers of care, patients and caregivers, several professional bodies and organisations invested considerable resources in interest-holder relations. Clear, effective organisational leadership was an important enabler for interest-holder-engaged work (Walker et al., 2022). For example, incorporating stakeholding activities into their business plan and including these in the job description of professional guideline development coordinators (Ilott et al., 2006). This can facilitate engagement as the interest-holder engagement process becomes a part of the organisation’s everyday work. Guideline developers may also develop and incorporate engagement models into their guideline development (organisational) processes for future use (Ilott et al., 2006; van der Ham et al., 2014). This model sometimes results from patient and interest-holder engagement being a criterion to get development grants (van de Bovenkamp & Zuiderent‐Jerak, 2013).
Two engagement models have been evaluated: The Heart and Stroke Foundation of Canada developed an inclusive and integrated model of patient engagement named the Community Consultation and Review Panel (CCRP) (Lindsay et al., 2020). The goal of the CCRP model was to facilitate development of recommendations that are grounded in real-life experience and the unique needs of patients while maintaining scientific rigour. The model provided members with opportunities to regularly review and offer feedback on draft recommendations from the scientific writing group, and provided context related to their values and preferences (Lindsay et al., 2020). Similarly, a team of researchers from RAND and clinicians, caregivers, and patients from the Parent Project Muscular Dystrophy (PPMD) developed the RAND/PPMD Patient-Centeredness Method (RPM). The RPM is an online modified-Delphi approach to patient engagement that allows patients and their representatives to provide input on the patient centeredness of draft guideline recommendations (C. Armstrong et al., 2019). This model has been suggested for use throughout all stages of guideline development to continuously engage a panel of patients and their representatives (C. Armstrong et al., 2019).
Provide Interest-Holders With Financial Compensation And/Or Professional Accreditations (CERQual: Moderate Confidence, n = 9 Studies)
Guideline development organisations may facilitate interest-holder involvement by budgeting for and obtaining funds to financially compensate interest-holders, particularly for patient representatives as well as caregivers, patient organisations, and providers of care. This may include reimbursement for travel, printing, and other costs (e.g. childcare), as well as honoraria (financial support) to cover their time (M. J. Armstrong et al., 2017; Boivin et al., 2010; Brouwers et al., 2017, 2018; Chalmers et al., 2017; Duff et al., 1996; Ilott et al., 2006; Jarret, 2004). For providers of care, alternative compensation in the form of professional accreditations (e.g. CME) could be considered (Ilott et al., 2006). For all interest-holders, inclusion in publications if they meet the criteria for authorship should also be considered. One study suggested that covering expenses and providing compensation can facilitate engagement of patients/public from diverse identities (e.g. from different geographies, race/ethnicity, culture, language, etc.) (Synnot et al., 2022).
Guideline Developers Should Provide Training for Patient and Public Interest-Holders (CERQual: Moderate Confidence, n = 12 Studies)
Several studies highlighted the need for training or education and support for all interest-holders, but particularly patient and public members, to facilitate better understanding of the guideline development process (Biggane et al., 2022; Boivin et al., 2010; Brouwers et al., 2017, 2018; Chalmers et al., 2017; Duff et al., 1996; Hämeen-Anttila et al., 2016; Jarret, 2004; van de Bovenkamp & Zuiderent‐Jerak, 2013). Training on how to effectively participate could be provided before and during involvement. A study by Daraz et al. (2019) found that pre-engagement training may be especially beneficial for patients as this can help assist with incorporating their input and preferences early into the guideline development process. They recommended that the training should be delivered prior to the first guideline panel meeting and before the final recommendation deliberation (Daraz et al., 2019). Training could include information about clinical guidelines, research methodology, evidence frameworks, and basic statistics. Daraz et al. (2019) described a training curriculum targeting patients’ knowledge, skills, and attitudes based on content from the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group and readability principles. In a webinar, the focus was on didactics and concepts, and in a workshop, the focus was on roleplay and practical issues (Daraz et al., 2019). Another study suggests that training could come in the form of orientation sessions by teleconference or in-person, by watching a video of a guideline development group in action, or through mentoring opportunities from other patient/public representatives (Jarret, 2004). Patient and public members should also be advised of various resources and support available and be provided with examples illustrating the impact of lay members in previous guideline developments (Biggane et al., 2022). Guideline development group members should be notified of the level of training and resources that patients receive before meetings (Brouwers et al., 2018). A study by Chartres et al. (2020) suggested that training all guideline development group members who are unfamiliar with evidence frameworks should be an ongoing process, and Armstrong et al. (2020) suggests this promotes co-learning.
Priority Setting
We did not identify any barriers and facilitators to interest-holder engagement specific to the priority setting stage of guideline development.
Guideline Group Membership
Barriers
Challenges Identifying “Representative” Patients to Engage in Guideline Development (CERQual: Low Confidence, n = 10 Studies)
Studies reported that identifying patients, caregivers and patient representatives to participate in clinical practice guideline development is a challenge (M. J. Armstrong et al., 2020; Biggane et al., 2022; Kim et al., 2021). There is some debate as to whether patients involved in guideline development should represent the larger patient population or simply provide their own unique patient perspective. In several studies, guideline development groups were uncertain about whether patients were being properly represented (Brouwers et al., 2018; Chalmers et al., 2017; Hämeen-Anttila et al., 2016; Lanza & Ericsson, 2000; van der Ham et al., 2014). One article specifically expressed concerns regarding patient proxies (such as caregivers or healthcare professionals) who may be included within a guideline development group to represent the patient’s interests, stating “interests and needs of patients [are] sometimes distinct to those of [other interest-holders]” (Biggane et al., 2022). Guideline development group members pointed out that it may be difficult to find appropriate persons from the target group who would be capable of representing the larger patient population. Consequently, balancing representation and interest-holder opinion was difficult in participatory development processes (Walker et al., 2022). In two studies, it was difficult to represent the diversity of the target group in the guideline development process with respect to ethnic diversity and health conditions (Kim et al., 2021; van der Ham et al., 2014). Armstrong et al. (2020) found that the experiences and perspectives of patients included in guideline development may be different from those who choose not to participate in the guideline development process, and emphasised that self-selection may occur and that diverse patient perspectives may be insufficiently articulated.
Patients Perceive ‘Tokenistic Engagement’ Which Limits Their Ability to Meaningfully Contribute to Guideline Development (CERQual: Low Confidence, n = 7 Studies)
Seven articles identified tokenistic engagement, that is, offering engagement as a gesture, as a challenge to meaningfully engaging patients (M. J. Armstrong et al., 2017; Atkins et al., 2013; Biggane et al., 2022; Brouwers et al., 2017; Daraz et al., 2019; Jarret, 2004; Murad et al., 2019). Instead of being equal members of guideline development groups, some patients expressed frustration and felt they were included merely to fill a quota (Biggane et al., 2022). This was described as being a “tokenized member of the team” rather than a fully integrated member (Brouwers et al., 2017) and highlighted the need for strategies to guard against tokenism, ensure that all members contributed equally to discussion and decision making, and ensure that the patient voice was adequately captured in the final guideline (M. J. Armstrong et al., 2017; Atkins et al., 2013).
Language Fluency May Limit the Representativeness of Patients and Their Ability to Engage in Group Processes (CERQual: Low Confidence, n = 3 Studies)
Three studies highlighted the potential for language and manner of speaking to become barriers to patient engagement (M. J. Armstrong et al., 2017; Chalmers et al., 2017; Piggott et al., 2021). For example, there may be a requirement for patients to speak only English in order to participate in guideline development meetings, leading to lack of representation from other linguistic groups (Chalmers et al., 2017). Community members (patients and the public) also expressed concern that physicians from other countries who are members of the guideline development group may have accents or speak differently, making it difficult for them to understand discussion (M. J. Armstrong et al., 2017). Interest-holders [guideline methodologists and content experts] have reported that different cultures and languages can have a major impact on group processes and that respect and consideration for this should be given by all guideline development group members (Piggott et al., 2021).
Facilitators
Patient/Public and Wider Interest-Holder Selection Should Consider the Representativeness and Diversity of Membership (CERQual: Low Confidence, n = 13 Studies)
Several studies mentioned the importance of recruiting ‘appropriate’ or ‘representative’ patients to participate in guideline development (M. J. Armstrong et al., 2017; Boivin et al., 2010; Brouwers et al., 2018; Chalmers et al., 2017; Den Breejen et al., 2014; Harding et al., 2010; van der Ham et al., 2014; van Wersch & Eccles, 2001). Clear eligibility criteria is a facilitator to engaging people who represent the types of lived experience that are sought by the guideline developers; a “job description” may help to specify the knowledge, experience and social skills needed (Chalmers et al., 2017). Online modified-Delphi processes could be used to propose and prioritise desired characteristics of guideline development group members, including the type of expertise or experience group members should have (C. Armstrong et al., 2019). Involving multiple patient representatives or patient advocates may facilitate peer support and representation (Chalmers et al., 2017).
Project leaders may consider diversity as they are recruiting - this includes perspectives, roles, ethnicities, and other differences within the patient group. In order to properly consult marginalised patient sub-groups, it may be necessary to pay special attention to their inclusion. A study by Armstrong et al. (2020) found that local organisations could help to identify relevant patient representatives, as this could aid in overcoming recruitment difficulties as well as limit the use of resources required for their engagement. Using this recruitment approach, patient interest-holders had at least a basic knowledge of the relevant medical terminology (M. J. Armstrong et al., 2020). An additional article emphasised the importance of establishing respectful partnerships with organisations that work with or represent diverse groups, as they can assist with recruitment (Synnot et al., 2022). Power imbalances between members (e.g. between content experts and evidence review groups) could also potentially be minimised by paying attention to differences in age, experience, gender, or region and intentionally selecting members with diverse experiences and characteristics (Chartres et al., 2020). One study highlighted that special considerations would be needed to engage children in guideline development, as “clear requirements to give children a voice in the process of guideline development do not exist” (Schalkers et al., 2017).
The Recruitment of Patient Representatives Should Prioritise the Identification of Knowledgeable and Experienced Patient Representatives (CERQual: High Confidence, n = 6 Studies)
Many articles suggested that guideline development groups should recruit patients that are familiar with the guideline topic and process (M. J. Armstrong et al., 2017; Batalden et al., 2015; Brouwers et al., 2018; Hämeen-Anttila et al., 2016; Kim et al., 2021; van der Ham et al., 2014). Patient panellists should be knowledgeable, have relevant expertise, and be willing to actively contribute and ask questions (M. J. Armstrong et al., 2017). Several potential groups were identified – such as patient trainers, peer support patients, and patient representatives in hospital ethics committees – that could be engaged without extra training because they are already experienced in illustrating patient views (M. J. Armstrong et al., 2017). One study shared the perspective of a ‘professional’ patient representative, whose professional role provided additional insight and enabled her to share knowledge developed through her work (Biggane et al., 2022). This professional background and role enabled her to be involved throughout the clinical guideline development process, in a way that was different from other patient and public members (Biggane et al., 2022). Patient advocates, who had experience sharing their opinions in multidisciplinary groups and committees, felt more confident in guideline development groups (van Wersch & Eccles, 2001). The ‘helicopter view’ that certain patient representatives have is considered beneficial, meaning that they have an overview of a wide range of experiences from a range of service users (M. J. Armstrong et al., 2017; Brouwers et al., 2018; Hämeen-Anttila et al., 2016; van der Ham et al., 2014). One study explicitly stated that guideline developers should aim to involve patients with prior engagement or guideline experience (Kim et al., 2021). In another study, patient/public members representing patient charities required less training in order to be involved due to their professional background and role (Biggane et al., 2022). Whilst these members did sometimes have difficulty understanding their role and the process of guideline development, when compared to patient members with lived experience of a condition, members from patient charities did not seem to struggle with the technical content of the guideline development. These “professional” patient/public members were often able to understand the scientific content and language they were presented with during the guideline development process (Biggane et al., 2022).
Guideline Developers Should Consider the Ratio and Diversity Characteristics of Patients and Physicians Within a Guideline Panel (CERQual: Moderate Confidence, n = 1 Study)
In one article (M. J. Armstrong et al., 2017), some patients felt that they would be comfortable as the only patient representative on a guideline panel, whereas others said this would make them feel like they didn’t belong and it would be intimidating. Those who expressed feeling more comfortable with the idea of serving as the sole patient representative were older and described more experience with either physicians or group dynamics. There was general consensus that including multiple patient representatives would be ideal. Some patients indicated that simply having multiple patient members would be enough for them to feel comfortable, while others indicated that they would want there to be an equal number of patients and physicians. Regardless of panel composition ratio, patients agreed that smaller panels would make them more comfortable presenting their views, particularly to physician panellists (M. J. Armstrong et al., 2017). In this study, most patients said that physician race or ethnicity would not matter, but several African-American participants indicated that they would be more comfortable if there was a physician of the same race on the panel.
Guideline Development Benefits From the Inclusion of Methodological Experts (CERQual: Moderate Confidence, n = 3 Studies)
Three studies identified the engagement of principal investigators as an enabler to developing guidelines (Chartres et al., 2020; Kim et al., 2021; Piggott et al., 2021). Specifically, they recommended including experts in qualitative research and systematic reviews to offset the training requirements needed by other guideline group members (Kim et al., 2021). In two studies, these “technical experts” or “methodologists” were described as working as part of an evidence review group, independent from the guideline development group (Chartres et al., 2020; Piggott et al., 2021). However, their inclusion in the larger guideline development group was identified as an important enabler for guideline development (Chartres et al., 2020; Kim et al., 2021; Piggott et al., 2021). Methodologists who are members of guideline development groups could facilitate discussions of the reviews presented by the evidence review groups. Their understanding of the challenges associated with evidence review could bridge a gap in communication between the guideline development group and evidence review groups (Chartres et al., 2020). By integrating the groups, with subgroups to evaluate the evidence for particular questions, the tensions from lack of contact and communication may be significantly reduced (Chartres et al., 2020).
Taking a Network Approach to Guideline Development Benefits the Engagement of Interest-Holders in the Guideline Development Process (CERQual: Low Confidence, n = 6 Studies)
Six studies suggested that a network approach (e.g. formation of a network of organisations or individuals in guideline development) could facilitate the engagement of patients, providers of care, payers of health services/research, policymakers and principal investigators (Carter et al., 1995; Den Breejen et al., 2014; Duff et al., 1996; Ilott et al., 2006; Jarret, 2004; Rapu, 2005; van der Ham et al., 2014). Guideline development groups can use these networks to identify potential members and/or to share learning, operational systems, and resources among organisations (Rapu, 2005).
Establishing Guideline Group Processes
Barriers
Unclear Roles and Responsibilities for Guideline Development Group Members May Weaken or Disrupt Group Processes (CERQual: Moderate Confidence, n = 7 Studies)
When organisations do not have clear interest-holder engagement goals, the precise roles and responsibilities of each member of a guideline development group is often unclear. In the included studies, there was a lack of clear discussion about patients’, providers of care, and principal investigators’ roles and tasks in the guideline development groups, causing representatives to be uncertain as to what was expected of them (Carter et al., 1995; Jarret, 2004). Patients, in particular, were described as not being prepared for their role and were not aware of how their input influenced the guideline (Kim et al., 2021; Lanza & Ericsson, 2000). This can lead to members misinterpreting why they were selected and what they are supposed to do in the group, leading to tension between interest-holder groups (Chartres et al., 2020), creating confusion over how to use resources and which groups to involve (Carter et al., 1995; Jarret, 2004; van der Ham et al., 2014).
Failing to Manage Power Imbalances Between Patients and Providers of Care May Affect Group Dynamics and Limit Patient Engagement (CERQual: Moderate Confidence, n = 5 Studies)
Power differences between professionals and patients may cause issues within the guideline development group (M. J. Armstrong et al., 2017; Atkins et al., 2013; Carter et al., 1995; Harding et al., 2010; Kredo et al., 2018). If some personalities in the group are stronger than others, especially if members are posturing or power positioning, it may influence how patients participate in discussions. Guideline development meetings might be dominated by some interest-holders to the exclusion of others (Atkins et al., 2013). Perceived power imbalances can come from various sources - patients may not be comfortable if their own physician is on the panel (M. J. Armstrong et al., 2017), patients may interpret the use and explanation of jargon by other interest-holders as a display of power (Harding et al., 2010), and discussion chairs or leaders may not consider patient input. This can also extend to the national level, where territoriality and “turf protection” can interfere with national coordination and development efforts (Carter et al., 1995).
A Lack of Confidence in Their Skills and Legitimacy May Reduce Patient Members’ Level of Engagement (CERQual: Moderate Confidence, n = 5 Studies)
Lack of confidence was a commonly cited barrier to patients sharing their opinions in guideline development groups in five studies (M. J. Armstrong et al., 2017; Brouwers et al., 2018; Chalmers et al., 2017; Harding et al., 2010; Kim et al., 2021), especially if they did not have prior experience participating in guideline development. Some patients did not want to appear uninformed or unhelpful when talking with experts and felt that their experiences were not as important as professional opinion (M. J. Armstrong et al., 2017; Brouwers et al., 2018; Chalmers et al., 2017; Harding et al., 2010). This led to feelings of nervousness among patients and led to difficulties speaking up and voicing their opinions (Kim et al., 2021).
Facilitators
Clear Roles and Expectations for Interest-Holder Involvement in Guideline Development Facilitate Meaningful Engagement, Particularly for Patients and Principal Investigators (CERQual: Moderate Confidence, n = 12 Studies)
All interest-holders in a guideline development group should be aware of their roles, the tasks to be completed, and the timelines within which the guidelines are expected to be completed (Biggane et al., 2022; Boivin et al., 2010; Brouwers et al., 2017, 2018; Carter et al., 1995; Chalmers et al., 2017; Chartres et al., 2020; Den Breejen et al., 2014; Duff et al., 1996; Lanza & Ericsson, 2000; Piggott et al., 2021; Rapu, 2005). Among included studies, the most common role in need of more clarity was the role of patients. Patient roles should be clarified so that they understand what is expected of them, as well as the roles of others in the group. This can be done through orientation meetings delivered by a coordinator/navigator where patients can be told about and ask questions about the purpose and limits of their role (Brouwers et al., 2018). To clarify overall participant roles, guideline group members should have a strong understanding of their own and others’ experiences, clear project objectives and principles (Boivin et al., 2010; Brouwers et al., 2017, 2018; Carter et al., 1995; Chalmers et al., 2017; Den Breejen et al., 2014; Duff et al., 1996; Ilott et al., 2006; Lanza & Ericsson, 2000; Rapu, 2005). One article highlighted that the role of the evidence review groups needs to be clearly articulated from the start and reinforced throughout the guideline development process (Chartres et al., 2020). The evidence review groups also need to feel appropriately supported in their roles when presenting the evidence reviews, particularly by the working groups chairs (Chartres et al., 2020).
Guideline Developers Should Foster Group Processes That Promote Patient Empowerment (CERQual: Low Confidence, n = 10 Studies)
Ten included studies discussed ways to empower patients (C. Armstrong et al., 2019; M. J. Armstrong & Bloom, 2017; Biggane et al., 2022; Brouwers et al., 2017; Chalmers et al., 2017; Daraz et al., 2019; Ilott et al., 2006; Jarret, 2004; Murad et al., 2019; Synnot et al., 2022; van Wersch & Eccles, 2001). Empowering and fostering self-determination in patients can facilitate their participation in guideline development and can promote the involvement of patients as equal members of guideline panels (Biggane et al., 2022; Murad et al., 2019). Personal motivators that empowered patient engagement included a feeling of appreciation from the members of the guideline development group (Brouwers et al., 2017), acknowledging patient effort (Synnot et al., 2022), providing information and reading material in advance (M. J. Armstrong et al., 2017; Synnot et al., 2022), and/or pre-specifying mechanisms for soliciting opinions (M. J. Armstrong et al., 2017). Panel chairs can also give the option of passing on questions or submitting written answers if patients feel uncomfortable in discussions, as well as creating separate spaces for patient feedback (M. J. Armstrong et al., 2017). Protected opportunities to share patient feedback, such as dedicated slots on meeting agendas, also helped patients to feel more capable of contributing (Jarret, 2004). Two studies reported that patient-specific training resulted in feelings of empowerment (Biggane et al., 2022) and improved self-efficacy and confidence (Daraz et al., 2019). During training, patients developed a script to use during panel deliberations (e.g., what to do when the conversation includes too much jargon) and an instruction sheet for other panellists on how to empower and engage patients (Daraz et al., 2019).
Allowing Interest-Holders to Identify Their Preferred Mode of Participation can Facilitate More Equitable and Accessible Engagement (CERQual: Low Confidence, n = 9 Studies)
There are often individual preferences around engagement, and guideline development interest-holders should be asked about these preferences (M. J. Armstrong et al., 2017; Synnot et al., 2022). Guideline developers can consider engaging all interest-holders in-person or virtually (C. Armstrong et al., 2019; Brouwers et al., 2018; Selby et al., 2017; Synnot et al., 2022). For providers of healthcare, there is a preference to network, collaborate and provide feedback in-person rather than online and to participate in structured discussions (Selby et al., 2017). Direct, in-person engagement can also encourage patients to participate. This may include holding structured workshops in-person whenever possible as well as prompting patients during discussions (Daraz et al., 2019; Selby et al., 2017). In-person meetings should be held at venues which are physically accessible. Where travel is prevented through ill health or other issues, teleconference facilities should be offered to avoid excluding patients’ valuable contributions (M. J. Armstrong et al., 2017; Chalmers et al., 2017; Selby et al., 2017). Patients and caregivers also reported online engagement processes to be a convenient way to overcome physical, geographical, and time constraints (C. Armstrong et al., 2019; Brouwers et al., 2018). However, online methods can preference the participation of well-educated, online-proficient consumers with internet access and this should be taken into consideration (Synnot et al., 2022). Online methods can be cost-effective ways to engage the public and patient groups (Bartelmann et al., 2019). In one study of the RAND/PPMD Patient-Centeredness Method (RPM), the convenience of an online approach was amplified by the asynchronous nature of the engagement process (C. Armstrong et al., 2019). Participants found the engagement process accommodating given that they could complete each round at their convenience rather than finishing all of it in one sitting (C. Armstrong et al., 2019). Online methods facilitate engagement for people with physical or social disabilities and people who are disadvantaged and living in low and low-middle income countries (Synnot et al., 2022). A study among people with disability (traumatic brain injury) found that focus group discussions and online Wiki’s were acceptable and feasible - with a slight preference for focus groups due to their social nature (Lamontagne et al., 2021). However, in another study use of a wiki platform by providers of care was minimal (Selby et al., 2017).
Involving a Skilled Moderator/Chair in the Guideline Development Process can Improve Group Processes and Facilitate Engagement (CERQual: High Confidence, n = 9 Studies)
Effective guideline group process is impacted by the quality of organisation and support from guideline sponsor organisations and effective chairing (Piggott et al., 2021). Chairs are essential to ensure that meetings occur according to an agenda and to provide clear expectations to ensure that discussions are held in a timely manner and activities are kept on target (Piggott et al., 2021). In order to give patients and caregivers more voice in guideline development groups, guideline groups should appoint a skilled moderator with “a great deal of sensitivity” to lead the group (M. J. Armstrong et al., 2017). Patient and caregiver members described a good chair as inclusive, skilled, open, honest, and able to influence guideline development group discussions (Jarret, 2004), and able to actively support involvement of diverse groups (Synnot et al., 2022). Two facilitation techniques were suggested for effectively engaging patients during meetings: notifying working group members of the training and resources that patients receive before meetings, and designating space for patient input in meeting agendas allowing patients to contextualise the clinical guideline content in a patient relevant manner (Biggane et al., 2022). An effective chair can also help to enforce accessible discussions by asking patients for their opinions, asking professionals to explain their terminology, and protecting participant boundaries (M. J. Armstrong et al., 2017; Brouwers et al., 2018; Duff et al., 1996; Harding et al., 2010; Jarret, 2004; Kim et al., 2021). One study by Chartres et al. (2020) found that power imbalances between working groups [providers of care] and evidence review groups [principal investigators] could be minimised through appropriate facilitation. Facilitation can allow groups to feel their opinions are respected and appreciated throughout the guideline development process.
The Provision of Practical, Technical and Emotional Support Is a Facilitator of Engagement for all Interest-Holders (CERQual: Moderate Confidence, n = 9 Studies)
Support was identified as needed throughout the development process for all guideline development group members (Chartres et al., 2020; Den Breejen et al., 2014; Duff et al., 1996; Ilott et al., 2006; Jarret, 2004; Kim et al., 2021; Selby et al., 2017; Synnot et al., 2022; van der Ham et al., 2014). Guideline developers should designate an ongoing, proactive support person to provide practical, technical or emotional support (Synnot et al., 2022). Providers of healthcare described their employers as fully supportive of them doing guideline development activities during work hours, and support took the form of both time and encouragement (Ilott et al., 2006). Patients would benefit from support within their own patient organisations as well as collaborating centres (Jarret, 2004). This can be either process-related support, such as monitoring of patients and their needs throughout the process by the project manager, or content related support, for example by organising collective input from patient organisations. Guideline development group leaders should also consider contracting with local agencies or consultants to support engagement in certain geographic areas (Selby et al., 2017). Training all interest-holders in the skills required to take part in guideline development, such as communication and team-working, should also be provided (Kim et al., 2021).
Identifying Target Audience and Topic Selection
Barriers
We did not identify any evidence of barriers to interest-holder engagement specific to identifying a target audience or topic of guideline development.
Facilitators
Online Delphi Methods Facilitate Patient Engagement in Topic Selection and Priority Setting Exercises (CERQual: Low Confidence, n = 1 Study)
One article described how patients can help nominate and prioritise topics that they find important (C. Armstrong et al., 2019). Online modified-Delphi processes can be used by patients to systematically propose and develop consensus on the most important guideline topics. Specifically, patients could propose potential topics in an initial and open ended “brainstorming” round, and then prioritise them during subsequent rounds using close-ended rating or ranking questions. Alternatively, an online modified-Delphi process could help engage patients in prioritising an existing list of potential topics (C. Armstrong et al., 2019).
Consumer and Stakeholder Involvement
This project moves engagement from a stand-alone topic to a cross-cutting consideration for guideline developers to consider throughout each step of guideline development.
Conflict of Interest Considerations
Barriers
Patient Engagement May Be Limited by Concerns Around Confidentiality and Judgement in the Guideline Development Process (CERQual: Low Confidence, n = 3 Studies)
Some patients may refrain from sharing their opinion due to fears surrounding confidentiality or judgement (M. J. Armstrong et al., 2017). Patients may be worried about privacy issues, stigma, and/or denial when sharing personal experiences (Brouwers et al., 2017). Patients may be under the care of a member of the guideline development group, and this might not be a barrier for all patients, but it is possible that patients may feel coerced to participate or hide their true opinion if it is at odds with their physician’s opinion (Chalmers et al., 2017).
Facilitators
Guideline Developers Should Manage Conflicts of Interest and Patient Concerns Around Confidentiality (CERQual: Low Confidence, n = 4 Studies)
Project chairs should consider how they will manage patients’ conflicts of interest, especially if patients are receiving care from another guideline group member (Chalmers et al., 2017). While patients who had good relationships with their physicians felt comfortable sharing their opinions, this is not always the case (Chalmers et al., 2017). Chairs should take special care to maintain patient confidentiality and ensure that physicians do not disclose any personal patient information to the panel members without consent. Ideally, guideline developers should engage patients with no prior relationships with one another or the professionals on the panel (M. J. Armstrong et al., 2017). One article suggested that an online modified-Delphi process could be used by patients to indicate which conflicts of interest and mitigation strategies are most salient for developing a given guideline (C. Armstrong et al., 2019). In one study, guideline developers created working groups and ensured that a majority of members were without conflicts of interest. Members of the guideline development group without commercial conflicts were asked to adjudicate the final recommendations, thereby diluting the effects of any potential commercial bias (Selby et al., 2017).
(PICO) Question Generation
Barriers
Patients and the Public Perceived That Engagement in Health Outcome Selection Is Outside of Their Purview (CERQual: Low Confidence, n = 1 Study)
In one study conducted in the UK, patient and public members became involved in other aspects of the clinical guideline development, but their involvement in selecting health outcomes was relatively limited (Biggane et al., 2022). When asked, most patients and members of the public did not mention health outcomes as an area they were, or even wanted to be involved with. During committee meetings, they rarely participated in discussions about health outcomes or their selection and characterised this step of guideline development as a “system the technical team would go through”. How the role of patient/public members was perceived and the medical and scientific technicality of the guideline content all had a part in constraining their involvement in this topic (Biggane et al., 2022).
Time Constraints and Inflexible Timelines May Limit Patient and Public Engagement in Outcome Selection and Question Generation (CERQual: Low Confidence, n = 2 Studies)
Lack of time restricted the input of patient and public members in health outcome selection (Biggane et al., 2022) and patient and caregiver involvement in question generation (Köpke et al., 2018). One study used surveys to gather PICO questions from an international sample of patients and caregivers, and reported that this was time and resource consuming (Köpke et al., 2018). In the UK, this influenced guideline development later on, as the guideline development group was committed to the original questions and search terms developed earlier in the guideline process, and these were then not adaptable even if the resulting evidence was not relevant or understood by patient and public members (Biggane et al., 2022).
Facilitators
Patient and Public Engagement May Benefit From the Provision of Training Specific to Health Outcome Selection (CERQual: High Confidence, n = 1 Study)
The NICE Public Involvement Programme (PIP) facilitates training sessions to introduce patient and public members to the guideline development process, including topics such as understanding scientific evidence, how it is used, and what are outcomes (Biggane et al., 2022). Patient and public members largely described this training as “helpful” and “empowering”, praising the way their role, the process and the scientific terminology were explained (Biggane et al., 2022). It is important that clinical guideline developers continue to understand and embrace the value of providing training opportunities to the various members of the committee. Training should include the importance of patient and public member involvement and the rationale for outcome selection. This includes training for: the chair of the committee, health professionals, technical team members and patient/public members themselves. Opportunities for this could include expanding on the outcome section of the PIP training session or communicating it at committee level in the early meetings (Biggane et al., 2022).
Guideline Developers Should Consider Establishing Flexible Processes and Timelines (CERQual: High Confidence, n = 1 Study)
Guideline developers could explore whether there is flexibility in the timeline for the development of PICO questions and outcome selection. Specific suggestions include: seeking input to outcome selection at other junctures, establishing “break-out” working groups about specific topics, and increasing the timeframe for guideline development by having pre-meetings or engagements with committee members (Biggane et al., 2022).
Guideline Developers Should Use Empirical Methods for Engaging With Patients (CERQual: Low Confidence, n = 3 Studies)
Once the guideline development group is established, patients can assist with developing the PICO questions for the evidence review to ensure that its scope will be relevant and useful to real-world issues that the guideline aims to address. Online Delphi processes have been used to finalise the questions and outcomes (C. Armstrong et al., 2019). Specifically, patients can help refine clinical questions through an online modified-Delphi process that develops consensus on the importance of sub-populations, outcomes, aspects of treatments, and other elements (C. Armstrong et al., 2019). One study suggested that this information could also be collected qualitatively using focus group discussions (Biggane et al., 2022). They also recommended referring to already existing resources such as core outcome sets (Biggane et al., 2022). A third study used an international survey to select PICO questions, which allowed participation of a geographically varied population and by being anonymous facilitated openness and trustworthiness (Köpke et al., 2018). However, the authors noted that there was variable survey participation across countries, and that among those who did respond, they were online-proficient and well-educated participants, where the majority had completed post-secondary education (Köpke et al., 2018).
Guideline Development Should Involve ‘Champions’ and Representatives From Disease-specific Societies or Organisations (CERQual: High Confidence, n = 1 Study)
Disease-specific societies or advocacy groups can play a crucial role in the process of PICO question generation. In one study, the National Multiple Sclerosis (MS) Societies of participating European countries provided valuable feedback, particularly if interest-holders were both patients and MS Society members (Köpke et al., 2018). For example, one specific task force member was an MS patient and a member of the MS International Federation, and of the Danish national MS society. She was key in bridging the issues of the MS patients to those of the health professionals and researchers (Köpke et al., 2018), suggesting a valuable role of boundary spanners (i.e. institutions, groups, or individuals that straddle the divide between information producers and users) in health guideline development.
Considering Importance of Outcomes and Interventions, Values, Preferences and Utilities
Barriers
Patients’ Engagement May Be Limited by Feelings of Intimidation Derived From the Process or Other Group Members (CERQual: Moderate Confidence, n = 8 Studies)
Intimidation may be a barrier to engaging patients (M. J. Armstrong et al., 2017; Atkins et al., 2013; Brouwers et al., 2017, 2018; Duff et al., 1996; Harding et al., 2010; Jarret, 2004). In one study, a few participants said that preferences are hard to accurately capture, that clinicians did not always accept patient preferences, and it was challenging to meaningfully involve patients (Kim et al., 2021). Healthcare professional panel members are usually highly-trained and have experience working with other professionals but may not have experience working with those outside of their field and/or partnering with patients. Additionally, patient members may not have any training or experience within professional meetings, leading to hesitancy in commenting or questioning in the group discussions (M. J. Armstrong et al., 2017). This may be amplified if there is only one or few patients representative in the guideline development group (Atkins et al., 2013; Jarret, 2004). Some patients indicated that they did not want to appear uninformed or unhelpful, especially if the medical terminology and issues discussed are very complex (Brouwers et al., 2018). These participants may compare themselves to other guideline development group members or have less influence, which can undermine confidence.
Power Imbalances Between Patients and Providers of Care in Guideline Development Groups May Promote Unequal Influence in Decision-Making (CERQual: Moderate Confidence, n = 6 Studies)
There may be inequality in guideline development group members’ influence on decision-making (Atkins et al., 2013; Harding et al., 2010; Ilott et al., 2006; Jarret, 2004; Lanza & Ericsson, 2000; van de Bovenkamp & Zuiderent‐Jerak, 2013), especially if there is a perceived hierarchy of opinion where the perspectives of patient representatives are considered to be less valuable than those of healthcare providers (Atkins et al., 2013). Some guideline development group members may feel that patients do not easily fit into the decision-making structure (Harding et al., 2010). This can lead to limited influence of patient input, overturned group consensus, and a general undermining of the patient focus in guideline development (Harding et al., 2010). On the other hand, patients may not realise that their input is only one part of the guideline process and that the guideline development group is not obligated to follow the patients’ recommendations. One study reported that unequal influence was seen by some members as beneficial for decision making by preventing unfocussed discussion (Atkins et al., 2013).
Facilitators
We did not identify any evidence of facilitators to engaging interest-holders in determining the importance of outcomes, interventions, values, preferences, and utilities.
Deciding What Evidence to Include and Searching for Evidence
Barriers
Patient Engagement in Guideline Development May Be Limited by a Lack of Technical Skills Applicable to Reviewing Scientific Evidence (CERQual: High Confidence, n = 6 Studies)
Multiple studies mentioned the concern that the search, review, and discussion of scientific literature may be too technical or specialised for patient representatives (Hämeen-Anttila et al., 2016; Kredo et al., 2018; Schalkers et al., 2017; van de Bovenkamp & Zuiderent‐Jerak, 2013; van der Ham et al., 2014). The average patient’s lack of expertise - for both guideline development and literature reviews - was frequently cited as a barrier to engagement, for example: “He didn’t have the sort of skills – his contribution could only be […] from his experience that couldn’t meaningfully be fed into that process” (Brouwers et al., 2017).
The Impact and Depth of Patient Engagement May Be Limited by a Devaluation of Experiential Knowledge (CERQual: Moderate Confidence, n = 8 Studies)
The strong focus on evidence-based medicine and the systems used to evaluate scientific studies and categorise them in terms of the strength of the evidence makes it hard to give the experiential knowledge of patients a place in the guideline as it does not fit this categorization structure. One of the key challenges faced during guideline development processes is the dynamic tension between using an evidence-based approach (i.e., science, rigour and established expert contributions) versus tapping into tacit knowledge of all interest-holders (Walker et al., 2022). Guideline development group members may have different ways of defining evidence, ranging from rigorous scientific trials to “evidence of experience”. Within this, the medical knowledge of professionals is considered superior to the experiential knowledge of patients (Atkins et al., 2013; Harding et al., 2010; van de Bovenkamp & Zuiderent‐Jerak, 2013), and that this might be even more the case with children than with adults, suggesting that children’s experience of illness is excluded from guideline development (Schalkers et al., 2017). Several professionals indicated that it is often difficult to integrate input from patients into the guideline because it differed from other professional interest-holders (M. J. Armstrong et al., 2020) or focused on aspects of care, such as the organisation of care or the patient-provider relationship, which are often not the focus of a guideline (van der Ham et al., 2014). A single study also suggested that there are similar challenges engaging providers of healthcare and ensuring an appropriate balance between the evidence base, expert opinion and practitioner feedback (Selby et al., 2017).
Diverging Disciplines and Approaches Between ‘Evidence Review Groups’ (Technical Teams) and Guideline Development Groups May Promote Intergroup Conflict and Division (CERQual: Moderate Confidence, n = 1 Study)
Members of guideline development groups often have divergent disciplinary backgrounds (eg, physicians, toxicologists, clinical and public health researchers) compared to those on evidence review groups (eg, systematic review experts) and hold varying, sometimes conflicting beliefs about what constitutes ‘good’ evidence (Chartres et al., 2020). In one study, content experts often viewed ‘good’ or ‘important’ evidence from the perspective of their own knowledge and expertise and not from the standards of methodological rigour used by the evidence review groups (Chartres et al., 2020). These divergent roles and epistemological beliefs led to conflict and division between the groups. Many of the working committee members believed that in order to best protect the public’s health, the evidence presented by the review groups should at times be challenged (Chartres et al., 2020).
Facilitators
Guideline Developers Should Value Experiential Evidence and Elicit Patient Knowledge Beyond Guideline Development Group Membership (CERQual: Moderate Confidence, n = 4 Studies)
Patients are more likely to contribute if they feel appreciated by other members of the guideline development group and/or that their input matters. There are opportunities to access patient input from other sources over and above patient membership in the guideline development group (M. J. Armstrong et al., 2017; Hämeen-Anttila et al., 2016; Jarret, 2004). Patient and caregiver organisations have good channels for gathering information from the patient perspective. One possible means of gathering patient views were surveys distributed by the patient organisations (Hämeen-Anttila et al., 2016). Focus groups with patients and carers were also sources that guideline group members (including patient members) could draw on to inform their own contributions. Patients and caregivers appreciated interactive processes that include feedback loops (e.g. Delphi) and opportunities to directly engage with and learn from other patients (C. Armstrong et al., 2019). The iterative nature of data collection can foster better participant engagement by increasing their willingness to share and discuss their opinions and experiences with others, resulting in the development of new and shared perspectives (C. Armstrong et al., 2019).
Summarizing Evidence and Considering Additional Information
Barriers
Interest-Holder Engagement May Be Restricted by a Separation of Duties and Limited Communication Between Guideline Development Groups and Evidence Review Groups (CERQual: Moderate Confidence, n = 1 Study)
One study described how evidence review groups (composed primarily of researchers and methodologists) worked independently from the guideline development group for most of the process (Chartres et al., 2020). The transfer of ideas, knowledge and expertise pertaining to the inclusion and presentation of evidence was limited by this separation. By not working with the evidence review groups regularly from the outset and understanding the methods thoroughly, some guideline development group members felt limited in their ability to contribute to the guideline process due to their lack of methodological knowledge or disagreement with the included evidence. The irregularity of the meetings at which the evidence reviews groups presented their findings to the guideline development group meant it became an ineffective way of communicating how the evidence was being evaluated (Chartres et al., 2020). In turn, tensions developed between the two groups. Evidence review groups recognised an unequal and unfair power dynamic which led to them feeling that they were not valued or respected contributors to the guideline process (Chartres et al., 2020). The evidence review groups members felt that the tensions that were experienced could have been limited if guideline development groups were provided with clear information about the different roles of the two groups.
Facilitators
We did not identify any evidence of facilitators to engaging interest-holders in summarising evidence and considering additional information.
Judging Quality, Strength or Certainty of a Body of Evidence
Barriers
Interest-Holder Engagement in Evaluating Evidence May Be Limited by an Inadequate Understanding of Methods (CERQual: Low Confidence, n = 1 Study)
Guideline development group members and evidence review group members viewed understanding the methods [related to evaluating evidence] as a significant challenge to the current guideline development process (Chartres et al., 2020). Inadequate understanding of the methods restricts the level of input certain guideline development group members can have in the process and creates tensions between the groups (Chartres et al., 2020).
Facilitators
Providing Training on the Frameworks Employed in Assessing the Body of Evidence May Facilitate the Engagement of Guideline Group Members (CERQual: Moderate Confidence, n = 1 Study)
Pragmatic advice and training by the guideline development organisation for the guideline group members unfamiliar with the frameworks used to assess the body of evidence is necessary, not only at the commencement of the process but should be ongoing thereafter (Chartres et al., 2020).
Developing Recommendations and Determining Their Strength
Barriers
Patient and Public Engagement May Be Limited by Their Lack of Content Knowledge, Clinical Expertise and Technical Skills (CERQual: Moderate Confidence, n = 12 Studies)
Some patients felt that they had a lack of clinical content knowledge, insufficient academic education, guideline process training, and/or computer skills, which would hinder the complex scientific discussion and the recommendation development process. Unless they are educated about the guideline topic or have specific prior knowledge and training, patients may stay quiet during group discussion (Harding et al., 2010; Schalkers et al., 2017; Selby et al., 2017; van de Bovenk & Zuiderent‐Jerak, 2013; van der Ham et al., 2014). Additional barriers included the need for in-depth scientific discussion, which may not always be accessible for patients or caregivers, leading to the concern that discussions may be stifled if conversations are constantly being interrupted for lay explanations (M. J. Armstrong et al., 2020; M. J. Armstrong & Bloom, 2017; Brouwers et al., 2017, 2018; Chalmers et al., 2017; Den Breejen et al., 2014). In one study, discussions among health professionals in committee meetings for this guideline indicated that they also believed that the content was too clinically driven for lay members to understand (Biggane et al., 2022).
Insufficient and Competing Demands of Time May Limit Interest-Holder Engagement in Recommendation Development (CERQual: Moderate Confidence, n = 11 Studies)
Time was a frequently cited barrier across multiple studies regarding the engagement of patients, caregivers, patient organisations, and providers of care (M. J. Armstrong & Bloom, 2017; Atkins et al., 2013; Bartelmann et al., 2019; Brouwers et al., 2017, 2018; Chalmers et al., 2017; Den Breejen et al., 2014; Ilott et al., 2006; Kredo et al., 2018). This included insufficient time for discussion and recommendation formulation, as well as the time-consuming nature of travel and attending meetings. Participants may feel rushed or unable to fulfil their responsibilities. For example, challenges were encountered in getting responses from professional societies within the four-week consultation deadlines, and these problems were more severe for organisations with limited capacity (under 2,000 members) (Ilott et al., 2006). One study noted that it was difficult to keep patients interested in attending meetings because of conflicts with patients’ work or personal lives (Brouwers et al., 2018). A lack of dedicated time for guideline development may also lead to guideline development group members working voluntarily on their own time and afterhours (M. J. Armstrong et al., 2017; Atkins et al., 2013; Brouwers et al., 2017, 2018; Chalmers et al., 2017; Den Breejen et al., 2014; Ilott et al., 2006; Kredo et al., 2018). The length of time required to develop guidelines may also prevent healthcare professionals from engaging in their development (Bartelmann et al., 2019). Clear timelines are also needed for the technical experts conducting evidence syntheses, so that sufficient time is allowed for the review of evidence in advance of guideline meetings (Piggott et al., 2021). Patients should be given sufficient notice of meetings and allocated enough time to understand the materials and provide feedback (Brouwers et al., 2018; Chalmers et al., 2017; Duff et al., 1996).
Unequal Access to Information Technology Resources May Limit Interest-Holders’ Ability to Engage in Guideline Development (CERQual: Moderate Confidence, n = 3 Studies)
Two studies, conducted among patients (Synnot et al., 2022) and providers of care (Kredo et al., 2018) identified that the guideline development process favoured those with access to and the ability to use the Internet and technology. One study noted that a patient representative had trouble carrying out her work because she had no printer and the documents she needed were soft copy only (Jarret, 2004).
Technical Concepts and Professional Jargon May Negatively Impact Patient and Public Comfortability and Ability to Contribute to the Guideline Development Process (CERQual: Moderate Confidence, n = 6 Studies)
Patients thought meeting dialogue was sometimes overly technical and that professionals’ use of jargon was like “speaking a foreign language” and represents a display of power (M. J. Armstrong et al., 2020; Biggane et al., 2022). This technical content comprised terminologies, abbreviations and topic content which lay members believed were largely only accessible to health professionals or those with specialised knowledge (Biggane et al., 2022). Many patients may not be familiar with the medical language spoken and ask for clarification of terms and abbreviations (M. J. Armstrong et al., 2020). Patients without some research knowledge reported difficulties grasping the concept of the evidence framework and the statistics surrounding it. Some patients who found the documents and reading material too technical or scientific decided that they had no contribution to make on these issues and felt excluded (Harding et al., 2010; Jarret, 2004; van Wersch & Eccles, 2001). Such patients were reported to be “participating observers of technical language to which they could hardly offer any input” (Brouwers et al., 2018).
Facilitators
Guideline Developers Should Provide Advance Preparation Materials and Resources Prior to Meetings and Throughout the Project (CERQual: Moderate Confidence, n = 5 Studies)
Providing resources to all interest-holders, especially hard-copy documents, before meetings and offering opportunities for clarification can help all guideline development group members contribute effectively (Jarret, 2004). This may include supplying meeting agendas and talking points, questions, guidance and terms of reference documents, and/or materials to take notes with (Brouwers et al., 2018). There should also be a point of contact for guideline development groups members (M. J. Armstrong et al., 2017); those in charge should make time to return calls and keep track of their contacts. It is particularly helpful for patients to meet with the chairs prior to meetings (and particularly the first meeting) so that they know what to expect and get to know some other people in the meeting prior to joining larger group meetings (M. J. Armstrong et al., 2017; Brouwers et al., 2018; Chalmers et al., 2017; Jarret, 2004; Lanza & Ericsson, 2000).
Guideline Developers Should Offer Explanations and Promote the Use of Plain Language to Ensure That Participation in the Guideline Development Process Is Accessible to all Members (CERQual: Moderate Confidence, n = 4 Studies)
Guideline developers should ensure the language used in all aspects of the guideline development is accessible to all committee members (Biggane et al., 2022). In particular technical experts must communicate concisely, effectively, and collaboratively toward the production of the guidelines (Piggott et al., 2021), preferably in plain language avoiding scientific or medical abbreviations (Biggane et al., 2022). Other activities may include translation of scientific literature and content into plain language formats as needed (Biggane et al., 2022). A pivotal success factor for engaging patients seemed to be the extent to which people explained terms to one another. Project chairs or navigators can meet with patients prior to meetings so that patients can have an explanation of the context of the expected discussion. Guideline development group members can also use natural breaks in discussion to explain to patients in lay terms what is being discussed and ask patients specific questions where their input is needed. Some people linked this with an (often subtle) display of power which could be redistributed. Others felt these strategies contributed by preventing unhelpful assumptions from influencing proceedings (Chalmers et al., 2017; Harding et al., 2010).
Wording of Recommendations and of Considerations of Implementation, Feasibility and Equity
Barriers
We did not identify any evidence of barriers to interest-holder engagement specific to wording recommendations or considering implementation, feasibility and equity for guideline development.
Facilitators
Guideline Developers Should Consider Using Online Methods/Tools to Refine Recommendations (CERQual: Low Confidence, n = 3 Studies)
We identified 3 studies which suggested the use of online tools to engage interest-holders in the refinement or wording of recommendations (C. Armstrong et al., 2019; Lamontagne et al., 2021; Pottie et al., 2021). The GRADE-FACE is proposed as an efficient means to collect interest-holder perspectives on drafted recommendations whereby respondents assess the feasibility, acceptability, cost, and equity of each drafted recommendation on a GRADE rating scale and provide qualitative comments to justify and explain their response (Pottie et al., 2021). The authors note that for underserved and disadvantaged populations, electronic surveys may not be feasible or appropriate. In-person semistructured interviews may be best suited to collect guideline perspectives from these groups (Pottie et al., 2021). Similarly, the RPM is an online modified-Delphi approach to patient engagement in guideline development that allows patients and their representatives to provide input on the patient-centeredness of draft guideline recommendations by participating in an idea generational round, two rating rounds, and an online discussion round (C. Armstrong et al., 2019). The RPM can also be used to explore the existence of consensus among key interest-holders on other dimensions of the Evidence-to-Decision framework, such as feasibility, equity, resource use and issues related to disadvantaged patient subpopulations (C. Armstrong et al., 2019). A third study proposed an online wiki approach (Lamontagne et al., 2021). Participants considered the wiki method effective to the extent that it allowed for analysis and reflection before answering and because participants could express themselves without being stopped or interrupted. Other key advantages mentioned included anonymity, not feeling judged by others, the freedom to choose the time and place to answer the wiki questions, and doing it at the participant’s own pace, respecting his or her capacity. However, participants were individuals with traumatic brain injury, and generally expressed preference for in-person options such as focus groups, as they more easily fit with the communication habits of individuals with a head injury who prefer oral expression to writing (Lamontagne et al., 2021).
Reporting and Peer Review
Barriers
We did not identify any evidence of barriers to interest-holder engagement in the reporting and peer review stage of guideline development.
Facilitators
Interest-Holder Engagement in the External Review Process May Benefit From Longer Review Periods Combined With Individualised Reminder Emails (CERQual: Low Confidence, n = 1 Study)
We identified only one study which reported facilitators to interest-holder engagement during the external peer review process of a guideline (Bartelmann et al., 2019). They reported measures to encourage people to take part in the external review of guidelines might include longer review periods combined with individualised reminder emails to people within all categories of interest-holders. Paper-based advertisements in clinical practices and online advertising on patient-focused websites might be an option to identify patients and members of the general public who could participate. These measures, however, require time and funding that may not be available to the average guideline development group. Less expensive solutions might include the use of online platforms that allow external reviewers of guidelines to obtain recognition for their efforts even in cases where their reviews are anonymous. Having external review efforts formally recognised and rewarded by medical faculties or colleges might also have a salutary effect on participation rates (Bartelmann et al., 2019).
Dissemination and Implementation
Barriers
We did not identify any evidence of barriers to interest-holder engagement specific to dissemination and implementation for guideline development.
Facilitators
Online Delphi Methods can Be Used to Engage Patients in Identifying Implementation Considerations and Dissemination Strategies (CERQual: Low Confidence, n = 1 Study)
Patients can assist in disseminating and implementing guideline recommendations. For example, patients and patient advocacy groups can endorse guidelines to improve their legitimacy and trustworthiness to patient communities, as well as increasing patient adherence to guideline recommendations. Online modified-Delphi processes can be used to engage patients in identifying items to include in decision aids, as well as barriers and solutions to guideline uptake (C. Armstrong et al., 2019).
Evaluation and Use
We did not identify any evidence of barriers and facilitators to interest-holder engagement specific to evaluation and use during guideline development.
Updating
Barriers
Guideline Developers May Face Challenges Engaging Interest-Holders in Guideline Updates due to Insufficient Financial Resources (CERQual: Low Confidence, n = 2 Studies)
Two studies reported barriers to maintaining interest-holder engagement for patients, providers of care, policy makers, and principal investigators during guideline updates (Selby et al., 2017; van der Ham et al., 2014). In one study, lack of ongoing funding prevented the secretariat from keeping the original guideline development group engaged in guideline activities (Selby et al., 2017). van der Ham et al. (2014) also suggests that it can be difficult for patients to ‘catch up’ and meaningfully contribute if they were not involved in the original guideline.
Facilitators
Using an Online Delphi Process to Assess the Patient-Centeredness of Guideline Recommendations May Help Facilitate Patient Engagement in Guideline Updates (CERQual: Low Confidence, n = 1 Study)
According to one article, patients can assist in updating and evaluating guidelines, such as determining if and when guidelines require an update, assessing whether patients made meaningful contributions to a guideline, and informing improvements to future patient engagement strategies in guideline development. To facilitate patient engagement at this stage, online modified-Delphi processes can be used to continually assess the current patient centeredness of guideline recommendations. For example, patients can define and prioritise significant changes in views from the patient community since a guideline has been developed, such as outcomes and interventions that should be included in a guideline update (C. Armstrong et al., 2019).
Discussion
Summary of Main Results
This qualitative evidence synthesis identified a total of 51 findings encompassing 23 barriers and 28 facilitators to interest-holder engagement in health guideline development. While the large majority of reported evidence is about patient engagement, many findings are applicable to all interest-holder groups and should be considered by guideline developers who aim to apply a multi-interest-holder engagement approach.
Several barriers may limit meaningful engagement. At the organizational level, many guideline development groups lack the financial and human resources needed to coordinate engagement activities. Goals for patient and public involvement are often vague or implicit, resulting in inconsistent implementation. In some cases, guideline developers deliberately exclude patients, believing that guideline topics are too complex or that engagement may become tokenistic rather than meaningful. Representation poses another major challenge, as organizations struggle to identify diverse and representative patient participants. Indeed, ‘representation’ within the context of health research is highly contested (Maguire & Britten, 2017) and is often at odds with concepts of ‘experience’ and ‘expertise’. When patients are involved, they may experience tokenism, feel intimidated by professional hierarchies, or encounter language barriers that prevent them from contributing fully.
Process-related challenges further restrict engagement. Unclear roles and expectations can create confusion within groups, while power imbalances between healthcare professionals and patient members can inhibit open discussion. Patients may also lack the confidence, time, or technical expertise to participate comfortably in guideline development meetings, particularly when discussions rely on complex scientific evidence and specialized terminology. Technical and structural barriers, such as limited access to technology, inflexible timelines, and confidentiality concerns, also constrain engagement.
Despite these challenges, several facilitators can promote meaningful engagement. Organizational commitment is essential. Embedding interest-holder engagement into institutional structures and staff roles normalizes participatory practices. Formal engagement models, such as the Community Consultation and Review Panel (CCRP) and the RAND/PPMD Patient-Centeredness Method (RPM), demonstrate how structured and sustained approaches can support patient input throughout the process. Capacity building is another key enabler: providing training, mentorship, and clear preparatory materials helps interest-holders understand the guideline process and increases their confidence.
Equity and inclusion play a vital role in effective engagement. Recruiting diverse, knowledgeable, and experienced patient representatives ensures that multiple perspectives are considered and reduces feelings of isolation. Financial compensation, reimbursement for expenses, and professional accreditation recognize the value of interest-holder contributions and lower barriers to participation. Facilitative processes also enhance engagement. Skilled chairs and moderators can create safe, inclusive environments, manage power dynamics, and ensure that all voices are heard. Flexible approaches such as offering in-person and online options, including Delphi and teleconference methods, make participation more accessible and sustainable.
Importantly, meaningful engagement depends on valuing interest-holder knowledge as legitimate evidence alongside scientific data. Involving patient organizations, conducting surveys and focus groups, and incorporating patient perspectives into consensus-building activities may strengthen the relevance and acceptability of guidelines. Overall, this review highlights that meaningful interest-holder engagement requires sustained institutional investment, inclusive recruitment, and equitable participation mechanisms. Combining organizational commitment, training, financial support, accessible communication, and respect for experiential knowledge can transform engagement from a symbolic gesture into a foundational element of high-quality guideline development.
The findings summarized in this evidence synthesis closely align with several established theories and frameworks of interest-holder engagement in health research, including Arnstein’s Ladder of Participation (1969), the IAP2 Spectrum of Public Participation (International Association for Public Participation IAP2, 2024), and Cochrane’s ACTIVE (Authors and Consumers Together Impacting on eVidencE) framework (Pollock et al., 2019). Collectively, these theories conceptualize engagement as a dynamic process influenced by power relations, structures of participation, and recognition of diverse forms of knowledge.
Arnstein’s Ladder of Citizen Participation (Arnstein, 1969) depicts engagement as a continuum from non-participation to citizen control. The barriers identified in this synthesis such as tokenistic participation, unclear roles, and limited decision-making power, mirror the lower rungs of Arnstein’s ladder, where engagement is symbolic rather than substantive. Conversely, facilitators such as shared leadership, equitable compensation, and capacity-building correspond to higher rungs (e.g., partnership and delegated power). These findings suggest that most guideline development processes remain situated within the middle tiers of Arnstein’s ladder, achieving consultation but not full empowerment. Advancing toward the upper rungs requires shifting power to interest-holders so that they act as co-decision-makers rather than passive consultees.
The International Association for Public Participation (IAP2) Spectrum similarly categorizes participation from “inform” to “empower”, reflecting progressive levels of interest-holder influence. This synthesis demonstrates that most guideline initiatives currently operate at the “consult” or “involve” stages, where patients and the public provide input without direct authority over outcomes (International Association for Public Participation (IAP2), 2024). Facilitators such as embedding formal engagement models and creating supportive structures for training and feedback correspond to higher levels of participation on the IAP2 spectrum. These actions move organizations toward “collaboration” and “empowerment”, where engagement is embedded as an ongoing, institutionally supported practice rather than a discrete event.
Cochrane’s ACTIVE framework conceptualizes engagement as a partnership between researchers and consumers, emphasizing capacity-building, respect, reciprocity, and relationship development (Pollock et al., 2019). This synthesis identifies similar facilitators, including the need for clear roles, training, mentorship, and effective facilitation to build trust and sustain collaboration. Furthermore, our synthesis’ emphasis on embedding engagement across all stages of guideline development, rather than confining it to a single guideline development topic (Topic 6), reflects the ACTIVE framework’s vision of continuous, multi-level involvement across the evidence ecosystem.
The findings of this synthesis also align with theories of co-production, which advocate for the shared creation of knowledge among researchers, practitioners, and the public (Greenhalgh et al., 2016; Rycroft-Malone et al., 2016). The reported barriers such as the devaluation of experiential knowledge and dominance of technical expertise illustrate the absence of co-production principles in many guideline processes. Facilitators including patient empowerment, plain-language communication, and co-learning between professional and public members embody the co-production ethos, reframing engagement as a process of mutual respect and collective problem-solving rather than expert-driven consultation. Ultimately, embedding these theoretical insights into guideline development can help move engagement from symbolic participation to genuine co-creation of knowledge, thereby strengthening the relevance, legitimacy, and fairness of clinical practice guidelines.
Overall Completeness and Applicability of Evidence
While this review attempts to provide a comprehensive overview of research on interest-holder engagement in health guideline development, there are some notable challenges regarding the generalizability across different settings, populations and contexts. The findings suggest that substantial time and resources are required in order to embed interest-holder engagement across guideline development, a process which may not be achievable in low resource settings or under-funded institutions or topic areas. The evidence identified in this review covers many topics of guideline development, however we did not identify any barriers or facilitators for some critical topics. Notably, we did not identify any evidence of barriers or facilitators for interest-holder engagement in priority setting, a surprising finding given the promotion of interest-holder engagement in this area (e.g. (Grill, 2021; Manafò et al., 2018)) and the international recognition of the James Lind Alliance approach (James Lind Alliance, 2024). Additionally, evidence was also lacking for the following topics, and these remain areas in need of further research: Identifying Target Audience and Topic Selection Considering Importance of Outcomes and Interventions, Values, Preferences and Utilities Summarising Evidence and Considering Additional Information Wording of Recommendations and of Considerations of Implementation, Feasibility and Equity Reporting and Peer Review Dissemination and Implementation Evaluation and Use
However, we acknowledge that whether a finding is listed as a barrier or a facilitator and its categorisation within the topics of the GIN-McMaster checklist is arbitrary and depends on the way in which the evidence has been framed by the authors of the included studies and interpreted by the review team. Some of the identified findings are applicable across the guideline development process. Additionally, there is a larger body of literature available on patient and public engagement in health research more generally where such information might be identified.
Additionally, the availability of evidence was not evenly distributed across interest-holder groups. Patients and caregivers, members of the public, providers of care and principal investigators were most commonly engaged in guideline activities, and several barriers and facilitators to their engagement were identified. This is consistent with the broader availability of literature on engagement in health research. Other groups, such as policymakers, program managers, purchasers, peer review journal editors, product makers, and payers of research were infrequently engaged in guideline activities, and little to no barriers and facilitators to their engagement were reported. Despite this, many of the barriers and facilitators reported would apply to all interest-holder groups within the context of guideline development. To address this gap in the literature, additional research is being undertaken by the MuSE Consortium to establish interest-holder preferences for engagement in guideline development (Magwood et al., 2026; Petkovic et al., 2020, 2025).
The findings of this review have widespread applicability to any organisation or government that produces health recommendations or guidelines, including health technology assessments.
However, the findings should be considered with some reservations. Almost all evidence in this review originates from high-income settings and is conducted among privileged and well-educated participants with good access to resources and technology. Very little information was available from equity-deserving groups and low-income settings. It remains unclear whether there are particular conditions or guideline settings (e.g., public health vs health system) that influence the engagement of diverse interest-holders. Country-level capabilities to form national guideline groups are not the same across the globe (for e.g., we do not see national guidelines from all LMIC countries). This could be because of their limited experience in framing guidelines and recommendations, which needs to be considered when international guidelines are prepared. To decolonise the guideline development process, guideline development groups should be balanced between representatives from global south and global north countries. Novel methods to include the voice of patients who have experienced poverty-related diseases should be developed to establish equity in guideline development.
Quality of the Evidence
The quality of the available evidence varies, with the best evidence found for patients and the public. This is because these interest-holder groups were often the focus of the included studies, which supports the growing movement for co-production in health. Studies involving patients and the public were more frequently prospective, included ethical approval, and employed rigorous methods such as ethnographic approaches and cross-over studies. For other interest-holder groups, studies were often retrospective and descriptive. Our CASP critical appraisal indicates that the most common concern regarding study quality was a lack of reflexivity/positionality by the study authors and a focus on the perspectives of the guideline developers rather than the interest-holders themselves. This concept is particularly important given the reliance of some findings on the authors observations and opinions on engagement in guideline development. Such methodological limitations of the data informing the findings are reflected in the accompanying CERQual assessments of confidence of the evidence. Confidence in findings ranged from very low to high, with the majority of findings having low or moderate levels of confidence. This indicates that future research is likely to change or expand upon the findings. Due to the large reliance on evidence stemming from patient and public populations, findings that applied to multiple interest-holder groups were often downgraded due to relevance (limited representation). Further, there was substantial variation in the thickness of the available data, and future research should consider in-depth ethnographic studies of interest-holder perspectives to further elucidate these findings. More high-quality prospective research is needed to capture the perspectives and preferences of multiple interest-holder groups.
Given the high certainty of evidence for several findings, it is unlikely that additional research will influence the following: Patient engagement in guideline development may be limited by a lack of technical skills applicable to reviewing scientific evidence. The recruitment of patient representatives should prioritise the identification of knowledgeable and experienced patient representatives. Involving a skilled moderator/chair in the guideline development process can improve group processes and facilitate engagement. Patient and public engagement may benefit from the provision of training [specific to health outcome selection]. Guideline developers should consider establishing flexible processes and timelines.
Additionally, interest-holder characteristics were infrequently reported, and rarely available according to the PROGRESS-Plus acronym. This limited our ability to make conclusions about interest-holder engagement across diverse equity characteristics and settings. Ongoing updates to reporting guidance, such as the CONSORT-Equity 2017 extension for better reporting of health equity in randomised trials (Welch et al., 2017) and the STROBE-equity reporting guideline to improve social justice in observational studies (Dewidar et al., 2025) may help in this regard. This highlights a larger issue regarding the reporting of barriers and facilitators of guideline initiatives - while reporting guidance is available for particular research designs (e.g., PRISMA, CONSORT, STROBE) and guidelines generally (e.g., AGREE, RIGHT), none of these tools provide authors with assistance self-identifying challenges or lessons learned from their own work. Given this limitation, it is likely that barriers and facilitators exist beyond what is reported in the peer-reviewed literature.
Limitations and Potential Biases in the Review Process
This qualitative evidence synthesis offers a comprehensive overview of the barriers and facilitators to interest-holder engagement across all topics of guideline development for 10 different interest-holder groups. However, a number of limitations should be noted: (i) While we included diverse interest-holders in the development of the protocol to inform this review, no patients or members of the public were engaged in the development of the review findings. (ii) While no barriers or facilitators were identified for several topics of guideline development, it remains unknown where there are truly no issues, whether there were issues that were judged by authors as not worth commenting on in their publication, or whether study participants chose to remain silent about some topics. We did not attempt to contact study authors for missing data. (iii) We included studies only if they were specific to at least one topic of guideline development. We excluded health technology assessments, as these processes will be covered in future work by the MuSE Consortium. (iv) We searched for included studies in English language databases only, which may explain why the included studies originated primarily from high-income and anglophone countries. (v) Finally, our review did not consider the role of “knowledge brokers” as a distinct interest-holder group. Knowledge brokers in health systems are important intermediaries who play a critical role in bridging the gap between research, policy, and practice. A key component of the knowledge brokering process is facilitating collaboration and engagement among diverse interest-holders. Therefore, the inclusion of knowledge brokers may be an overlooked facilitator to engagement not considered in the current review, and this represents an important opportunity for future research.
Agreements and Disagreements With Other Studies and Reviews
In the last decade, several reviews have acknowledged the importance of involving various interest-holders in the development of health guidelines (M. J. Armstrong & Bloom, 2017; Bryant et al., 2022; Cluzeau et al., 2012; Kelson et al., 2012; Légaré et al., 2011; Pinho-Gomes et al., 2022; Synnot et al., 2022) Particularly, the engagement of patients has become more common and the importance of engaging patients in the decision-making process has been recognized (M. J. Armstrong & Bloom, 2017; Bryant et al., 2022; Grant et al., 2018; Kelson et al., 2012; Légaré et al., 2011; Pinho-Gomes et al., 2022). However, no previous reviews of the literature have considered the barriers and facilitators to engaging multiple interest-holder groups who are not traditionally a part of the guideline development process such as policymakers, program managers, payers of services or research, and peer review journal editors. Much of the existing literature focuses on the engagement of patients, the public, providers of care, and principal investigators.
One barrier that was consistent across our results and other reviews was challenges with guideline group membership. Our review found there can be challenges to recruit appropriate people from target groups that are capable of representing the wider patient population (Brouwers et al., 2018; Chalmers et al., 2017; Hämeen-Anttila et al., 2016; Lanza & Ericsson, 2000; van der Ham et al., 2014) There exists some debate as to whether patients engaged in guidelines should be “representative” -- there are no other group members who participate with the expectation of representation. For example, health care providers are diverse in the ways they practise and the contexts within which they practise, yet they are not expected to stand in for all providers. Additionally, there may be challenges in representing the target interest-holder group in terms of health conditions and ethnic diversity (van der Ham et al., 2014). Similarly, a review by Légaré et al. (2011) described challenges in ensuring appropriate representation of patients. In a review by Merner and colleagues, recruiting a range of patients, caregivers and community members who were reflective of the clinic’s demographic population was considered desirable, particularly by health providers (Merner et al., 2023).
Our review found that many studies emphasised the significance of choosing appropriate patients to be included in guideline development, with clear eligibility criteria in mind (M. J. Armstrong et al., 2017; Boivin et al., 2010; Brouwers et al., 2018; Chalmers et al., 2017; Den Breejen et al., 2014; Harding et al., 2010; van der Ham et al., 2014; van Wersch & Eccles, 2001). Légaré and colleagues (2011) suggested that participants should be as representative of the whole population as possible. Reviews by Légaré et al. (2011) and Kelson et al. (2012) found it can be challenging to identify appropriate patients to engage as interest-holders and identify interest-holders who are willing to contribute to guideline development. More recent publications have outlined factors for research teams to consider during identification and invitation of individual representatives in a multi-interest-holder partnership, including experience, inclusivity, willingness, communication skills, and time capacity (Parker et al., 2022). A review by Grant and colleagues (2018) concluded that as patients become more actively involved in clinical guideline development, additional research is needed for ways to engage large groups of diverse patients from various geographic locations. Using a “network approach” can help recruit representative and knowledgeable patients (or other interest-holders), however this risks recruiting the “usual suspects” and limiting perspectives.
Having a chair (moderator) that supports interest-holder-engaged guideline development was commonly noted as a facilitator. Our review found it is important to select a chair who can assist in asking patients for their opinions during discussions and offer explanation of terminology (M. J. Armstrong et al., 2017; Brouwers et al., 2018; Duff et al., 1996; Harding et al., 2010; Jarret, 2004) and help with comprehension of discussion content (Chalmers et al., 2017; Harding et al., 2010). Légaré and colleagues (2011) also emphasised the importance a chair has in supporting patients, ensuring patients are engaged and have a voice during discussions, and creating opportunities for meaningful contributions. The guideline chair can also ensure that all interest-holders are aware of their roles and responsibilities throughout guideline development. Our review found that all guideline members should clearly understand their roles and expectations within the group (Boivin et al., 2010; Brouwers et al., 2017, 2018; Carter et al., 1995; Chalmers et al., 2017; Den Breejen et al., 2014; Duff et al., 1996; Ilott et al., 2006; Lanza & Ericsson, 2000). Légaré and colleagues (2011) supported this, stating every group member must be aware of their scope within the guideline development process with well-defined goals.
The devaluation of patient experiential knowledge was a similar barrier found in our review and others. van der Ham and colleagues (2014) stated that there are challenges with including input from patients into guidelines because often their input focuses on aspects of care which may not be the focus of the guideline. Guidelines tend to focus on evidence-based medicine (i.e. empirical evidence) and this creates a challenge for including experiential knowledge of patients in guidelines (Atkins et al., 2013; Harding et al., 2010; Schalkers et al., 2017; van de Bovenkamp & Zuiderent‐Jerak, 2013). A review by Pinho-Gomes and colleagues (2022) found challenges and uncertainty around integration of patient experiences into guidelines that are heavily evidence-based. Reviews by Kelson et al., (2012) and Légaré et al. (2011) also found that a challenge exists around creating true evidence-based guidelines while including all relevant interest-holders, as the values and preferences of patients and caregivers may vary greatly from that of clinical experts. Armstrong and Bloom (2017 also found that differences between patient and physician views were significant barriers for the low frequency of public and patient engagement in guideline development. A review by Cluzeau and colleagues (2012) also found that various interest-holders may hold different perspectives, thus placing differing weights on outcomes. It is because of these differences that guideline development groups should include the diverse perspectives of all interest-holders in decision-making (Cluzeau et al., 2012).
Valuing patient contributions in the guideline development process was therefore identified as a facilitator. Our findings suggest that contributions from patients are to be equally valued, patients should have autonomy in the decision making processes, and should be able to freely provide feedback (C. Armstrong et al., 2019; Chalmers et al., 2017; Den Breejen et al., 2014; Ilott et al., 2006; Jarret, 2004; van de Bovenkamp & Zuiderent‐Jerak, 2013). Similarly, a review by Kelson and colleagues (2012) found that perspectives from both patients and caregivers are to be included and valued in the decision making processes for clinical guidelines. A review by Armstrong and Bloom (2017) also emphasised the importance that the public and patient groups have opportunities to contribute to guideline development. Pinho-Gomes and colleagues (2022) found that the patient and public group are to be valued equally in guideline development, as they can provide unique and valuable insights. We found studies suggesting that patients are to be included in meetings when their input would be beneficial and relevant (C. Armstrong et al., 2019; Chalmers et al., 2017; Den Breejen et al., 2014; Ilott et al., 2006; Jarret, 2004; van de Bovenkamp & Zuiderent‐Jerak, 2013).
Our review also found that involving patients at all stages of guideline development instead of serving occasionally provides more meaningful engagement (Atkins et al., 2013; Brouwers et al., 2017; Chalmers et al., 2017; Jarret, 2004; Selby et al., 2017; van der Ham et al., 2016). Reviews by Légaré et al. (2011) and Pinho-Gomes et al. (2022) also emphasised that patients are to be involved at every stage of guideline development, particularly at the beginning. We found that patients were empowered when they were given the options and flexibility in communication, such as passing on questions or providing written answers if they were uncomfortable engaging in discussions (C. Armstrong et al., 2019; M. J. Armstrong et al., 2017; Brouwers et al., 2017; Chalmers et al., 2017; Ilott et al., 2006; Jarret, 2004; van Wersch & Eccles, 2001). Providing safe and confidential opportunities for patients to share feedback assisted patients to feel more qualified to contribute to guideline development (C. Armstrong et al., 2019; M. J. Armstrong et al., 2017; Brouwers et al., 2017; Chalmers et al., 2017; Jarret, 2004; van Wersch & Eccles, 2001).
Additionally, our review found that guideline development processes favour interest-holders who have access to and know how to use technology and the Internet (Jarret, 2004; Kredo et al., 2018). Particularly, some patients may have insufficient computer skills needed to properly engage in the guideline development process (Harding et al., 2010; Schalkers et al., 2017; Selby et al., 2017; van de Bovenkamp & Zuiderent‐Jerak, 2013; van der Ham et al., 2014). Grant and colleagues (2018) reported that some patients may not be able to participate using online methods due to lack of access to resources. Despite this, our review found that caregivers and patients preferred online engagement methods, as geographical and time constraints were removed (C. Armstrong et al., 2019; Brouwers et al., 2018). Some advantages of using online methods are that they provide opportunity for transparency in responses and allows representation of diverse patient perspectives (Grant et al., 2018). Grant and colleagues (2018) found online patient engagement was most successful when the guideline development group properly trains patients to ensure understanding of tasks. Two other reviews by Kelson et al. (2012) and Légaré et al. (2011) also emphasised the importance of having proper training and support for all to enhance collaborative work. Proper education, training, and support for all interest-holder groups helps to create a better understanding of the guideline development process (Boivin et al., 2010; Brouwers et al., 2017, 2018; Chalmers et al., 2017; Duff et al., 1996; Hämeen-Anttila et al., 2016; Jarret, 2004; van de Bovenkamp & Zuiderent‐Jerak, 2013). This training can come in the form of online methods or in-person orientation sessions (Boivin et al., 2010; Brouwers et al., 2017, 2018; Chalmers et al., 2017; Duff et al., 1996; Hämeen-Anttila et al., 2016; Jarret, 2004; van de Bovenkamp & Zuiderent‐Jerak, 2013).
Finally, our review found that resource constraints were barriers for engaging patients, caregivers, patient organisations and healthcare providers as there is a high level of coordination needed for interest-holder engagement (Brouwers et al., 2017; Den Breejen et al., 2014; Hämeen-Anttila et al., 2016; Ilott et al., 2006; van Wersch & Eccles, 2001). Reviews by Kelson et al. (2012), Légaré et al. (2011), and Pinho-Gomes et al. (2022) also found that involving patients can be resource intensive. A review by Cluzeau et al.(2012) mentioned that engaging and involving interest-holders can be costly for organisations with insufficient funds and may be a barrier in guideline development. Our review also found the time required for guideline development was a barrier because many interest-holders are required to work after hours on their own time while maintaining their own jobs. Reviews by Légaré et al. (2011) and Pinho-Gomes et al. (2022) also emphasised how certain interest-holders do not have time to engage in guideline development. It is not practical for members to take time off work and voluntarily contribute to guideline development (Légaré et al., 2011). Interest-holders may be more engaged if there is financial compensation for involvement, for example covering travel and printing costs (M. J. Armstrong et al., 2017; Boivin et al., 2010; Brouwers et al., 2017, 2018; Chalmers et al., 2017; Duff et al., 1996; Ilott et al., 2006; Jarret, 2004). A review by Légaré et al. (2011) also emphasised that financial reimbursement and assistance can help with interest-holder engagement. Financial aid can come in the form of covering travel expenses, loss of earnings, or provision of childcare (Légaré et al., 2011). Compensation can also address equity concerns by ensuring that underrepresented groups are not disadvantaged due to their inability to engage in guideline development without financial support. For example, providers of care may be able to contribute without financial compensation due to institutional support, but other interest-holders may rely on honoraria to cover costs associated with their participation. However, there is concern that financial compensation could introduce biases or conflicts of interest. Transparency around compensation is, therefore, key to maintaining trust in the process. Co-authorship of guideline publications also represents another form of acknowledgement which has traditionally been reserved for academic principal investigators or providers of care. However, there has been a shift toward recognizing the contributions of other interest-holders, such as patients, in guideline publications. Consequently, organizations must develop clear criteria for co-authorship that reflect their different types of contributions to guideline development.
Authors’ Conclusions
Implications for Practice and Policy
This review identified barriers and facilitators to interest-holder engagement in health guideline development, allowing for the collation of current information on what works well in practice. The facilitators found will be used to develop an extension to the GIN-McMaster Checklist for Guideline Development and will support future efficient and productive multi-interest-holder work. We expect the findings to be considered by guideline development organisations such as GRADE, WHO, NICE and GIN. Notably, the majority of the current evidence focuses on patient engagement, and there are significant gaps in the literature which obscure or do not report on the engagement preferences of other interest-holder groups. Additionally, facilitators were not identified for all checklist topics so additional research should be done in order to understand how to best deal with known challenges. For example, how to manage diverging disciplinary perspectives, how to recruit interest-holders with diverse characteristics and experiences, or how to sustain interest-holder engagement to support future updates. Additionally, this work could be used as the basis for toolkits, template (e.g. role description) development and testing to offer guidance on ‘maximising’ interest-holder engagement. It argues for the development of training courses and tools to support meaningful involvement of all interest-holders, both public and professional. Taking a multi-interest-holder engagement approach to the development of health recommendations supports the growing movement of co-production in health care and policy, whereby patients/public are equal partners in shaping the healthcare system, including health guideline development (e.g. Batalden et al., 2015).
Implications for Research
The focus of this review was on barriers and facilitators to interest-holder engagement in the development of guidelines. As a follow up, further research should be done on the dynamics of guideline development groups, how these interactions are perceived by the group, and their impact on decisions that are made during the course of development (e.g. via participant observation). In addition, research on the implementation of guidelines and the impact of multi-interest-holder development on successful uptake and use in practice is needed.
Exploring the impacts and possible benefits of interest-holder involvement and co-production would be useful to guideline development organisations globally. A deeper dive into broader interest-holder engagement literature could be done to compare and contrast barriers and facilitators in other health practice and research-related activities as these may apply to specific aspects of guideline development and group work. Little information was found for several interest-holder groups under review, as most studies centred around patients, caregivers, and healthcare providers. It may be that their involvement is more often assumed and therefore less controversial, or they continue to be excluded and understudied. Future studies should explore the engagement of these interest-holder groups, such as policymakers, payers of services, payers of research, peer review editors and product makers. These future studies could consider employing longitudinal methods to observe interest-holder engagement over time or using mixed-method approaches to capture both qualitative and quantitative data which could provide a more comprehensive understanding of the impact of engagement. Additionally, future research could take a participatory action research (PAR) approach to actively involve interest-holders in the research process, which could enhance the relevance of findings and reduce bias introduced through author observations. Notably, few studies reported equity characteristics of interest-holders, and future research should consider how best to engage interest-holders from diverse backgrounds (e.g., ethnic or language groups).
Supplemental Material
Supplemental Material - Barriers and Facilitators to Interest-Holder Engagement in Health Guideline Development: A Qualitative Evidence Synthesis
Supplemental Material for Barriers and Facilitators to Interest-Holder Engagement in Health Guideline Development: A Qualitative Evidence Synthesis by Olivia Magwood, Jennifer Petkovic, Lyubov Lytvyn, Joanne Khabsa, Andrea Raymond, Marisha Palm, Danielle Pollock, Pauline Campbell, Thomas W. Concannon, Alex Young Soo Lee, Nicole Skoetz, Sean Grant, Sumanth Kumbargere Nagraj, Jessica Tiilikainen, Arnav Agarwal, Heather Limburg, Leonila F. Dans, Alison Riddle, Elie A. Akl, Vivian Welch, Peter Tugwell in Campbell Systematic Reviews.
Supplemental Material
Supplemental Material - Barriers and Facilitators to Interest-Holder Engagement in Health Guideline Development: A Qualitative Evidence Synthesis
Supplemental Material for Barriers and Facilitators to Interest-Holder Engagement in Health Guideline Development: A Qualitative Evidence Synthesis by Olivia Magwood, Jennifer Petkovic, Lyubov Lytvyn, Joanne Khabsa, Andrea Raymond, Marisha Palm, Danielle Pollock, Pauline Campbell, Thomas W. Concannon, Alex Young Soo Lee, Nicole Skoetz, Sean Grant, Sumanth Kumbargere Nagraj, Jessica Tiilikainen, Arnav Agarwal, Heather Limburg, Leonila F. Dans, Alison Riddle, Elie A. Akl, Vivian Welch, Peter Tugwell in Campbell Systematic Reviews.
Supplemental Material
Supplemental Material - Barriers and Facilitators to Interest-Holder Engagement in Health Guideline Development: A Qualitative Evidence Synthesis
Supplemental Material for Barriers and Facilitators to Interest-Holder Engagement in Health Guideline Development: A Qualitative Evidence Synthesis by Olivia Magwood, Jennifer Petkovic, Lyubov Lytvyn, Joanne Khabsa, Andrea Raymond, Marisha Palm, Danielle Pollock, Pauline Campbell, Thomas W. Concannon, Alex Young Soo Lee, Nicole Skoetz, Sean Grant, Sumanth Kumbargere Nagraj, Jessica Tiilikainen, Arnav Agarwal, Heather Limburg, Leonila F. Dans, Alison Riddle, Elie A. Akl, Vivian Welch, Peter Tugwell in Campbell Systematic Reviews.
Footnotes
Acknowledgements
The authors would like to thank Maureen Smith and Pearl Atwere for the critical role they played in the conception of this work and their contributions to the MuSE Consortium; Layal Hneiny of the American University of Beirut who helped refine and translate the search strategies; Sabrina Price, Jennifer Ogborogu, Rita Jarade, Rosiane Simeon, Angeline Pink, Caitlin Shyng, and Thomas Grant for their assistance in screening and data extraction; Gabriela Suka-Seignoret and Victoria Howard for assistance with writing; Reem Mustafa for contributions to the protocol and conception of this work.
Author Contributions
ORCID iDs
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Canadian Institutes for Health Research (FRN 155970).
Declarations of Conflicting Interests
The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: O Magwood, S Grant, P Tugwell, TW Concannon, EA Akl, and V Welch are co-authors of studies which have been included in this review. Authors of included studies were not involved in the decision to include, critical appraisal or data extraction of studies on which they were a coauthor.
V Welch is Editor-in-Chief of the Campbell Collaboration and was not involved in editorial decisions regarding this review.
P Tugwell provides independent medical consultation professional services to the Reformulary Group. He is [unpaid] Chair of the Management Group of a registered non-profit independent medical research organization, OMERACT, whose goal is to improve and advance the health outcomes for patients suffering from musculoskeletal conditions. OMERACT receives arms-length funding from 11 companies (Abbvie, Astra Zenaca, Aurinia, BMS, Centrexion, GSK, Horizon Pharma Inc, Janssen, Novartis, Pfizer & Sparrow).
S Grant’s spouse is a salaried employee of, and owns stock in, Eli Lilly and Company. SG has accompanied his spouse on company-sponsored travel.
S Kumbargere Nagraj is a Clinical Editor of Cochrane Oral Health and Co-convenor of Cochrane Priority Setting Methods Group.
The other authors declare no conflicts of interest.
Differences Between Protocol and Review
None.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
