Abstract
The European Society of Gastrointestinal Endoscopy (ESGE) and United European Gastroenterology present a list of key performance measures for endoscopy services. We recommend that these performance measures be adopted by all endoscopy services across Europe. The measures include those related to the leadership, organization, and delivery of the service, as well as those associated with the patient journey. Each measure includes a recommendation for a minimum and target standard for endoscopy services to achieve. We recommend that all stakeholders in endoscopy take note of these ESGE endoscopy services performance measures to accelerate their adoption and implementation. Stakeholders include patients and their advocacy groups; service leaders; staff, including endoscopists; professional societies; payers; and regulators.
Introduction
The quality of gastrointestinal endoscopy is important to patients. It is known that there is considerable variation in the quality and safety of endoscopy, indicating significant room for improvement.1–3 Historically, the focus of quality and safety has been on the performance of individual endoscopists, with the definition and measurement of performance metrics, and the use of these to target interventions designed to improve performance. There has been less focus on the environment within which endoscopists work, and the role or responsibility of an endoscopy service in the quality improvement cycle.
An endoscopy is part of a patient’s diagnostic or therapeutic journey. What happens before and after the procedure impacts on his or her experience and safety. An endoscopist performs the procedure, but he or she is dependent on a team to perform the procedure well and safely. Thus, the quality and safety of endoscopy depends on the environment within which endoscopists work (including the facilities and equipment) and the staff who work in that environment. Individual endoscopists and their staff have to be aware that there is room for improvement, believe that improvement will make a difference, be motivated to improve, participate in further development, and finally audit that improvement to ensure the required level has been achieved.
The endoscopy service has a key role to play in providing high-quality, safe, and patient-centred endoscopy. Collecting performance data and feeding it back to endoscopists provides metrics that may be used to target interventions designed to improve performance; for example, by motivating endoscopists to change their practice; providing time and opportunity to improve, possibly by in-house training; and finally by applying restrictions if the individual does not achieve the required levels of performance. Such ongoing monitoring, in the context of an understanding of the roles and responsibilities that an endoscopy service has in the quality improvement cycle, should lead to continuing improvement.
Within this context, the purpose of this guideline is to provide recommendations on what an endoscopy unit should have in place to meet these requirements. It is recognized that the recommendations may require new roles and information-gathering systems, and that there will be implications for the types of staff and staffing levels. Therefore, to achieve the recommendations, there will need to be extra resource allocation. Moreover, we appreciate that an excellent patient experience, and high-quality and safe endoscopy brings potential savings. This guideline provides explicit recommendations about what is required to deliver a modern endoscopy service. We recognize that payers, the organizations within which endoscopy services sit, and those who allocate resources, as well as those who work within the service, already understand the importance of quality.
Methodology
The group followed the European Society of Gastrointestinal Endoscopy (ESGE) Quality Improvement Committee (QIC) performance measures processes, as outlined in an earlier paper. 4 The literature base that informs the requirements for endoscopy services is potentially huge, but there is paradoxically relatively little high-quality evidence on which to base recommendations. Moreover, technical and safety requirements are usually defined by current regulations and legislation in force in each country. A comparative analysis of national safety regulations in European Union (EU) countries is beyond the aims of this project. Therefore, on the basis that legislative requirements and regulations will be addressed within the organizational context, the group decided to focus on a review of general guiding principles that should be considered when organizing endoscopy services and evaluating their performance.
It was decided that the approach to the potentially vast literature was to focus the search on recently published guidelines or recommendations (details of the search strategy and results can be viewed in the Supporting Information, available online). Two consensus documents discussing quality and safety indicators common to endoscopy services,5,6 as well as the EU quality assurance guidelines focused on colorectal cancer (CRC) screening activity, 7 were retrieved and assessed using the AGREE II checklist. We performed a comparative analysis of these documents considering the domains addressed, the quality indicators (and eventually performance targets) proposed, including the rationale for the choice of the indicator, and finally a judgement of the quality of the evidence reported by each document.
In addition, the working group agreed to take note of a fourth document, the UK Endoscopy Global Rating Scale (GRS) that, while not published in a peer-reviewed journal and not based on literature directly related to health services, was nevertheless founded on a wide consensus established within a nation, over a 10-year period. The GRS framework 8 has been used to plan the organization of the endoscopy services involved in the National Health Service (NHS) bowel cancer screening programme. The GRS framework is now integrated into the Joint Advisory Group on Gastrointestinal Endoscopy (JAG) endoscopy service accreditation standards. 9 The working group agreed it would act as a ‘sense check’ on the other guidance. Another reason for including this approach was because it was the starting point for two of the published guidelines and has also been tested in experimental studies evaluating the impact of quality-promoting interventions on endoscopy outcomes.10–12 These studies support an association between procedural requirements indicated in the GRS recommendations and clinically relevant outcomes.
Endoscopy Global Rating Scale
The GRS 8 arose from a need to have a measure of performance of endoscopy services within England in 2004 and, ultimately, for all the countries of the United Kingdom. While the initial intention was to use the GRS as a performance measure, its main function has been to act as a service improvement tool – a roadmap for services to follow to improve the quality of care. It currently consists of a series of statements in 19 domains: quality and safety (6); patient experience (7); staffing (3); and endoscopist training (3). The statements within each domain are layered to provide a measure of performance from D to A. All endoscopy units in England are required to self-assess against the GRS at least once a year, and the output from this self-assessment forms part of an accreditation process.
The initial versions of the GRS grew from a consensus view of what an endoscopy service should have in place. It has been subject to constant challenge from the service and has undergone several reviews, the latest in 2016. Thus, as feedback from the entire endoscopy service has formed part of its evolution, one might consider the GRS has evolved with the ultimate consensus process. However, its weakness is that at no stage did the evolution involve a review of the relevant literature.
EU guideline on quality assurance of CRC screening
Chapter 5 of the EU guideline published in 2010 13 was devoted to recommendations for quality assurance of endoscopy services involved in CRC screening. The first step in the process was to agree fundamental principles on which the guidance would be based. The second step was to create a series of PICO (population; intervention; comparator; outcome) questions that were subject to evidence search and review. Finally, the authors of the chapter developed a set of recommendations (in line with the methodology of the other chapters of the guideline) that were graded in two domains: strength of the evidence and strength of the recommendation.
The recommendations were reached by consensus of the group without a Delphi process. The chapter was subject to formal review and feedback from external experts. In keeping with the paucity of high-quality literature in this area, many of the recommendations did not have a sound evidence base, but nevertheless came with strong recommendation.
Canadian consensus guideline
The Canadian guideline process 6 used the GRS as a starting point to generate a series of questions, with a subsequent search for and review of the evidence. Almost 2500 publications were identified. A working group was presented with summary evidence and required to vote on a series of recommendations using a Delphi process. The eventual output of this methodology was similar to the EU guideline: overall strong recommendations but weak evidence.
Quality Indicators for Gastrointestinal Endoscopy Units – ASGE Endoscopy Unit Quality Indicator Taskforce
The American Society of Gastrointestinal Endoscopy (ASGE) 5 created a taskforce with subgroups charged with identifying and reviewing the literature in five domains: patient experience; employee experience; efficiency and operations; procedure-related unit issues; and safety and infection control. The process followed three stages: systematic literature review; generation of potential endoscopy unit quality indicators; and rating of these potential indicators on several parameters by invited participants (beyond the taskforce) in two rounds of voting using a modified Delphi process. The taskforce reached consensus on a final set of endoscopy unit quality indicators. The outcome of this approach was similar to the EU and Canadian guidelines: strong recommendations based on weak evidence.
Development process
A critical appraisal of the four documents indicated that three of them were based on systematic extensive searches of the literature, which found relatively little substantive evidence. Furthermore, all four documents had used consensus methods to make recommendations. The ESGE Endoscopy Service Working Group, in consultation with the parent ESGE QIC, agreed that three of the four documents should form the basis of the initial draft of consensus statements, being the three that were based on systematic reviews of the literature. It was agreed that areas/domains deemed to be relevant by the working group but not covered by these three documents should be subject to further literature review. Four areas of interest were identified:
Determining the importance of leadership in an endoscopy service; The impact of programmes of monitoring and reviewing adverse events; The effect of recognition and reward systems for endoscopy staff; The effectiveness of objective setting and plans for improvements.
Clinical questions, structured using the PICO framework, were formulated/defined to inform searches for available evidence to support the performance measures related to these areas of interest (see Supporting Information, available online).
Summary position and a way forward
It is clear that there is a substantial literature relating to guidelines for endoscopy services, but that this literature is unable to answer key questions as there are few high-quality intervention studies that provide evidence on which to base recommendations for practice. This is perhaps not surprising, because the recommendations in these various guidelines are largely recommendations about process. Demonstrating that process impacts on quality and safety is difficult because patient outcomes are usually dependent on a variety of correctly applied processes. Teasing out the relative contributions is difficult and, in some situations, impossible. We searched for intervention studies, in addition to evidence found in previous guideline publications.
Terminology
The various guidance documents use different language to describe similar concepts: recommendations; indicators; metrics; measures; or facilities, services, and units. This guideline will use the words ‘services’ and ‘recommendation’ or ‘suggestion’. The recommendations are intended to help endoscopy services become better organized and more effective: a vehicle for service improvement. The word ‘suggestion’ is used when the strength of the recommendation is weaker but nevertheless positive.
Some jurisdictions may want to use the recommendations as part of a process of quality assurance, such as the JAG accreditation process in the UK. 9 If used for this purpose, it is suggested that the word ‘requirement’ be used in place of ‘recommendation’ for things that must be in place for the assurance process – whatever they might be. It is not expected that all the recommendations would become requirements. The ESGE is not in a position, and neither is it appropriate, to mandate requirements because different jurisdictions will have different challenges, varied demands on their services, and varied resources to meet them.
The performance measures are laid out in tabular form with the rationale and proposed minimum and target standards. Unlike the performance measures for endoscopic procedures, it has not been possible to recommend a target percentage. In most circumstances, the standard refers to a process or structure that needs to be in place: it is either in place or not in place. The 30 performance measures identified across nine different domains are summarized in Figure 1.
Overview of endoscopy services performance measures.
For performance measures referring to the four areas identified for further exploration, the results of the review of the literature and the evidence appraisal are discussed below the measure. For the other measures, we have indicated whether they were mentioned/considered by the reference documents and the eventual reported judgement about the quality of available evidence.
1 Domain: Leadership and organization
1.1 Leadership roles and responsibilities
Overall, the evidence about the impact of the introduction of a leadership team, with defined roles and responsibilities, and accountability on continued improvements in technique, quality, and safety of services/care provided, is sparse, heterogeneous, and of low quality (many of the studies are uncontrolled studies). The most reliable evidence is about the association of nursing leadership and patient outcomes, showing that relational leadership practices are positively associated with some categories of patient outcomes and nurse satisfaction. Walk rounds* seem to give promising results in increasing a climate of safety.
Finally, there is a randomized controlled trial that shows the positive impact on colonoscopy performance of targeting leaders of endoscopy services with an educational intervention aimed at both leadership and endoscopic skills. 21
1.2 Annual operational plan
2 Domain: Facilities and equipment
2.1 Review of facilities and equipment
2.2 Programme of inspection, calibration, and maintenance
2.3 Shortfalls of facilities and equipment are addressed
2.4 Compliance with national decontamination requirements
3 Domain: Quality
3.1 System to capture procedural indicators
3.2 Performance data fed back to endoscopists
Two Cochrane reviews22,23 and three systematic reviews1,24,25 providing evidence about the effects of audit and feedback interventions, as compared with usual care, were retrieved. Most studies considered in these reviews were focused on the assessment of the effects of audit and feedback on the practice of healthcare professionals, but some of them also assessed outcomes related to patients’ health. The available evidence suggests that audit and feedback generally lead to small, but potentially important, improvements in professional practice. The effects are generally small to moderate, and vary based on the way the intervention is designed and delivered.
Provider assessment and feedback strategies may be effective in increasing breast, cervical, and colorectal faecal occult blood test screening uptake, with positive effects observed from studies using both continuous and dichotomous outcome measures, while results were not consistent when assessing other patient-related outcomes, such as immunization rates or hypertension control.
There is a recent publication 26 that outlines a pragmatic approach to identifying and supporting underperformance of endoscopists. The proposed approach recognizes that performance may be influenced by several factors pertaining to individuals or to their departments. The strategies aimed to address underperformance should adopt a stratified approach, modulating the intensity of the interventions (ranging from feedback and audit to specific re-training) based on the level of risk to patient safety.
3.3 Action is taken for persistent underperformance
3.4 Register of who can perform which procedures
4 Domain: Safety
4.1 Policies in place to mitigate known risks
4.2 Known adverse events are captured
Three Cochrane reviews27–29 and one systematic review, 30 including randomized controlled trials and observational studies using a before/after, or interrupted-time-series design, were retrieved.
Evidence for effectiveness has been documented for approaches based on technique or personnel changes, for educational interventions, for interventions involving structured process changes (including triage protocols, feedback steps, addition of a checklist, and quality improvement processes), the adoption of technology-based system interventions (including computerized decision-support systems and alerting systems), or the introduction of additional review methods (i.e. an additional review step, usually by a separate reader).
Overall, all the processes and methods assessed seemed to be beneficial in reducing diagnostic errors. The evidence seemed strongest for approaches using technology-based systems (for example, text message alerting) and specific techniques (for example, testing equipment adaptations), which could be implemented in endoscopy settings. However, the studies were very heterogeneous for study design, settings, and outcomes.
4.3 Root cause analysis of major adverse events
4.4 Stop procedures when balance of benefits/risk is unfavourable
5 Domain: Appropriateness
5.1 Referral guidelines for all procedures
5.2 Compliance with guidelines is assessed
6 Domain: Information, consent, and further care
6.1 Informed consent complies with national requirements
6.2 Patient information available for all procedures
6.3 Comprehensive discharge information given to patient
7 Domain: Comfort, privacy, and dignity
7.1 Patient comfort assessment undertaken
7.2 Action taken to improve patient comfort
7.3 Environment suitable to preserve patient privacy
8 Domain: Staffing
8.1 Review of staffing in relation to activity
8.2 New staff have an induction
8.3 Staff are adequately trained for their role(s)
8.4 Methods in place to motivate staff
It was not possible to evaluate the efficacy of the recognition/reward of professionals as none of the retrieved studies24,31,32 assessed the efficacy of this component alone: one study assessed the impact of financial incentives, in combination with other interventions, such as goal setting or WHO-5 (multimodal strategy consisting of five components: system change, training and education, observation and feedback, reminders in the hospital, and a hospital safety climate).
8.5 Confidential reporting is available to staff
9 Domain: Patient involvement
9.1 Patient feedback is collected
9.2 Patient complaints and suggestions are reviewed
9.3 All patient feedback reviewed and acted upon
Implementation – accelerating adoption of the guideline
A guideline will not benefit patients if it is not used. At first sight it might seem that it is the responsibility just of the leaders of an endoscopy service to implement this guideline; however, while the leaders of endoscopy services are in most control of their units, there are many other stakeholders who have an influence on how quickly the guidance can be adopted. We provide here an idea of who should take note of this guideline and what they might do to accelerate its adoption.
Patients
Patients (and their carers/representatives) have a role in advising endoscopy services and providing feedback on the quality of the service. Patients are encouraged to demand that endoscopy services should achieve the recommendations and ask for an explanation if they are not met.
Payers
Payers should require endoscopy services to achieve the recommendations and be able to demonstrate that they have been achieved.
Endoscopy staff
Endoscopy staff are well placed to know whether the recommendations have been followed and should challenge both the wider organization and the service itself if there are shortcomings.
Endoscopy team leaders
Endoscopy team leaders should use the recommendations to structure their approach to improving their service and acquiring the resources they need to meet the recommendations. The recommendations will empower them to manage poor performance and bad behaviours.
Other groups and departments
The effectiveness of an endoscopy service depends on pathways both into and out of the service, much of which can be outside the direct control of the endoscopy service itself. The department that perhaps impacts most on endoscopy is pathology; for example, processes for samples; timeliness and method of reporting; integration of pathology into performance reports (such as adenoma detection rate); and review of complex pathology. It is vital for the endoscopy service to both identify other relevant groups (such as its referrers) and departments, and collaborate with them both to optimize the care of patients and the use of resources.
Wider organizations
The organizations within which endoscopy services are delivered should have an interest in ensuring the recommendations are met and that their endoscopy service has the necessary resources both to deliver the recommendations and to demonstrate they have been achieved.
Patient advocacy groups
Patient advocacy groups can use the guideline to hold endoscopy services to account to ensure they are patient centred and delivering high-quality and safe endoscopy. These groups might create ‘key’ questions, based on the recommendations, for patients to ask of endoscopy services, thereby empowering them to ensure they receive a high-quality service.
National professional societies
These societies should decide whether they have a role in the governance of the endoscopy service in their country, or whether they see their role as supportive and educational. At the very least, they should be recommending services follow the guideline. At the other extreme, they might implement a process to assess whether services meet the recommendations, as currently occurs in the UK.
Ministries of health
Health ministries should require endoscopy services to meet the recommendations in this guideline and ideally set up a process to assess services to ensure the recommendations are met.
Regulators of health
It is only expected regulators of health will become interested in these guidelines if they become requirements within a quality assurance infrastructure, as is occurring in England. Currently, the health regulator in England (the Care Quality Commission) is working with professional accrediting bodies, such as the JAG, to use accreditation of services to inform, improve, and reduce the burden of regulation.
ESGE
The next phase of work for the QIC Endoscopy Service Working Group of the ESGE will be to determine in what ways the ESGE can support its members and endoscopy services throughout Europe in the implementation of this guideline.
Addendum
There have been recent developments in the proposed methodology for assessing the strength of evidence supporting recommendations in clinical guidelines.
33
This new thinking recognizes that there are situations/topics where it is obvious that the proposed recommendations would do substantially more good than harm, even if there are no studies available specifically designed to answer the clinical question. Guyatt et al., in their paper entitled ‘Guideline panels should not GRADE good practice statements’,
33
state: ‘Good practice statements typically represent situations in which a large body of indirect evidence, made up of linked evidence including several indirect comparisons, strongly supports the net benefit of the recommended action’. In such cases it can be inappropriate, disproportionally time-consuming, and/or unrewarding to review the literature. Guyatt et al.
33
propose that some recommendations be considered for re-classification as ‘good practice statements’ when:
the statement is clear and actionable the message is really necessary the net benefit is large and unequivocal the evidence is difficult to collect and summarize specific issues, such as equity, need to be considered the rationale is explicit the approach is better than formal GRADEing.
In the light of this new recommendation on the need for evidence supporting clinical guidelines, it is clear that many (if not most) of the quality measures included in this guideline could be classified as good practice statements. It was decided, because the project had been implemented using the classic GRADE methodology and the evaluation had already been done, to continue with the GRADE process. However, the latest perspective on when and when not to perform a literature review supports the working group’s approach and conclusions that it is appropriate to include quality measures that do not have a traditional evidence base.
The quality measures have not been formally assessed against the criteria proposed for ‘good practice statements’; however, it is evident that the majority would, with the exception of ‘considering equity’, meet the criteria. It is expected that future iterations of this guideline will adopt the latest recommendations and consider which quality measures should be GRADEd and which should be judged against criteria for good practice statements.
Supporting information
The detailed literature searches performed by an expert team of methodologists, as well as evolution and adaptation of the different PICOs and clinical statements during the Delphi voting process can be viewed in Supporting Information on the ESGE website.
Online content viewable at: https://www.esge.com/performance-measures-for-endoscopy-services.html
Footnotes
Acknowledgments
The authors gratefully acknowledge the contributions from: Dr. Stuart Gittens, ECD Solutions in the development and running of the web platform; Iwona Escreet and all at Hamilton Services for project administrative support; Debbie Johnston, Craic Consultancy Ltd, for support creating the minimum and target standards; the Scottish Intercollegiate Guidelines Network for hosting the critical appraisal module; and the Research Foundation-Flanders (FWO) for providing funding for Professor Raf Bisschops; also thank you to all members of the wider working group who attended meetings, participated in discussions, and voted on the measures including Bamakhrama K. (UAE), Fretwell I. (UK), Hamoudi W. (Jordan), Kariis T. (Estonia), Lauri A. (Italy), Madácsy L. (Hungary), Narra Figueiredo P. (Portugal), Nashat A. (Egypt), Ørmeci N. (Turkey), Paraskeva K. (Greece), Patchett S. (Ireland), Stefanovic M. (Slovenia), Wurm Johansson G.G. (Sweden), Seebach L. (Switzerland), Stirna D. (Latvia); Nurses: Bakic B. (Croatia), Beilenhoff U. (Germany), Christensen J.S. (Denmark), Jorgensen A. (Norway), Martisen K. (Norway), Petraki S. (Greece); Patient representative: Gore-Booth J. (France – EuropaColon). United European Gastroenterology (UEG) supplied co-funding and additional project governance to this endeavour.
Competing interests
C. Bennett owns and works for Systematic Research Ltd., and received a consultancy fee from ESGE to provide scientific, technical, and methodological expertise for the present project (2014–2018). R. Bisschops has received: speaker’s fees from Covidien (2009–2014) and Fujifilm (2013); speaker’s fee and hands-on training sponsorship from Olympus Europe (2013–2014); speaker’s fee and research support from Pentax Europe; and an editorial fee from Georg Thieme Verlag as co-editor of Endoscopy. M. Bretthauer receives fees for being a member of the Norwegian government CRC screening advisory group (2012 to present) and receives funds from the American College of Physicians for editorial work for Annals of Internal Medicine. T. de Lange is a National Endoscopy Board’s member of the Norwegian Gastroenterology Association. M. Dinis-Ribeiro receives funds from Georg Thieme Verlag for editorial work for Endoscopy. M. Kaminski receives speakers and teaching fees and travel support from Olympus Erbe. C. Senore’s department received PillCAM2 Colon devices from Medtronics to conduct a comparative study (2014–2017); along with C. Belissario and S. Minozzi, he also received consultancy fees from ESGE to provide methodological expertise (PICOs evaluation, literature searches, and evidence summaries) for the present project (2014–2017). R. Valori is a director of AnderVal Ltd., a company providing endoscopy skills training (2015 to present). O. S. Balfaqih, G. Cortas, M. de Pater, D. Domagk, P. Eisendrath, P. Falt, C. Hassan, I. Koruk, A. Ono, C. J. Rees, N. Rustemović, M. Rutter, E. Schoon, C. Spada, and A. Veitch have no competing interests.
In brief
The European Society of Gastrointestinal Endoscopy (ESGE) and United European Gastroenterology (UEG) present a list of performance measures for endoscopy services, comprising 30 measures in nine domains. A systematic and scientifically based methodology was applied to substantiate the proposed measures with available evidence where possible. The measures include those related to the leadership, organization, and delivery of the service, as well as those associated with the patient journey. Each measure includes a recommendation for a minimum and target standard for endoscopy services to achieve. Adoption of these performance measures by all endoscopy services across Europe is recommended.
*
Regular rounds by senior leaders providing an informal method for leaders to talk with front-line staff about safety issues in the organization and to show their support for staff-reported errors, ensuring two-way communication, with both the executives and the staff talking honestly and listening carefully. Walk rounds can be conducted in patient care departments (such as the emergency department, operating rooms, and radiology), the pharmacy, and laboratories. This approach, or a customized version of it, can easily be adopted in the endoscopy environment.