Abstract
The European Society of Gastrointestinal Endoscopy (ESGE) and United European Gastroenterology (UEG) have a vision to create a thriving community of endoscopy services across Europe, collaborating with each other to provide high quality, safe, accurate, patient-centered and accessible endoscopic care. Whilst the boundaries of what can be achieved by advanced endoscopy are continually expanding, we believe that one of the most fundamental steps to achieving our goal is to raise the quality of everyday endoscopy. The development of robust, consensus- and evidence-based key performance measures is the first step in this vision. ESGE and UEG have identified quality of endoscopy as a major priority. This paper explains the rationale behind the ESGE Quality Improvement Initiative and describes the processes that were followed. We recommend that all units develop mechanisms for audit and feedback of endoscopist and service performance using the ESGE performance measures that will be published in future issues of this journal over the next year. We urge all endoscopists and endoscopy services to prioritize quality and to ensure that these performance measures are implemented and monitored at a local level, so that we can provide the highest possible care for our patients.
Abbreviations
adenoma resection rate Appraisal of Guidelines for Research and Evaluation Assessing the Methodological Quality of Systematic Reviews American Society for Gastrointestinal Endoscopy Complete Adenoma Resection [study] cecal intubation rate colorectal cancer expression of interest endoscopic retrograde cholangiopancreatography European Society of Gastrointestinal Endoscopy gastrointestinal Grading of Recommendations Assessment, Development and Evaluation Importance, Scientific acceptability, Feasibility, and Usability National Quality Measures Clearinghouse post-colonoscopy colorectal cancer population/patient, intervention, comparison, outcome, study design Quality Assessment Tool for Diagnostic Accuracy Studies Quality Improvement Committee Scottish Intercollegiate Guidelines Network United European Gastroenterology
The importance of quality
Tens of millions of people undergo endoscopic procedures every year in Europe. Endoscopy is the pivotal investigation in the diagnosis of gastrointestinal pathology and a powerful tool in its management. High quality endoscopy delivers better health outcomes and a better patient experience. 1 yet there is clinically significant variation in the quality of endoscopy currently delivered in endoscopy units.2–6
An example of this is post-colonoscopy colorectal cancer (PCCRC). It is known that the majority of PCCRCs arise from missed lesions (premalignant polyps or cancers]. or incomplete polypectomy.7,8 Back-to-back colonoscopy studies show that 22% of all adenomas are missed,9–14 and that there is a three- to sixfold variation in adenoma detection rates between endoscopists.15,16 Even when polyps are found, removal may be incomplete: the Complete Adenoma REsection (CARE) study concluded that 10% of nonpedunculated polyps of 5–20 mm and 23% of nonpedunculated polyps of 15–20 mm were incompletely resected. 17 Furthermore, low cecal intubation rates and poor bowel preparation regimens may explain the relative failure of colonoscopy to protect against proximal colorectal cancer that was found in many studies.18–25 This results in clinically important differences in quality of care and patient outcomes: a recent study in the UK demonstrated a more than fourfold variation in PCCRC rates between hospitals. 26
In the upper GI tract, gastric cancers and precursor lesions are frequently missed: in one series, 7.2% of patients with gastric cancer did not have the lesion detected at endoscopy performed in the preceding 1 year. Of these cases, almost three quarters were felt to be due to endoscopist error. 27 Equally, in ERCP, which is one of the most complex and highest risk procedures performed regularly in endoscopy practice, there is evidence of wide variation in both completion and complication rates.28–35
Performance measures
Providers and users of services can only know whether their service is delivering good quality care if it is measured. Performance measures are measurements that are used to assess the performance of a service or aspect of a service; other terms used for these include quality measures, quality indicators, key performance indicators, or clinical quality measures. Evidence-based performance measures provide endoscopists and endoscopy units, both often working in relative isolation, with a framework and benchmark against which they can assess their service.
Knowledge of the significant variation in quality between endoscopists does not improve quality per se, but setting minimum and target standards within these measures incentivizes improvement: when clinicians and services see their own performance data, they act to improve them. Open publication of performance measures also permit users of the service to assess quality for themselves, thus making better informed choices and further incentivizing improvements in healthcare. However, although open publication has potential benefits, it can cause unintended damage if handled poorly, for example if data are open to misinterpretation or inappropriate comparison. Thus it is important to consider both the benefits and risks of open publication for each case.
The provision of high quality endoscopic care is complex, involving myriad people, processes, and equipment. Healthcare professionals work hard to deliver this service, yet failure of any aspect may result in suboptimal care and poor health outcomes. Performance measures help a service to identify, appraise, and monitor the key steps in the process and the key outcomes, showing where systems are suboptimal and whether the service is providing high quality patient-centered healthcare.
Carefully constructed performance measures should allow providers to identify and address specific deficits in their service, resulting in better patient outcomes. Good performance measures should therefore correlate with an important health outcome. These measures should be evidence-based, clear, objective, reproducible, and realistic. They should also be practical to measure and meaningful for their target audience (for example endoscopists, patients, or healthcare providers). In an ideal construct, there should be a small number of carefully selected performance measures assessing all important aspects of the service (domains). Each measure assesses performance from a specific angle. Together they provide a holistic snapshot of the quality of the service. Some performance measures may relate to broad procedures (for example, cecal intubation rate), whereas others may relate to specific steps in a specific procedure (for example the optimal biopsy strategy for surveillance of Barrett’s esophagus).
Performance measures can be used to measure the quality of organizational structure, healthcare processes, or clinical outcomes. They can be applied in the pre-, intra- or post-procedural time periods.
Performance measures describe what to measure. However, it is usually desirable to take this further, identifying a minimum standard and a target standard within the measure. For example, it might be decided that cecal intubation rate is an important performance measure of colonoscopy; within this, a minimum standard might be set at 90% or 95%, with a target standard of 97%. Whereas performance measures will remain relatively static over time, the standards within such measures will be more dynamic, changing over time as techniques and technology improve. Moreover, the standards may vary according to procedure: for example, the minimum standard for adenoma detection rate will be higher for diagnostic colonoscopy performed because of fecal occult blood findings compared with colonoscopy prompted by symptoms. Occasionally no clear minimum standard currently exists for a performance measure (for example, patient comfort), yet its assessment may still be considered important. These are sometimes described as “auditable outcomes,” and it is hoped that in time, further research will help determine appropriate standards. Owing to small sample size, rates for rare events, such as missed cancers, may be best examined at endoscopy unit level rather than endoscopist level, whilst a qualitative review of each case is also performed (root cause analysis).
Terminology used in measuring quality
The ESGE Quality Improvement Initiative
The ESGE Quality Improvement Committee (QIC) was instigated in 2013. Its aims are:
To improve the global quality of endoscopy and the delivery of patient-centered endoscopy services To promote a unifying theme of quality of endoscopy within ESGE activities, achieved by collaborating with other ESGE committees and working groups and underpinned by a clear quality improvement framework To assist all endoscopy units and endoscopists in achieving these standards.
QIC committee membership comprises the QIC chairperson (M.R.), ESGE president and president-elect, chairs of the other three ESGE committees (guidelines, education and research) and chairs of QIC working groups.
A QIC strategy was developed to aid fulfilment of ESGE QIC aims. Quality improvement is a dynamic process and as such the strategy details will evolve over time, although the broad quality remit will not. An initial key objective was to help improve the quality of gastrointestinal endoscopy by producing a framework of performance measures for endoscopy, including quality of independent endoscopists and quality of endoscopy services (covering all aspects of the service including equipment, decontamination, waiting times, and patient experience), by developing robust, evidence-based performance measures. The aim of this was to set a minimum standard for individual endoscopists and for the endoscopy service, and to permit endoscopy units to measure their services against this patient-centered framework.
It was determined that such performance measures should be constructed using a rigorous evidence-based consensus process, incorporating a wide variety of stakeholders, including patients, from as wide a geographical area as possible. The aim was to delineate the core domains of a quality endoscopy service, to identify performance measures within each domain, and precisely to define and describe a small number of key performance measures covering each domain.
As the project fulfilled a key aim of the UEG Strategic Plan 2015–2018, ESGE approached UEG regarding potential collaboration and UEG agreed to this collaboration. Both ESGE and UEG co-funded the project and provided additional project governance.
The QIC committee created four working groups related to different areas of the gastrointestinal (GI) tract: upper GI, lower GI, pancreatobiliary, and small-bowel. A fifth “Endoscopy Service” working group was also created. An open call for expressions of interest (EOI) in participation was launched by ESGE, by emailing all individual members and all ESGE-affiliated endoscopy societies and by placing an article in the ESGE newsletter. A total of 90 EOIs were received from over 30 nations. The QIC committee nominated, approached, and appointed working group chairs and a meeting with these chairs was held to discuss the project in detail. Utilizing the list of EOIs, each working group chair established their working group membership, aiming to ensure as wide a geographical spread as possible, with between 10 and 20 members per GI tract group. Because of the nature of the Endoscopy Service group with regards to varying practice between nations, membership of this working group was deliberately larger and each ESGE-affiliated national endoscopy society was asked to nominate an individual to participate in the group, which comprised 34 members. No individual was permitted to be in more than one group. The American Society for Gastrointestinal Endoscopy (ASGE) was approached regarding collaborative involvement and agreed to provide input specifically into the small-bowel working group, along with overall comment or endorsement of the project output as appropriate.
The QIC committee contracted an expert team of methodologists to provide methodological support and to conduct the detailed literature searches (Literature Group). The Literature Group leader (C.S.) was co-opted onto the QIC committee for the duration of the project. To facilitate the program, a bespoke web-based platform was commissioned (ECD Solutions, USA). Within this platform, modules were created corresponding to the steps in the development process. All working group members had access to these modules, permitting both open and anonymized discussion around each aspect of the performance measure development. An expert in guideline methodology with significant prior experience of working with similar web-based platforms (C. Bennett) was commissioned to facilitate the integration of the information technology component.
Performance measures project process
Performance measures project: process steps
QIC, Quality Improvement Committee; SIGN, Scottish Intercollegiate Guidelines Network; GI, gastrointestinal; PICOS, population/patient, intervention, comparison, outcome, study design; ISFU, Importance, Scientific acceptability, Feasibility, and Usability; GRADE, Grading of Recommendations Assessment, Development and Evaluation; ESGE, European Society of Gastrointestinal Endoscopy: UEG, United European Gastroenterology.
Modified Delphi consensus process
Each working group developed an exhaustive list of potential areas for literature review, using the PICOS (Population/Patient, Intervention, Comparison, Outcome, Study design) process.39–41 The questions were focused on the assessment of the relationship between specific indicators and procedure outcomes (e.g. completion rate) or patient outcomes (e.g. interval cancer rate, change in clinical management). PICOS were reviewed by the Literature Group and revisions made until a final precisely defined list was reached. The PICOS components of each prioritized question were used by the Literature Group to define specific keywords for the comprehensive bibliographic searches. If more than one comparison was deemed to be relevant, the results of each comparison were reported.
Searches were performed on the Cochrane Central Register of Controlled Trials (CENTRAL), Medline and Embase, from 1 January 2000 to 28 February 2015, using MESH terms and free-text words, without language restriction. In the first instance systematic reviews were searched. If updated systematic reviews addressing the PICOS questions were retrieved, the search for primary studies was limited to those studies published after the last search date of the most recently published systematic review. If no systematic reviews were found, a search of primary studies since 2000 was performed. In order to avoid repetition or double counting of primary studies, where a literature search retrieved many systematic reviews addressing the same PICOS question, only the best systematic review, based on the evaluation of their methodological quality, update of the bibliographic search, level of overlapping, and quality of evidence of included primary studies, was considered for data extraction.
A hierarchy of the study designs to be considered for each type of question (e.g. on effectiveness, diagnostic accuracy, acceptability, and compliance) was produced by the epidemiologists of the Literature Group. For effectiveness questions, randomized controlled trials were considered as the best source of evidence and were searched in the first instance. For diagnostic accuracy questions, cross-sectional studies with verification by reference standard were considered as the best source of evidence.
The risk of bias of included studies was assessed using the following validated checklists:
systematic review: AMSTAR (Assessing the Methodological Quality of Systematic Reviews) checklist
42
randomized controlled trials: The Cochrane Collaboration’s tool for assessing risk of bias in randomized trials
43
cohort studies, case–control studies and cross-sectional surveys: Newcastle-Ottawa Scale
44
diagnostic accuracy studies: QUADAS 2 (Quality Assessment Tool for Diagnostic Accuracy Studies 2) checklist
45
interrupted time series analysis: criteria suggested by the Cochrane Effective Practice and Organisation of Care Review Group.
46
The draft results of the bibliographic search and of the selection process produced by the Literature Group were reviewed by the clinical experts of the working groups, to determine whether the inclusion of additional evidence or the exclusion of nonrelevant papers was required. Once necessary revisions were made, for each question or group of questions pertaining to the same topic, the Literature Group provided an evidence table with the main characteristics of each included study (study design, objective of the study, comparisons, participant characteristics, outcome measures, results, risk of bias). They also provided a summary document with a description of the search strategy used for each database, the overall number of titles retrieved, and the number of potentially relevant studies acquired in full text; the number of studies finally included was given, as well as a synthesis of their characteristics and risk of bias, and of their results, overall conclusions, and quality of evidence.
An overview of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) system. 47
Once the literature review was completed, initial draft evidence statements with comprehensive supporting documentation were uploaded onto a customized web platform, for all working group members to review and comment in a modified Delphi process (see Table 3), to allow modification and to identify additional references. Where necessary, further literature reviews were undertaken and further revisions made in subsequent voting rounds.
Importance, Scientific acceptability, Feasibility, and Usability (ISFU) system, customized and adapted to our working group needs
Customized and adapted descriptive framework for each final performance measure
Along with the final list of precisely defined key performance measures, the working groups compiled a longer list of other performance measures that had been identified during the development process, a list of areas with weak evidence base for priority research, and a list of training/educational needs. The final draft was then reviewed by the ESGE QIC Committee and the ESGE Governing Board. Finally, review and approval was obtained from ESGE-affiliated national societies, UEG, ASGE, and patient groups.
The ESGE quality improvement vision
ESGE and UEG have a vision to create a thriving community of endoscopy services across Europe, collaborating with each other to provide high quality, safe, accurate, patient-centered, and accessible endoscopic care. Whilst the boundaries of what can be achieved in advanced endoscopy are continually expanding, we believe that one of the most fundamental steps to achieving our goal is to raise the quality of everyday endoscopy. The development of robust, consensus- and evidence-based key performance measures is the first step in this vision.
Implementing performance measures, along with additional measures such as structured training programs, can result in significant improvement in endoscopy quality. In the UK for example, a decade of quality improvement initiatives resulted in cecal intubation rate improving from 76.9% to 92.3%. 52
Having a performance measure does not result in improved health outcomes per se: in order to improve quality, it is essential to measure local performance regularly against this benchmark. Services and individuals are unlikely to improve unless they are aware of their performance and how it compares with benchmark performance measures. Measuring allows the identification of potential underperformance, which provides an opportunity for discussion and support for the endoscopist. In addition, the simple act of monitoring a service will improve performance (the “Hawthorne effect”): it is powerful, essentially free, and results in improved quality of patient care.
The standardization of performance measure definitions and measurement methodology is crucial to permit comparative assessment. Quality improvement requires political will. At a local level, it requires support from hospital management. Whilst not essential, the best examples of quality improvement in endoscopy have also had commitment from, indeed have often been led by, regional or national authorities and we call upon such organizations to share responsibility for and to facilitate this program. The implementation of appropriate information technology infrastructure, based around electronic endoscopy reporting systems, is an important step in allowing timely data collection and automated, standardized performance measure reporting.
A strong case can be made for setting a minimum number of procedures per endoscopist per year. Firstly, a large sample size increases the accuracy of the performance measurement (i.e., it reduces the probability that apparent underperformance is a chance event). Secondly, there is evidence that endoscopy proficiency increases with increasing number of procedures performed, and that endoscopy complications are more common with endoscopists who perform fewer procedures per year 1 ; this is also well described in many other clinical areas such as surgery. 53 A trend towards fewer endoscopists each performing more procedures may be appropriate, and setting a minimum number of procedures per year for endoscopists may be one strategy to improve quality.
It is important that we help endoscopists with lower levels of performance to improve. Quality assurance should be about improvement, not punishment. One of the biggest gains in endoscopy quality improvement would be to raise the standards of the lower performers to above minimum quality standard thresholds. Various organizations have developed structured processes for the management of underperforming endoscopists, and experience shows that when handled sensitively but robustly, most endoscopists embrace such support. However, there may at times be barriers to the uptake of endoscopy quality improvement by individuals and even services, ranging from complacency (“I’m fine and don’t need to measure”) to fear that one’s abilities might be demonstrated to be suboptimal. The latter may be particularly relevant if there are financial or service imperatives to continue with the status quo. Nevertheless, we owe it to our patients to overcome these barriers to ensure that endoscopy is of the highest quality.
ESGE and UEG have identified quality of endoscopy as a major priority. We recommend that all units develop mechanisms for audit and feedback of endoscopist and service performance, using the ESGE performance measures that will be published in future issues of Endoscopy over the next year. Regional and national organizations have a responsibility to support and, where required, provide resources for such quality improvement initiatives. We urge all endoscopists and endoscopy services to prioritize quality and to ensure that these performance measures are implemented and monitored at a local level, so that we can provide the highest possible care for our patients.
Competing interests
Competing interests: M. Rutter’s department receives research funding from Olympus for a colitis surveillance trial (2014 to present). C. Senore’s department receives PillCam Colon devices from Covidien-Given for study conduct, and loaner Fuse systems from EndoChoice. R. Bisschops has received: speaker’s fees from Covidien (2009–2014) and Fujifilm (2013); speaker’s fee and hands-on training sponsorship from Olympus Europe (2013–2014); speaker’s fee and research support from Pentax Europe; and an editorial fee from Thieme Verlag as co-editor of Endoscopy.
R. Valori is a director of Quality Solutions for Healthcare, a company providing consultancy for improving quality and training in healthcare. C. Spada has received training support from Given Imaging (2013 and 2014). M. Bretthauer receives funds from Thieme Verlag for editorial work for Endoscopy. C. Bennett owns and works for Systematic Research Ltd, and received a consultancy fee from ESGE to provide scientific, technical, and methodological expertise for the present project. C. Hassan has received equipment on loan from Fujinon, Olympus, Endochoice, and Medtronic; and consultancy fees from Medtronic, Alpha-Wasserman, Norgine, and EndoChoice. C. Rees’s department receives research funding from Olympus Medical, ARC Medical, Aquilant Endoscopy, Almirall, and Cook (from 2010 to the present). M. Dinis-Ribeiro receives funds from Thieme Verlag for editorial work for Endoscopy; his department has received support from Olympus for teaching protocol (from August 2014 to July 2015). T. Ponchon has received: advisory board member’s fees from Olympus, Ipsen Pharma, and Boston Scientific (2014 and 2015) and from Cook Medical (2014); speaker’s fees from Fujifilm, Ipsen Pharma, and Olympus (2014 and 2015) and from Covidien (2014); training support from Ferring (2014); and research support from Boston Scientific and Olympus (2014 and 2015). P. Fockens has been receiving consulting support from Olympus, Fujifilm, Covidien, and Creo Medical. L. Aabakken, C. Bellisario, D. Domagk, T. Hucl, M. Kaminski and S. Minozzi, have no competing interests.
Footnotes
Acknowledgments
The authors gratefully acknowledge the contributions from: Stuart Gittens, ECD Solutions in development and running of the web platform; Iwona Escreet and all at Hamilton Services for project administrative support; The Scottish Intercollegiate Guidelines Network, especially Duncan Service, for hosting the critical appraisal module; and The Research Foundation - Flanders (FWO), for funding for Prof. Raf Bisschops.