Sage Journals: Discover world-class research

Abstract

Importance

Quality indicators are used to evaluate the quality of healthcare delivery and as a speciality, otolaryngology–head and neck surgery (OHNS) is beginning to transition toward this empirical understanding of healthcare quality and delivery.

Objective

To describe the number and quality of studies that have developed novel quality indicators for any subdiscipline in OHNS.

Design

We performed a database search of MEDLINE (Ovid), EMBASE (Ovid), Web of Science, and Cochrane Database of Systematic Reviews. We did not employ language or study-type restrictions and included studies published from database inception to October 2024.

Study Selection

Following abstract screening, 184 articles underwent full-text screen. Eligible studies developed quality indicators in any subdiscipline within OHNS. Article screening and full-text review was performed in duplicate.

Data Extraction and Synthesis

We extracted study-specific and methodological data in duplicate. Quality appraisal was assessed using the Appraisal of Indicators through Research and Evaluation instrument.

Results

We identified 10,592 studies, of which 25 studies developed new quality indicators. Quality indicator development studies primarily focused on otology/neurotology, pediatric OHNS, and head and neck surgery. Few studies investigated facial plastics, rhinology and skull base surgery, and laryngology. Most studies employed Delphi consensus methods and patient engagement was rare. Consensus methodology reporting was poor and indicators were often not validated. Outcome indicators were often measured with fewer studies investigation structure or process indicators.

Conclusions

Quality indicators may help standardize and improve patient care in OHNS. Future research should focus on structure and process indicators, while improving reporting, optimizing panel composition, and validating quality indicators.

Graphical Abstract

Keywords

quality indicators otolaryngology head and neck surgery quality improvement systematic review

Key Messages

• Quality indicator development research is ongoing in otolaryngology–head and neck surgery (OHNS)

• Most studies developed quality indicators in otology/neurotology, pediatric OHNS, and head and neck surgery with few studies investigating facial plastics, rhinology, and laryngology

• Optimizing consensus panel composition, improving reporting, and developing structure and process indicators will improve the impact of quality indicator research in OHNS

Introduction

Healthcare systems are shifting toward an empirical understanding of patient care.^1,2 Quality indicators are performance metrics that help standardize the evidence-based evaluation of healthcare quality and delivery, in addition to clinical outcomes.³ A common framework is the Donabedian model, which categorizes quality indicators into structure, process, and outcome measures (Table 1).¹ An important and renowned application of surgical quality indicators is the American College of Surgeons National Surgical Quality Improvement Program (NSQIP), which is a quality indicator database connecting healthcare institutions and surgical disciplines across the United States.⁴ Since the beginning of this quality initiative, NSQIP-affiliated institutions have helped to improve clinical outcomes and healthcare delivery, for patients and healthcare practitioners.^4
-6

Table 1.

Quality Indicators as Defined by the Donabedian Framework.

	Structure indicator	Process indicator	Outcome indicator
Definition	What is the context/setting in which healthcare is delivered to patients?	What actions are being undertaken to facilitate healthcare delivery to patients?	What is the outcome on patient health following healthcare delivery?
Examples	• Institution • Staff and equipment • Surgical volumes	• Preventive care • Diagnosis and treatment planning • Patient education	• Patient satisfaction • Morbidity and mortality • Health-related quality of life

Quality indicators are developed via a multistage process that requires analyzing peer-reviewed literature and clinical practice guidelines, while incorporating input from allied healthcare professionals, patients, and their families to generate target performance metrics for further scientific validation.^7,8 An often-used development method for quality indicators involves a consensus approach, or Delphi method, whereby an expert panel of medical professionals establish candidate quality indicators that are then subject to successive rounds of discussion and selection until final quality indicators are identified for further validation.⁹ Over the past decade, researchers in otolaryngology–head and neck surgery (OHNS) have begun generating quality indicators to standardize and improve healthcare quality and delivery. So far, however, there has not yet been a fulsome review of the breadth of quality indicators in OHNS. Hence, the objective of this scoping review was to characterize and evaluate the quality of studies that have developed quality indicators in OHNS.

Materials and Methods

This study was reported in accordance with the Preferred Reporting Items for a Systematic Review and Meta-Analysis (PRISMA) Scoping Review Extension Guidelines.¹⁰ As this is a scoping review, it did not require PROSPERO database registration.¹¹

Search Strategy

We reviewed all studies that reported quality indicators within OHNS in Medline, Embase, Web of Science, and Cochrane Database of Systematic Reviews from database inception to October 24, 2024, with the help of a librarian specialist (C.W.) (Appendix A). We performed a qualitative search of the reference lists of included articles and review articles. We did not include a language restriction. Further, a gray literature search of global professional organizations in OHNS and healthcare quality as well as gray literature databases was also performed (i.e., Google, Google Scholar, OAlster, OpenGrey). We searched the first 10 pages of all web page results.

Study Eligibility

We included studies that addressed quality indicators in any subdiscipline in OHNS, including facial plastics and reconstructive surgery, general OHNS, head and neck surgery, laryngology, otology and neurotology, pediatric OHNS, and rhinology and skull base surgery. We defined development studies as those that utilized any consensus methods or other methodologies to identify quality indicators and/or outcome sets. We included adult and pediatric patients in any clinical setting. We did not include pathologies or clinical interventions involving the esophagus or thoracic cavity. We also did not include quality indicators in radiation or medical oncology for head and neck oncology.

We included peer-reviewed original research, systematic reviews, consensus and clinical guidelines, and guideline extraction articles that developed novel quality indicators in the context of clinical care. There was no language restriction. We also excluded any narrative reviews and editorials. We excluded any studies that proposed or evaluated existing quality indicators from the final descriptive analysis.

Study Selection and Data Abstraction

Articles were reviewed following PRISMA guidelines (Figure 1). All review stages were completed by 2 reviewers (P.S., L.W.) and conflicts were resolved via discussion including 2 senior authors (S.S., J.C.). We utilized Covidence (Melbourne, Australia) for article screening. Data abstraction was performed by 3 authors (P.S., S.S., and L.W.), and all discrepancies were addressed by a senior reviewer (E.M.). We collected data pertaining to authorship, year of publication, study location, conflicts of interest, study design, methodology, as well as individual quality indicators and their associated details, including descriptor numerator and denominator.

Figure 1.

PRISMA flowchart for included studies. PRISMA, Preferred Reporting Items for a Systematic Review and Meta-Analysis.

Quality Appraisal and Statistical Analysis

All included studies that developed novel quality indicators were evaluated using the validated Appraisal of Indicators through Research and Evaluation (AIRE) tool.¹² The AIRE is a validated instrument that was designed to evaluate studies of quality indicators in healthcare.¹³ It comprised 20 items that address 4 quality domains: (1) purpose and organizational context, (2) stakeholder involvement, (3) scientific evidence, and (4) additional evidence. Each item is scored on a 4-point scale (i.e., 1: strongly disagree and 4: strongly agree). A standardized domain score was calculated: [(total score − minimum possible score)/(maximum score − minimum possible score) × 100%]. A score closer to 100% indicated higher methodological quality. Each domain was independently evaluated by 2 reviewers (P.S., L.W.) with discrepancies resolved by a third reviewer (S.S.). We analyzed all included studies via descriptive analyses, including measures of central tendency and standard deviation. We calculated frequencies and proportions for all categorical variables. All statistical analyses and tables were generated using Microsoft Excel (Ridgefield, WA, USA).

Results

We identified 10,592 studies from the database search wherein 25 studies met eligibility criteria (Figure 1).^14-38 All included studies developed new quality indicators in OHNS (Table 2). We found that most studies (n = 18, 72%) utilized Delphi, modified Delphi, or RAND-UCLA consensus methods. Candidate quality indicators were primarily selected from clinical practice guidelines and/or scientific literature (n = 21, 84%). These consensus studies primarily used an expert panel composed of medical specialists (n = 19, 76%) with few studies including patient or family engagement (n = 7, 28%). The most studied OHNS subdiscipline was pediatric OHNS (n = 7, 28%) with most studies evaluating otitis media or tonsillitis (Table 3). In otology and neurotology (n = 6, 24%), most studies evaluated tinnitus or vestibular disease. In head and neck surgery (n = 6, 24%), most studies evaluated head and neck cancer. Supplemental Table S1 describes the quality indicators selected in each included study.

Table 2.

Summary of Quality Indicator Development Studies (N = 25).

Study characteristic	No. of studies (%)
Publication year
2010—present	24 (96)
2000—2009	1 (4)
Study country
Canada	8 (32)
USA	6 (24)
Netherlands	3 (12)
UK	2 (8)
Australia	2 (8)
Spain	2 (8)
Portugal	1 (4)
Brazil	1 (4)
Consensus methods
Delphi/modified Delphi	13 (52)
RAND-UCLA appropriateness	7 (28)
Not reported	5 (20)
Expert panel composition
Medical specialists only	17 (68)
Medical specialists and other healthcare professionals^a	7 (28)
Not reported	1 (4)
Patient engagement
No patient involvement reported	18 (72)
Patient and/or patient-centered organizations	7 (28)

Abbreviation: CPG, clinical practice guideline.

Includes audiologists, nurses, speech-language pathologists, methodologists, psychologists, and administrators.

Table 3.

Summary by Otolaryngology–Head and Neck Surgery Subdiscipline (N = 25).

Otolaryngology–head and neck surgery subdiscipline	No. of studies (%)
Pediatric OHNS	7 (28)
Otitis media^a	3 (42.9)
Tonsillitis	2 (28.6)
Lymphatic malformations	1 (14.3)
Laryngeal anomalies	1 (14.3)
Otology and neurotology	6 (24)
Tinnitus/vestibular disorders	2 (33.3)
Hearing loss^b	1 (16.7)
Cochlear implants	1 (16.7)
Cerumen impaction^c	1 (16.7)
Cholesteatoma	1 (16.7)
Head and neck surgery	6 (24)
Head and neck cancer^d	5 (83.3)
Primary hyperparathyroidism	1 (16.7)
Rhinology and skull base surgery	3 (12)
Chronic rhinosinusitis	2 (50)
Acute rhinosinusitis	1 (25)
General OHNS	2 (8)
General OHNS	1 (50)
Dysphagia	1 (50)
Facial plastics and reconstructive surgery	1 (4)
Rhinoplasty	1 (100)

Abbreviation: OHNS, otolaryngology–head and neck surgery.

Includes acute otitis media, chronic suppurative otitis media, and otitis media with effusion.

Includes sudden sensorineural hearing loss.

Michel et al³⁰ also investigated allergic rhinitis.

Includes oral cavity, oropharyngeal, laryngeal, hypopharyngeal cancer, dysphagia in the context of head and neck cancer, and mandibular osteoradionecrosis.

Study Quality Appraisal

All studies were evaluated using the AIRE instrument (n = 25; Supplemental Table S2). We found that the mean value for the purpose domain was 63.8% (range: 27%-87%). The primary deficiency in this domain was poor reporting of the criteria for selecting quality indicators. The mean value for the stakeholder domain was 54.5% (range: 5.5%-78%) and its primary deficiency was a lack of patient/family and allied health professional engagement. The mean value for the scientific evidence domain was 55.8% (range: 0%-89%) with reporting deficiency in the search methods and types of resources identified for quality indicator development. The mean value for the additional evidence domain was 41.8% (range: 15%-74%). These studies reported construct validity with adequate detail; however, we found limited reporting with regards to quality indicator description and measurement, test reliability, risk adjustment, and quality indicator piloting.

Pediatric Otolaryngology–Head and Neck Surgery

There were 7 studies (28%) of pediatric OHNS that developed new quality indicators.^{15,16,18,21,23,25,34} Most studies investigated pediatric otitis media (Table 3).^21,23,34 Four studies used a Delphi/modified Delphi consensus method, 2 used the RAND-UCLA method, and 1 study did not report the consensus method. Expert consensus panels included surgeons and physicians in all studies, while 2 studies also included allied health professionals (e.g., audiologists, nurses, speech language pathologies).^23,34 One study incorporated patient and family surveys in developing outcome indicators for otitis media with effusion.²³ Three studies extracted candidate indicators from peer-reviewed clinical practice guidelines, while all remaining studies drew candidate indicators from the peer-reviewed literature and expert opinion.^18,21,25 Across all eligible studies, 113 novel quality indicators were developed using consensus methods: 72 (63.7%) were outcome indicators, 41 (36.3%) were process indicators, and none were structure indicators (Supplemental Table S1). Empirical validation was performed in one study.²⁵

Otology and Neurotology

There were 6 studies (24%) of otology and neurotology that developed new quality indicators (Table 3).^{20,22,30,35,37,38} Most studies evaluated tinnitus and Meniere’s disease. Two studies used the Delphi method while 2 studies used the RAND-UCLA method. Three studies (50%) extracted candidate measures from clinical practice guidelines. Two studies included patient input or health user stakeholders in their consensus panels.^22,35 Three studies extracted candidate quality indicators from clinical practice guidelines.^20,30,38 None of the studies performed validation of proposed indicators. Across all included studies, 41 novel quality indicators: 34 (82.9%) were outcome indicators, 7 (17.1%) were process indicators, and none were structure indicators (Supplemental Table S1).

Head and Neck Surgery

There were 6 studies (24%) of head and neck surgery that developed novel quality indicators (Table 3).^{24,27,29,32,33,36} Five studies investigated head and neck cancer and one investigated primary hyperparathyroidism. Three studies (50%) used the modified RAND-UCLA appropriateness method, and 3 studies (50%) used Delphi or modified Delphi methods. Two studies employed patient-based organizations while remaining studies used expert panels that included both physicians and allied health professionals. All studies derived candidate indicators from published clinical guidelines, peer-reviewed studies, and expert opinion. No studies performed any empirical testing of selected quality indicators. Across all studies, 166 novel quality indicators were developed: 83 (50%) were outcome indicators, 77 (46.4%) were process indicators, and 6 (3.6%) were structure indicators.

Rhinology and Skull Base Surgery

Three studies (12%) developed novel quality indicators in rhinology.^17,19,26 These studies investigated acute bacterial and chronic rhinosinusitis. One study developed quality indicators for both cerumen impaction and allergic rhinitis.³⁰ Two studies employed a RAND-UCLA method while the other study used a Delphi method. Three studies identified candidate quality indicators via clinical practice guidelines, while the other study identified candidate indicators via the peer-reviewed literature. Two studies described the expert panel to consist of physicians, while one study included patient engagement.²⁶ A total of 38 quality indicators were developed with 20 (52.6%) being process indicators and 18 (47.4%) being outcome indicators. None of these studies performed quality indicator validation.

Other Otolaryngology–Head and Neck Surgery Subdisciplines

Two studies investigated new quality indicators in general practice OHNS.^14,31 These studies evaluated inpatient OHNS and dysphagia. One study used the Delphi consensus method.¹⁴ These authors also included allied health specialists and medical specialists within their expert consensus panel. Moraes et al describe 12 process outcomes related to dysphagia and swallowing assessment.³¹ Manahan et al identify candidate quality indicators from clinical practice guidelines, but do not describe a selection method.²⁸ These authors identify 1 outcome and 3 process indicators. None of these studies performed quality indicator validation.

Gray Literature

We identified 2 citations: (1) entnet.org, which is hosted by the American Academy of OHNS and (2) healthmonix.org, which is hosted by a private company involved with healthcare quality improvement. There were 49 quality indicators involving OHNS and general medicine, of which 47 were process indicators and 2 were outcome indicators. We did not identify any studies appraising these quality indicators.

Discussion

With the success of the NSQIP, surgical disciplines continue to develop and validate quality indicators that connect healthcare networks and improve healthcare delivery.^8,39 Herein, we present a scoping review aimed at describing all published OHNS quality indicators so that investigators can identify gaps in their knowledge and application. Schneider and Lavin identified several quality improvement databases and registries that are being used in OHNS to improve patient care and healthcare efficiencies.⁴⁰ In our review, quality indicator studies primarily investigated patient management in otology, neurotology, pediatric OHNS, and head and neck surgery. These studies focused on otitis media, tinnitus and vestibular disorders, sudden and chronic hearing loss, tonsillitis, and head and neck cancer. Otology and neurotology are often limited by low-quality research and research volumes that make up less than half of all OHNS publications.^41,42 There has also been a paucity of high-quality pediatric OHNS research published over the past 3 decades.⁴³ Head and neck surgery is often studied in OHNS, but randomized trials in head and neck surgical oncology suffer from poor methodological quality.⁴⁴ Core outcome sets are being developed to help standardize outcome assessment and reporting in head and neck surgery.⁴⁵ Minimal quality indicator development research has been performed in facial plastics, rhinology and skull base surgery, and laryngology. A drive toward an evidence-based understanding and application of quality indicators will contribute to clinical research and standardization of healthcare quality in all subdisciplines of OHNS.

Most studies employed the Donabedian framework, wherein quality indicators were categorized as structure, process, and outcome measures. The Donabedian framework has been validated in other surgical specialities and may help facilitate the implementation of eHealth into healthcare systems.^46
-48 The Institute of Medicine described 6 domains of healthcare quality: patient safety, effectiveness, patient-centeredness, timeliness, efficiency, and equity that have been applied to medical populations, but we did not find any consistent use of this framework and it has been less applied to surgical patients.⁴⁹ When applying the Donabedian framework to OHNS quality indicators, we found that structure, process, and outcome indicators were all included, but most studies developed only process and outcome indicators. Similarly in vascular surgery, structure and process indicators were rarely studied when compared to outcome indicators.⁵⁰ In a review of surgical oncology studies, however, process indicators were studied more often than both structure and outcome indicators.⁵¹ In other non-OHNS surgical specialties, process indicators have shown to be better drivers of hospital quality improvement when compared to outcome indicators.^50,52 Further, Rademakers et al suggested that structure and process indicators were more likely to improve patient ratings of quality of care.⁵³ Hence, structure and process indicators are important for internal quality improvement, while outcome indicators help inform the public about institutional differences in healthcare.⁵⁴ As such, we suggest that in addition to outcome indicators, future studies focus on structure and process indicators as these are less often studied in OHNS but contribute to evidence-based healthcare quality improvement initiatives.

At present, there are no standardized methodologies or guidelines for conducting consensus research for developing quality indicators.^55
-57 In our review, we found that a Delphi or modified Delphi method was most often employed by authors, but reporting detail was often lacking, which is consistent with the literature.⁸ Further, consensus panels primarily included physicians but often did not gather the opinions of other stakeholder, such as administrators, allied health professionals, patients, and families, which is consistent with other studies.⁵⁸ In addition, quality indicators were primarily refined by local experts from regional hospital networks, but often did not incorporate opinions from other countries. We also found that quality indicators were not piloted or validated in these studies, nor were they provided to quality improvement entities (i.e., Agency for Healthcare Research and Quality) for feedback. Validating healthcare quality indicators is important as it prevents improper data entry and bias by providers while ensuring the chosen indicators are reflective of healthcare quality.^59,60 In evaluating these studies using the AIRE instrument, deficiencies existed in the lack of patient engagement and allied health professionals within the expert panel, poor reporting of consensus methodologies and methods used to identify candidate quality indicators, and a lack of empirical piloting or risk adjustment for chosen quality indicators. We also recommend that investigators consider developing reporting guidelines for consensus methods used to develop quality indicators. Advances in the standardization of electronic medical record management across North America may help to drive empirical piloting and validation of quality indicators. Moreover, future studies may benefit from evaluating and developing quality indicators for marginalized communities and patients in OHNS, such as head and neck cancer patients, who require multidisciplinary care.^61,62

There are several limitations of this study. First, we did not empirically evaluate or validate these quality indicators. Second, we employed broad library search parameters using terms relevant to OHNS, but these terms may not have captured quality indicators related to allied health or patient engagement in OHNS.

Quality indicators are being developed and tested to improve healthcare quality and delivery in OHNS. Consensus methods for identifying quality indicators are gaining popularity, but investigators must improve study reporting, patient and relevant stakeholder engagement, and perform empirical validation of novel quality indicators. We should also consider focusing on structure and process indicators since they are less studied but may better inform quality improvement initiatives when compared to outcome indicators. As we proceed toward an era of eHealth and personalized medicine, a better understanding of quality indicators in OHNS may help to improve healthcare standardization, institutional accountability, and professional collaboration around the world.

Supplemental Material

sj-docx-1-ohn-10.1177_19160216251330627 – Supplemental material for Quality Indicators in Otolaryngology–Head and Neck Surgery: A Scoping Review

Supplemental material, sj-docx-1-ohn-10.1177_19160216251330627 for Quality Indicators in Otolaryngology–Head and Neck Surgery: A Scoping Review by Phillip Staibano, Shireen Samargandy, Justin Cottrell, Lily Wang, Michael Au, Michael K. Gupta, Han Zhang, Doron D. Sommer, Christopher Walsh and Eric Monteiro in Journal of Otolaryngology - Head & Neck Surgery

Supplemental Material

sj-docx-2-ohn-10.1177_19160216251330627 – Supplemental material for Quality Indicators in Otolaryngology–Head and Neck Surgery: A Scoping Review

Supplemental material, sj-docx-2-ohn-10.1177_19160216251330627 for Quality Indicators in Otolaryngology–Head and Neck Surgery: A Scoping Review by Phillip Staibano, Shireen Samargandy, Justin Cottrell, Lily Wang, Michael Au, Michael K. Gupta, Han Zhang, Doron D. Sommer, Christopher Walsh and Eric Monteiro in Journal of Otolaryngology - Head & Neck Surgery

Footnotes

Appendix A: Search Strategy

Database: Ovid MEDLINE(R) and Epub Ahead of Print, In-Process & Other Non-Indexed Citations, Daily and Versions(R) <1946 to January 20, 2020> Search Strategy:

Data Availability

Available from authors upon request.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical Disclosures

Not applicable.

ORCID iDs

Phillip Staibano

Shireen Samargandy

Supplemental Material

Additional supporting information is available in the online version of the article.

References

Donabedian

The quality of care. How can it be assessed?

JAMA. 1988;260(12):1743-1748. doi:10.1001/jama.260.12.1743

Heenan

Khan

Binkley

From boardroom to bedside: how to define and measure hospital quality. Healthc Q. 2010;13(1):55-60. doi:10.12927/hcq.2013.21615

Mainz

Defining and classifying clinical indicators for quality improvement. Int J Qual Health Care. 2003;15(6):523-530. doi:10.1093/intqhc/mzg081

Hall

Hamilton

Richards

Bilimoria

Cohen

CY.

Does surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: an evaluation of all participating hospitals. Ann Surg. 2009;250(3):363-376. doi:10.1097/SLA.0b013e3181b4148f

Castaldi

George

Turner

McNelis

NSQIP impacts patient experience. J Patient Exp. 2020;7(1):89-95. doi:10.1177/2374373518817081

Thanh

Baron

Litvinchuk

An Economic Evaluation of the National Surgical Quality Improvement Program (NSQIP) in Alberta, Canada. Ann Surg. 2019;269(5):866-872. doi:10.1097/SLA.0000000000002708

Vuk

Quality indicators: a tool for quality monitoring and improvement. ISBT Sci Ser. 2012;7(1):24-28. doi:10.1111/j.1751-2824.2012.01584.x

Boulkedid

Abdoul

Loustau

Sibony

Alberti

Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One. 2011;6(6):e20476. doi:10.1371/journal.pone.0020476

Stelfox

Straus

SE.

Measuring quality of care: considering conceptual approaches to quality indicator development and evaluation. J Clin Epidemiol. 2013;66(12):1328-1337. doi:10.1016/j.jclinepi.2013.05.017

10.

Tricco

Lillie

Zarin

, et al PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467-473. doi:10.7326/M18-0850

11.

McGowan

Straus

Moher

, et al Reporting scoping reviews-PRISMA ScR extension. J Clin Epidemiol. 2020;123:177-179. doi:10.1016/j.jclinepi.2020.03.016

12.

Wagner

Schaffert

Mockli

Zuniga

Dratva

Home care quality indicators based on the Resident Assessment Instrument-Home Care (RAI-HC): a systematic review. BMC Health Serv Res. 2020;20(1):366. doi:10.1186/s12913-020-05238-x

13.

Joling

van Eenoo

Vetrano

, et al Quality indicators for community care for older people: a systematic review. PLoS One. 2018;13(1):e0190298. doi:10.1371/journal.pone.0190298

14.

Arce

Martin Cleary

Cenjor

Ramos

Ortiz

Designing a method with physician participation to assess and improve quality of healthcare in otolaryngology. J Int Adv Otol. 2017;13(1):128-135. doi:10.5152/iao.2017.2880

15.

Balakrishnan

Bauman

Chun

, et al Standardized outcome and reporting measures in pediatric head and neck lymphatic malformations. Otolaryngol Head Neck Surg. 2015;152(5):948-953. doi:10.1177/0194599815577602

16.

Balakrishnan

Sidell

Bauman

, et al Outcome measures for pediatric laryngotracheal reconstruction: international consensus statement. Laryngoscope. 2019;129(1):244-255. doi:10.1002/lary.27445

17.

Cottrell

Yip

Chan

, et al Quality indicators for the diagnosis and management of chronic rhinosinusitis. Int Forum Allergy Rhinol. 2018;8(12):1369-1379. doi:10.1002/alr.22161

18.

Cottrell

Yip

Campisi

, et al Quality indicators for the diagnosis and management of pediatric tonsillitis. Int J Pediatr Otorhinolaryngol. 2020;139:110441. doi:10.1016/j.ijporl.2020.110441

19.

Cottrell

Yip

Chan

, et al Quality indicators for the diagnosis and management of acute bacterial rhinosinusitis. Am J Rhinol Allergy. 2020;34(4):519-531. doi:10.1177/1945892420912158

20.

Cottrell

Yip

Agrawal

, et al Quality indicators for the diagnosis and management of Meniere’s disease. Otol Neurotol. 2021;42(8):e1084-e1092. doi:10.1097/MAO.0000000000003206

21.

Cottrell

Namavarian

Yip

, et al Proposed quality indicators for aspects of pediatric acute otitis media management. J Otolaryngol Head Neck Surg. 2024;53:19160216241248538. doi:10.1177/19160216241248538

22.

Hall

Smith

Hibbert

, et al The COMiT’ID Study: developing core outcome domains sets for clinical trials of sound-, psychology-, and pharmacology-based interventions for chronic subjective tinnitus in adults. Trends Hear. 2018;22:2331216518814384. doi:10.1177/2331216518814384

23.

Harman

Bruce

Kirkham

, et al The importance of integration of stakeholder views in core outcome set development: otitis media with effusion in children with cleft palate. PLoS One. 2015;10(6):e0129514. doi:10.1371/journal.pone.0129514

24.

Hernandez

JJC

Arrula

Alvarez

, et al Indicators to evaluate quality of care in head and neck cancer in Spain. Clin Transl Oncol. 2024;26(5):1089-1097. doi:10.1007/s12094-023-03298-z

25.

Hibbert

Stephens

de Wet

, et al Assessing the quality of the management of tonsillitis among Australian children: a population-based sample survey. Otolaryngol Head Neck Surg. 2019;160(1):137-144. doi:10.1177/0194599818796137

26.

Hopkins

CHronic Rhinosinusitis Outcome MEasures (CHROME), developing a core outcome set for trials of interventions in chronic rhinosinusitis. Rhinology. 2017;56(1):22-32. doi:10.4193/Rhino17.247

27.

Joaquim

Nobre

Araujo

, et al A collaborative approach for the development of a standardized set of patient-centered outcomes in head and neck cancers. Acta Med Port. 2023;36(7-8):475-486. doi:10.20344/amp.18180

28.

Manahan

Fedok

Davidson

, et al Evidence-based performance measures for rhinoplasty: a multidisciplinary performance measure set. Plast Reconstr Surg. 2021;147(2):222e-230e. doi:10.1097/PRS.0000000000007598

29.

Manduchi

Fitch

Ringash

Howell

Martino

A core outcome set for patient-reported dysphagia for use in head and neck cancer clinical trials: an international multistakeholder Delphi study. Head Neck. 2024;46(4):831-848. doi:10.1002/hed.27626

30.

Michel

Schwartz

Dawson

, et al Quality improvement in otolaryngology-head and neck surgery: developing registry-enabled quality measures from guidelines for cerumen impaction and allergic rhinitis through a transparent and systematic process. Otolaryngol Head Neck Surg. 2022;166(1):13-22. doi:10.1177/01945998211011987

31.

Moraes

Andrade

CR.

Quality indicators for integrated care of dysphagia in hospital settings. J Soc Bras Fonoaudiol. 2011;23(1):89-94. doi:10.1590/s2179-64912011000100018

32.

Noltes

Cottrell

Madani

, et al Quality indicators for the diagnosis and management of primary hyperparathyroidism. JAMA Otolaryngol Head Neck Surg. 2022;148(3):209-219. doi:10.1001/jamaoto.2021.3858

33.

Ouwens

Marres

Hermens

, et al Quality of integrated care for patients with head and neck cancer: development and measurement of clinical indicators. Head Neck. 2007;29(4):378-386. doi:10.1002/hed.20532

34.

Sibthorpe

Agostino

Coates

, et al Indicators for continuous quality improvement for otitis media in primary health care for Aboriginal and Torres Strait Islander children. Aust J Prim Health. 2017;23(1):1-9. doi:10.1071/PY16096

35.

Ten Tije

Pauw

Braspenning

, et al Uniform registration agreements on cholesteatoma care: a nationwide consensus procedure. Otol Neurotol. 2020;41(8):1094-1101. doi:10.1097/MAO.0000000000002770

36.

van Overveld

Braspenning

Hermens

RP.

Quality indicators of integrated care for patients with head and neck cancer. Clin Otolaryngol. 2017;42(2):322-329. doi:10.1111/coa.12724

37.

Vila

Lieu

Hullar

Buchman

CA.

Developing quality measures for adult cochlear implant centers: preliminary findings. Otolaryngol Head Neck Surg. 2016;155(5):748-752. doi:10.1177/0194599816665583

38.

Cottrell

Zahr

Yip

, et al Quality indicators for the diagnosis and management of sudden sensorineural hearing loss. Otol Neurotol. 2021;42(8):e991-e1000. doi:10.1097/MAO.0000000000003205

39.

Wong

SL.

The state of quality indicators in surgical oncology. J Surg Oncol. 2009;99(1):7-8. doi:10.1002/jso.21172

40.

Schneider

Lavin

JM.

Publicly available databases in otolaryngology quality improvement. Otolaryngol Clin North Am. 2019;52(1):185-194. doi:10.1016/j.otc.2018.08.004

41.

O’Byrne

Copperthwaite

Rente

Fenton

Coelho

DH.

The quality of otology and neurotology research in otology journals. Otol Neurotol. 2022;43(2):153-158. doi:10.1097/MAO.0000000000003425

42.

Socolovsky

Krishnan

Ramraj

, et al Quality of otology and neurotology research in otolaryngology journals. Otol Neurotol. 2020;41(8):1021-1026. doi:10.1097/MAO.0000000000002731

43.

Silver

Yeung

Almutawa

Szwimer

Nguyen

LHP

. Evaluating strength of evidence of pediatric otolaryngology research literature: a 20-year review. Laryngoscope. 2022;132(9):1869-1876. doi:10.1002/lary.29945

44.

Carlton

Kocherginsky

Langerman

AJ.

A systematic review of the quality of randomized controlled trials in head and neck oncology surgery. Laryngoscope. 2015;125(1):146-152. doi:10.1002/lary.24718

45.

Chera

Eisbruch

Murphy

, et al Recommended patient-reported core set of symptoms to measure in head and neck cancer treatment trials. J Natl Cancer Inst. 2014;106(7):153-158. doi:10.1093/jnci/dju127

46.

Moore

Lavoie

Bourgeois

Lapointe

Donabedian’s structure-process-outcome quality of care model: validation in an integrated trauma system. J Trauma Acute Care Surg. 2015;78(6):1168-1175. doi:10.1097/TA.0000000000000663

47.

Hamilton

MacMillan

Benson

Memtsoudis

McCartney

CJL

. Regional anaesthesia quality indicators for adult patients undergoing non-cardiac surgery: a systematic review. Anaesthesia. 2021;76(Suppl 1):89-99. doi:10.1111/anae.15311

48.

Tossaint-Schoenmakers

Versluis

Chavannes

Talboom-Kamp

Kasteleyn

The challenge of integrating eHealth into health care: systematic literature review of the Donabedian model of structure, process, and outcome. J Med Internet Res.2021;23(5):e27180. doi:10.2196/27180

49.

Guetterman

Buis

LR.

Examining telehealth through the Institute of Medicine quality domains: unanswered questions and research agenda. BMJ Qual Saf. 2024;33(9):552-555. doi:10.1136/bmjqs-2023-016872

50.

Ploeg

Flu

Lardenoye

Hamming

Breslau

PJ.

Assessing the quality of surgical care in vascular surgery; moving from outcome towards structural and process measures. Eur J Vasc Endovasc Surg. 2010;40(6):696-707. doi:10.1016/j.ejvs.2010.05.010

51.

McLeod

Leung

Pramesh

, et al Quality indicators in surgical oncology: systematic review of measures used to compare quality across hospitals. BJS Open. 2024;8(2):zrae009. doi:10.1093/bjsopen/zrae009

52.

Liu

Guo

Wang

Liu

Process indicators outshine outcome measures: assessing hospital quality of care in breast cancer treatment in China. Sci Rep. 2024;14(1):19137. doi:10.1038/s41598-024-70474-8

53.

Rademakers

Delnoij

de Boer

Structure, process or outcome: which contributes most to patients’ overall assessment of healthcare quality?

BMJ Qual Saf. 2011;20(4):326-331. doi:10.1136/bmjqs.2010.042358

54.

Salampessy

Portrait

FRM

van der Hijden

Klink

Koolman

On the correlation between outcome indicators and the structure and process indicators used to proxy them in public health care reporting. Eur J Health Econ. 2021;22(8):1239-1251. doi:10.1007/s10198-021-01333-w

55.

Nair

Aggarwal

Khanna

Methods of formal consensus in classification/diagnostic criteria and guideline development. Semin Arthritis Rheum. 2011;41(2):95-105. doi:10.1016/j.semarthrit.2010.12.001

56.

Jandhyala

Delphi, non-RAND modified Delphi, RAND/UCLA appropriateness method and a novel group awareness and consensus methodology for consensus measurement: a systematic literature review. Curr Med Res Opin. 2020;36(11):1873-1887. doi:10.1080/03007995.2020.1816946

57.

Spranger

Homberg

Sonnberger

Niederberger

Reporting guidelines for Delphi techniques in health sciences: a methodological review. Z Evid Fortbild Qual Gesundhwes. 2022;172:1-11. doi:10.1016/j.zefq.2022.04.025

58.

Kotter

Schaefer

Scherer

Blozik

Involving patients in quality indicator development—a systematic review. Patient Prefer Adherence. 2013;7:259-268. doi:10.2147/PPA.S39803

59.

Benin

Vitkauskas

Thornquist

, et al Validity of using an electronic medical record for assessing quality of care in an outpatient setting. Med Care. 2005;43(7):691-698. doi:10.1097/01.mlr.0000167185.26058.8e

60.

Hofer

Bernstein

Hayward

DeMonner

Validating quality indicators for hospital care. Jt Comm J Qual Improv. 1997;23(9):455-467. doi:10.1016/s1070-3241(16)30332-7

61.

Manalili

Siad

Antonio

Lashewicz

Santana

MJ.

Codesigning person-centred quality indicators with diverse communities: a qualitative patient engagement study. Health Expect. 2022;25(5):2188-2202. doi:10.1111/hex.13388

62.

Mendez

Seikaly

Eurich

, et al Development of a patient-centered functional outcomes questionnaire in head and neck cancer. JAMA Otolaryngol Head Neck Surg. 2020;146(5):437-443. doi:10.1001/jamaoto.2019.4788

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB

0.02 MB