Sage Journals: Discover world-class research

Abstract

French

Objectives: To determine the feasibility of implementing Ovarian-Adnexal Reporting & Data System (O-RADS) ultrasound (US) for reporting of adnexal masses at our institution, with a specific goal of increasing the use of O-RADS from a baseline of <5% to at least 75% over a 16-month period. Methods: A prospective interrupted time series quality improvement study was undertaken over a 16-month period. Plan, do, study, act cycles included: (1) Engagement of interested parties, (2) Targeted educational sessions, (3) Development of reporting templates, (4) Weekly audit-feedback. Inter-reader variability assessment was performed on 70% of O-RADS risk-category 2 to 5. The primary outcome was the reporting of an O-RADS risk category. Results: A total of 635 female pelvic US were performed at our centre between July 2022 and April 2023. An O-RADS risk category was provided on the final radiology report by the radiologist for 489/635 (77%) US. From November 2022 to April 2023, the weekly rate of O-RADS risk category reporting reached 88%. The O-RADS score was concordant between readers for 83/103 (81%) of US reports with kappa score of 0.69 corresponding to good agreement. Conclusions: The reporting of O-RADS risk category increased from <5% to 88% over a 16-month period with a high level of agreement among readers in assigning O-RADS risk category. Implementation of a standardizing reporting ultrasound system at a tertiary cancer centre is feasible with rapid learning and uptake curves.

Keywords

quality improvement O-RADS ovarian neoplasm implementation strategies

Introduction

Adnexal masses are an extremely common finding and while the vast majority are benign, a small number represent an aggressive ovarian cancer.^1
-3 These cancers need to be recognized early with timely referral to a gynaecologic oncology centre (GOC) for management in order to improve oncologic outcomes.^4,5 There is growing evidence on the effect of surgeon specialty on the outcome of gynaecologic malignancies. An Ontario population-based study showed a 2-fold increase in all-cause mortality if ovarian cancer surgery was performed by non-gynaecologic oncologist.^5,6 It is essential to triage patients with a suspicious adnexal mass to the appropriate surgical team while offering an option for conservative management in benign cases.

Ultrasound (US) of ovarian/adnexal lesions is performed to establish an accurate risk of malignancy assessment that is used to direct appropriate management.^3,7 Nonetheless, there is recognized heterogeneity in US reporting which contributes to ambiguity in risk-assessment and management.² Adoption of a standardized reporting system for adnexal masses leads to improvement in patient outcomes and optimization of healthcare resources.⁷ The current lack of standardization in US reporting can lead to inaccurate risk assessment resulting in unnecessary surgical procedures, “over-imaging” and poorer survival outcomes when individuals with malignant masses are not urgently referred to GOCs.⁸ The American College of Radiology developed the Ovarian-Adnexal Reporting & Data System (O-RADS) with the support of an international group of multidisciplinary specialists and multi-society representatives to address the lack of standardized terminology, risk assessment, and management pathways for women with ovarian lesions.⁹ The O-RADS US system combines a descriptor-based lexicon with evidence-based assessment data to assign an O-RADS category risk of malignancy (ROM) of 1 to 5 for each lesion.⁹ The risk of malignancy category values are as follows: 0% for O-RADS 1, <1% for O-RADS 2, 1 to <10% for O-RADS 3, 10 to <50% for O-RADS 4, and >50% for O-RADS 5. The inclusive O-RADS lexicon was published in 2018 and the O-RADS US Risk Stratification and Management recommendations^9,10 strategy was published in 2020.

Cancer Care Ontario/Ontario Health (CCO/OH), the agency that oversees delivery of cancer care and supervises funding for cancer services in Ontario, Canada, identified the need to standardize US reporting for ovarian/adnexal masses in Ontario as a high priority quality issue to improve management and outcomes of ovarian cancer patients. After a thorough literature review, CCO/OH adopted the American College of Radiology (ACR) O-RADS reporting system⁹ as the optimal reporting tool for adnexal masses, with minor adaptions for the Ontario health environment. These recommendations are expected to the published in the near future.

Despite provincial and international endorsement of O-RADS US, prospective data internationally¹¹ and retrospective data provincially¹² showing good diagnostic performance of the system, there was limited (<5%) uptake at our institution and in the province of Ontario, Canada. The primary objective of this study was to determine the feasibility of implementing O-RADS at our institution for reporting of adnexal masses in individuals and increase its use from <5% to at least 75% over a 16-month period.

Methods

Study Design and Setting

A prospective interrupted time series quality improvement study was completed from January 2022 to April 2023 at Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada. Sunnybrook Health Sciences Center is a 1300-bed tertiary care hospital in Toronto, Ontario, Canada. It includes a high-volume of general gynaecology, family medicine, and gynaecologic oncology. Every week, a high number of pelvic ultrasounds for characterization of adnexal masses are performed. A subset of patients provided consent for the study team to access longitudinal data on follow-up imaging (440 of 489 patients). Ethics approval was obtained from the Sunnybrook Health Sciences Centre Research Ethics Boards (REB #5276). Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0) was used for the formatting of this prospective interrupted time series quality improvement study.¹³

From January to June 2022, the use of O-RADS was <5%, based on an initial audit. Previously, radiologists used subjective assessment to determine whether a lesion was low or high-risk with varying addition of management recommendations depending on the reporting radiologist. During the audit period, there were 14 radiologists with expertise in reporting gynaecologic pelvic ultrasounds. Six radiologists were part of a dedicated gynaecologic imaging subspecialty group and participated in gynaecologic oncology multidisciplinary rounds. Two radiologists had significant experience in reporting adnexal masses with the O-RADS system and had received dedicated training with the O-RADS system (KH, PG). Two radiologists (KH, PG) performed double reads on a random subset of cases for quality control. There were 14 residents and 8 fellows that rotated yearly through the ultrasound department. A fishbone diagram was developed to understand the barriers and facilitators to implementation of a standardized ultrasound reporting system (Figure 1).

Figure 1.

Fishbone diagram addressing barriers analysis.

Interventions

Plan, do, study, act (PDSA) cycles were used (Table 1). The PDSAs included: (1) Engagement of interested parties, (2) Targeted educational sessions, (3) Development of reporting templates for sonographers and radiologists, and (4) Weekly audit-feedback. An email was sent to all end-users to inform them of the initiative and expectation of a change in the format of ultrasound reports for adnexal masses as of June 22nd 2022 by using the O-RADS system.

Table 1.

Plan-Do-Study-Act (PDSA).

PDSA	Interventions	Description
PDSA 1	Engagement of interested parties (January-June 2022)	Recruitment of a multidisciplinary team to identify barriers and facilitators to implementation of O-RADS. There was engagement of sonographers, radiologists, end-users such as family physicians, general gynecologists, gynaecologic oncologists.
PDSA 2	Educational sessions (January 2022-April 2023)	Educational sessions on the use of O-RADS to radiologists, sonographers, trainees, end-users (family physicians, gynecologists, gynaecologic oncologists). Topics covered included: why use O-RADS, the use of the worksheet, case-based learning, how to provide a risk assessment based on findings, booster session for fellows and how to interpret the O-RADS risk category and recommendations.
PDSA 3	Development of Reporting Templates for Sonographers and Radiologists (April-October 2022)	O-RADS reporting template was created and integrated into the hospital imaging software
PDSA 4	Audit-Feedback (November 2022-January 2023)	Weekly audit-feedback to radiologists on their use of O-RADS risk category

Engagement of Interested Parties

A mapping process of interested parties to identify the key participants was completed: radiologists, radiology trainees, gynaecologic oncologists, general gynecologists, family doctors, and sonographers. Project leads were identified for each group. Between January 2022 and March 2022, 2 meetings were held for each group including (1) a general education session outlining the clinical importance of O-RADS system and introduction to the imaging features and risk categories for all members and (2) kick-off meetings with project leads to introduce the project aims, set deadlines for launch, and identify potential barriers and facilitators to implementation (Figure 1). For both sonographers and radiologists, the major barrier identified was the increase in the time required to perform and report ultrasound while the most important facilitator identified was the access to a champion radiologist for support. For family practice, the major barriers were concerns regarding timely access to a gynecologist or gynaecologic oncologist and having access to the O-RADS summary chart readily available. The latter was embedded within the radiology report as a hyperlink to address this concern. For gynaecologic oncologists, the major barriers identified were: 1) concerns regarding the increased referral to resource limited MRI and 2) increased referral of non-malignant ovarian masses to resource limited gynaecologic oncologists. An SBAR (Situation-background-assessment-recommendation) document was created with the project leads and circulated widely to all interested parties in June 2022 to inform everyone about the new method of reporting adnexal masses (Supplemental Material 1).

Targeted Educational Sessions

Between January 2022 and April 2023, educational rounds were given to all interested parties outlining the rationale for O-RADS ultrasound and providing a case-based introduction to the system. Specific training sessions were provided to the radiologists, sonographers, and radiology trainees in April and May 2022. These included an introduction to a templated O-RADS sonographer worksheet and radiology reporting template (Nuance PowerScribe; Supplemental Material 2 and 3). Follow up education sessions were provided to radiology imaging trainees in August 2022 and sonographers in September 2022. During these sessions, the O-RADS reporting system was reviewed using interactive case-based examples and specific questions were addressed.

In addition to education sessions, reference materials were created based on sonographer and radiologist feedback and were made available in the US reporting room.

Development of Reporting Templates for Sonographers and Radiologists

A template sonographer worksheet including the O-RADS descriptors and risk categories was created and underwent iterative rapid PDSA cycles with feedback from the radiologists and sonographers between April 2022 and October 2022. A final sonographer worksheet was agreed upon and implemented on October 5, 2022. In February 2023, it was identified that the sonographer worksheets for (1) gynaecologic pelvic ultrasounds done specifically for first trimester early pregnancy evaluation, (2) sonohysterogram, and (3) abdominal scan with pelvic add-on did not include a section for O-RADS descriptors or risk category. New sonographer worksheets were designed for these specific indications to include O-RADS ultrasound descriptors and risk categories in February 2023 and circulated for use (Supplemental Material 2).

A standardized radiology reporting template for gynaecologic pelvic ultrasound was created in Nuance PowerScribe 360, which is the radiology reporting system integrated with the hospital Picture Archiving and Computer System (PACS) and electronic patient record. The template was reviewed by project leads in May 2022 and a revised template with feedback incorporated was presented to radiologists in June 2022. The reporting template was launched June 22, 2022 to auto-populate for all gynaecologic pelvic ultrasound (Supplemental Material 3). Similar to the sonographer worksheet, iterative PDSA cycles were performed between June and September 2022 based on radiologist and radiology trainee feedback.

Weekly Audit Feedback

The use of O-RADS US reporting system officially started June 22, 2022. Weekly audit was performed to determine the percentage uptake of the sonographer worksheet and the percentage of gynaecologic pelvic ultrasound final radiology reports that included an O-RADS risk category. Beginning November 27, 2022, weekly emails were sent to all the radiologists and radiology trainees summarizing O-RADS uptake and risk category assignment. The radiologists were encouraged to engage with project leads for any questions or concerns. The study team also made individual contact with imagers who had poor usage of the O-RADS US system to identify barriers and provide additional education and support.

Study of the Interventions

Interventions were assessed using iterative PDSA cycles. Reporting rates were retrospectively measured at the end of each week. All adnexal ultrasound performed during the time period (June 2022-April 2023) were included. Inter-reader reliability was performed by a radiologist with expertise in ultrasound reporting and O-RADS reporting system (KH). Reports with O-RADS risk category 1 (normal) were excluded. The reviewer reviewed 70% (103/142 cases) of ultrasounds with O-RADS risk category 2 to 5. The cases reviewed were randomly selected between June 2022 and April 2023. The reviewer was blinded to the O-RADS risk category reported by the original reader. The team met every 2 weeks to audit O-RADS report use and examine reasons for non-O-RADS report, monitor adherence and plan adjustments to increase the compliance with the ORADS system.

Measures and Statistical Analysis

Statistical analysis was performed throughout the entire study period given iterative small cycle and changes occurring throughout the QI project timeline. Data was collected over a 16-month period from January 2022 to April 2023. The primary outcome was the rate of assignment of a risk category by the radiologists using the O-RADS risk category. Secondary outcomes include the use of the O-RADS template by sonographers, the rate of MRI after O-RADS US, the rate of inter-reader variability using a radiologist with O-RADS expertise as reference.

A statistical process chart (p-chart) was developed to determine if the interventions had a significant impact on the reporting of O-RADS risk category. A p-chart was also developed to monitor the use of the O-RADS worksheet by the sonographers. Kappa value was calculated to assess inter-rater agreement.

Results

A total of 635 female pelvic ultrasounds were performed at our centre between June 22, 2022, and April 30, 2023. An O-RADS risk category was provided on the final radiology report by the radiologist for 489/635 (77%) ultrasounds during the study period (June 2022 to April 2023). From November 2022 to April 2023, the average O-RADS risk category reporting was 88.5% (Figure 2). Standardized reporting templates for sonographers and radiologists were developed using iterative feedback cycles and led to a rapid increase in O-RADS risk category reporting with 5 data points over 50% (Figures 2 and 3). An O-RADS risk category score of 1 (normal ovary) was assigned for 347 cases with the remaining 142 cases receiving an O-RADS score of 2 to 5. Inter-reader reliability was assessed for the ORADS 2 to 5 group in 103/142 (72%) of ultrasound reports. The O-RADS score was concordant between readers for 83/103 (81%) of ultrasound reports. The kappa value was 0.69 representing a high level of agreement.

Figure 2.

P-chart for the reporting of O-RADS risk category by radiologists per week. Annotation indicates interventions implemented.

Figure 3.

P-chart for the use of O-RADS reporting template by sonographers per week. Annotations indicate interventions implemented.

O-RADS score was downgraded by the expert reader for 19/20 cases. The reasons for downgrading included: no lesion seen by second reader (3), definition not applied correctly by first reader (8), subjective difference in colour score between 2 readers (2), subjective difference in multilocular versus 2 adjacent cysts (3), subjective difference in classic versus non-classic benign for an endometrioma (1), no solid component seen by second reader (1), and no peritoneal nodule seen by second reader (1). Three O-RADS 5 lesions were downgraded to O-RADS 4 by the expert reader due to differences in colour score. No O-RADS 5 lesion was downgraded to below an O-RADS 4 score. No lesion was upgraded to an O-RADS 5 risk category. In only 35 (8%) of those followed longitudinally (N = 440) was a pelvic MRI recommended for further characterization in the radiology report. One O-RADS score was upgraded from an O-RADS 2 to 4 lesion by the second reader as it did not strictly meet criteria for O-RADS 2 endometrioma. This likely represented an endometrioma with echogenic clot and does not necessarily represent a malignant lesion but was upgraded to O-RADS 4 as the appearance was not classic for a typical endometrioma.

Discussion

During a 16-month period, this quality improvement initiative resulted in an increase in the use of O-RADS risk category in female pelvic ultrasound reports from <5% to 88% with repeated engagement of interested parties and education, development of standardized reporting templates and ongoing monitoring of interventions. Radiologists and sonographers at our centre had varying degrees of knowledge of the O-RADS US system from prior educational rounds and conferences. Nonetheless, prior to the implementation of O-RADS US system, almost no pelvic ultrasound reports included an O-RADS risk category (<5%). A fishbone diagram was constructed at the outset of the project to analyze barriers to O-RADS usage and addressed radiologist factors, sonographer factors, end-user factors, and system factors. The major barriers initially identified were lack of perceived clinical benefit by radiologists, lack of time and training for sonographers, lack of timely access to specialists, lack of information on how to interpret reports for end-users, and lack of time at the system level.

The interventions implemented with each PDSA cycle were designed to target these major barriers. Engagement of interested parties was the first intervention and focused on highlighting the clinical utility of the O-RADS US system. This was achieved by developing an educational session presented by a radiologist and gynaecologic oncologist and focusing on: (1) The rational and need for standardization of US reporting for adnexal masses in Ontario, (2) Understanding the oncologic benefits for ovarian cancer patients who are managed at a cancer centre with a gynaecologic oncologist, (3) Introducing the standardized lexicon and the risk stratification/clinical management tool used to describe adnexal masses in O-RADS and highlighting its clinical utility. Once radiologists, sonographers, and end-users understood that improving the reporting of adnexal lesions lead to direct benefit to patient care, it was easier to introduce the system and gain support. This was particularly true for the local group of radiologists who were already considered expert reporters in gynaecology ultrasound and initially were hesitant to adopt a new system being unsure of the added value.

Despite initial support for O-RADS implementation, uptake remained slow over the first 3 months (June-August 2022) likely due to a combination of new trainees and relative short-staffing in the summer months which may have translated into less time to learn a new system. The second PDSA cycle focused on repeated education sessions and the introduction of standardized templates. While education sessions are an important part of introducing a new system, it is not surprising that education alone did not lead to a significant increase in use of O-RADS. Other studies have shown didactic teaching alone leads to little change in clinical practice.^14
-16

The final PDSA cycle comprised audit and feedback of O-RADS risk category reporting to the radiologists and trainees directly by means of weekly group emails as well as individual emails and conversations with outliers. This stage was critical in achieving the culture shift that was required to achieve consistent use of the O-RADS risk-categories. Late adopters were approached and asked if there were modifiable factors that would increase their use of the O-RADS reporting system and encouraged to reach out to expert users for assistance if clarification was needed for specific cases. In all cases, no further education was required, but the audit information and direct communication provided the nudge necessary to achieve a sustainable shift in reporting practices.

One of the concerns raised by end-users at the start of the project was that reporting O-RADS risk categories would lead to an increase in the number of MRI studies recommended for patients who underwent a pelvic ultrasound. In our current Canadian healthcare system, access to MRI studies can be challenging and associated with delay of care. In our study, 8% of reports recommended additional MRI study. This is fewer than a recent population-based study in Ontario which reported that 12% of patients with gynaecologic pelvic ultrasound underwent additional imaging with MRI.¹⁷ A population-based study in Ontario including over 193 000 women aged 45 and older who had a pelvic ultrasound demonstrated that performing a pelvic MRI after pelvic ultrasound did not increase rate of referral of women with a high-risk adnexal mass to a gynaecologic oncologist.^17,18 The addition of MRI often further delayed time to surgery which has been supported in other studies.^17,18 Although the data is limited to make broad conclusions, we did not observe an increase in MRI recommendations and it is possible that more clarity, consistency, and confidence in ultrasound reports that include a risk category assessment such as O-RADS may in fact lead to a decrease in MRI recommendations.

As ORADS is a relatively new reporting and data system in radiology, another concern was whether the O-RADS risk category assigned by the radiologist was accurate. Following blinded evaluation by an expert reader, inter-rater reliability for ORADS 2 to 5 cases was 81% with a kappa value of 0.69 which is considered a high level of agreement. This is lower than what has been reported in retrospective reviews of the ORADS system (Cao, Lai) which reported kappa values of 0.83 and 0.71. The prospective inter-reader reliability for the O-RADS ultrasound system has not yet been reported and the relatively lower inter-rater agreement likely reflects the learning curve at our institution. A study by Zhou et al examining the learning curve for O-RADS ultrasound in junior doctors found that up to 40% of trainees had an incorrect score in 23/100 adnexal masses provided for assessment.¹⁹ Our relatively high agreement with kappa of 0.69 during the initial implementation of ORADS in our department may suggest an accelerated learning curve given the considerable experience of most of the staff radiologists with gynaecologic ultrasound imaging.

The main reasons for discrepant risk category in our study were incorrect colour score, incorrect classification of an O-RADS 1 finding of corpus luteum as O-RADS 2 category, incorrect classification of multilocular versus unilocular cyst, and incorrect classification of a classic benign lesion. A recent update to the O-RADS ultrasound reporting system, O-RADS US v2022, was recently published which changes the designation of multilocular cysts from lesions with one septation to lesions with at least 2 complete septations. The revised O-RADS v2022 would likely decrease discrepancies in classifying unilocular versus multilocular lesions with improved concordance.²⁰ Our findings were similar to that of Zhou which reported the main challenging points were colour score, incorrectly classified classic benign lesions, differentiating unilocular versus multilocular cysts, and differentiating solid-appearing versus solid lesions.¹⁹ The subjective nature of some of the O-RADS descriptors such as colour score and recognition of classic benign lesions likely requires additional training. A fifth PDSA cycle could be added specifically to address these more challenging aspects of O-RADS ultrasound system to increase the inter-rater agreement.

Strength-Limitations

This quality improvement study highlights a stepwise prospective approach to implementing a new ultrasound reporting system. Our study has several limitations that warrant discussion. The uptake and learning curve of O-RADS US reporting system was rapid in part because this was implemented at a tertiary-care centre where both sonographers and radiologists had a high level of expertise in performing and interpreting ultrasound including trained champions. A significant limitation was that it was not possible to accurately capture whether use of the O-RADS US reporting system led to an increase in the time required for radiologists to complete an ultrasound report. Radiologists are often interrupted when dictating a report and therefore the time captured by voice dictation software is not representative of the actual time reported to complete the report. The time required for sonographers to complete scans was also not measured. However, no additional time was requested by staff for pelvic ultrasound scans during the implementation phase and there was no reported major disruption to usual patient flow.

Future Directions

The ACR O-RADS US lexicon and system was recently endorsed by Ontario Health/Cancer Care Ontario as the recommended reporting system. New knowledge obtained from this initiative will support the creation of a toolkit for provincial roll-out of O-RADS US. The toolkit and education materials will be disseminated locally and provincially via webinars and continuing medical education courses. A prospective study on the incorporation of O-RADS is needed to evaluate the accuracy of the risk of malignancy, the impact on patient outcomes and healthcare resources used. In addition, further evaluations into the barriers to achieve 100% compliance for O-RADS implementation. One strategy could be to send individualized radiologist feedback and request for addendums when O-RADS score is not included in the report.

Conclusions

A quality improvement initiative improved reporting of an O-RADS risk category from <10% to 88% over a 16 month-period. The implementation of O-RADS ultrasound reporting system is feasible with the use of educational initiatives, stakeholder engagements and use of a standardized ultrasound worksheet. The inter-reader agreement was high during the implementation phase. One can expect a fast-learning curve among a group of experienced radiologists. Additional studies are needed to determine the impact on patients’ outcomes and optimization of health care resources.

Supplemental Material

sj-docx-1-caj-10.1177_08465371241301335 – Supplemental material for Implementation of O-RADS Ultrasound Reporting System: A Quality Improvement Initiative

Supplemental material, sj-docx-1-caj-10.1177_08465371241301335 for Implementation of O-RADS Ultrasound Reporting System: A Quality Improvement Initiative by Genevieve Bouchard-Fortier, Phyllis Glanc, Sarah E. Ferguson, Debbie Elman, Rachel Kupets, Leslie Po, Sarah Taleghani, Lisha Lo and Kalesha Hack in Canadian Association of Radiologists Journal

Footnotes

Author Contributions

Genevieve Bouchard-Fortier: Conceptualization; Data curation; Formal analysis; Funding acquisition; Methodology; Project administration; Supervision; Validation; Visualization; Writing—original draft; Writing—review & editing. Kalesha Hack: Conceptualization; Data curation; Formal analysis; Funding acquisition; Methodology; Project administration; Supervision; Validation; Visualization; Writing—original draft; Writing—review & editing. Phyllis Glanc: Conceptualization; Data curation; Formal analysis; Funding acquisition; Methodology; Project administration; Supervision; Validation; Visualization; Writing—original draft; Writing—review & editing. Debbie Elman: Formal analysis; Validation; Writing—review & editing. Rachel Kupets: Formal analysis; Validation; Writing—review & editing. Leslie Po: Formal analysis; Validation; Writing—review & editing. Sarah Taleghani: Data curation; Formal analysis. Lisha Lo: Formal analysis; Methodology; Validation; Writing—review & editing. Sarah E. Ferguson: Conceptualization; Data curation; Formal analysis; Funding acquisition; Methodology; Project administration; Supervision; Validation; Visualization; Writing—original draft; Writing—review & editing.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project was supported by the Innovation Fund of the Alternative Funding Plan from Academic Health Sciences Centres of Ontario and by the Department of Medical Imaging Internal Grant Fund.

ORCID iD

Genevieve Bouchard-Fortier

Supplemental Material

Supplemental material for this article is available online.

References

Glanc

Benacerraf

Bourne

, et al. First international consensus report on adnexal masses: management recommendations. J Ultrasound Med. 2017;36(5):849-863.

Timmerman

. Lack of standardization in gynecological ultrasonography. Ultrasound Obstet Gynecol. 2000;16:395-398.

Timmerman

Ameye

Fischerova

, et al. Simple ultrasound rules to distinguish between benign and malignant adnexal masses before surgery: prospective validation by IOTA group. BMJ. 2010;341:c6839.

Earle

Schrag

Neville

, et al. Effect of surgeon specialty on processes of care and outcomes for ovarian cancer patients. J Natl Cancer Inst. 2006;98(3):172-180.

Bouchard-Fortier

Gien

Sutradhar

, et al. Impact of care by gynecologic oncologists on primary ovarian cancer survival: a population-based study. Gynecol Oncol. 2022;164(3):522-528.

Liu

Chan

Bouchard-Fortier

Lheureux

Ferguson

Krzyzanowska

. Patterns of first-line systemic therapy delivery and outcomes in advanced epithelial ovarian cancer in Ontario. Curr Oncol. 2022;29(8):5988-6009.

Levine

Patel

Suh-Burgmann

, et al. Simple adnexal cysts: SRU consensus conference update on follow-up and reporting. Radiology. 2019;293(2):359-371.

Hack

Glanc

. The abnormal ovary: evolving concepts in diagnosis and management. Obstet Gynecol Clin North Am. 2019;46(4):607-624.

Andreotti

Timmerman

Strachowski

, et al. O-RADS US risk stratification and management system: a consensus guideline from the ACR Ovarian-Adnexal Reporting and Data System Committee. Radiology. 2020;294(1):168-185.

10.

Andreotti

Timmerman

Benacerraf

, et al. Ovarian-adnexal reporting lexicon for ultrasound: a white paper of the ACR ovarian-adnexal reporting and data system committee. J Am Coll Radiol. 2018;15(10):1415-1429.

11.

Timmerman

Valentin

Ceusters

, et al. External validation of the ovarian-adnexal reporting and data system (O-RADS) lexicon and the international ovarian tumor analysis 2-step strategy to stratify ovarian tumors into O-RADS risk groups. JAMA Oncol. 2023;9(2):225-233.

12.

Hack

Gandhi

Bouchard-Fortier

, et al. External validation of O-RADS US risk stratification and management system. Radiology. 2022;304(1):114-120.

13.

Ogrinc

Davies

Goodman

Batalden

Davidoff

Stevens

SQUIRE 2.0 (S tandards for QU ality I mprovement R eporting E xcellence): revised publication guidelines from a detailed consensus process. J Contin Educ Nurs. 2015;46(11):501-507.

14.

Semler

Keriwala

Clune

, et al. A randomized trial comparing didactics, demonstration, and simulation for teaching teamwork to medical residents. Ann Am Thorac Soc. 2015;12(4):512-519.

15.

Hamour

Yang

Lee

, et al. Association of the implementation of a standardized thyroid ultrasonography reporting program with documentation of nodule characteristics. JAMA Otolaryngol Head Neck Surg. 2021;147(4):343-349.

16.

Soong

Shojania

. Education as a low-value improvement intervention: often necessary but rarely sufficient. BMJ Qual Saf. 2020;29(5):353-357.

17.

Kupets

Fernandes

Miroshnichenko

Paszat

. Are too many imaging tests being performed in women with an adnexal mass? J Obstet Gynaecol Can. 2013;35(3):246-251.

18.

Rowe

. Ordering tests, delaying treatment. J Obstet Gynaecol Can. 2013;35(3):201-202.

19.

Zhou

Guo

Wen

Zhao

Liu

. The learning curve and difficult points of the O-RADS ultrasound risk stratification system in 54 trainees. Ultrasonography. 2022;41(2):365-372.

20.

Strachowski

Jha

Phillips

, et al. O-RADS US v2022: an update from the American College of Radiology’s ovarian-adnexal reporting and data system US committee. Radiology. 2023;308(3):e230685.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.04 MB