Interval cancer audit and disclosure in cervical screening programmes: An international survey

Abstract

Background

Legal cases involving the National Cervical Screening Programme in Ireland following non-disclosure of an interval cervical cancer audit prompted this first international comparative survey of interval cervical cancer audit.

Methods

A survey of 22 international population-based cervical screening programmes was conducted, to determine if they undertook audit of invasive cervical cancers. Those countries/regions that perform reviews were asked (i) how the audit was undertaken, including how the reviews were performed and how they controlled for retrospective bias, (ii) how women are informed of the audit process and how their consent is obtained, and (iii) how audit results were disclosed to patients.

Results

Seventeen countries/regions invited completed the survey (77%); 65% (11/17) have an audit process for interval cervical cancers. Five perform individual patient reviews; three perform programme-wide review, with calculation of interval cancer detection rates; one routinely performs programme-wide review with calculation of interval cancer detection rates and offers individual reviews, and one routinely performs local hospital-level reviews. In the remaining country/region, hospital laboratories audit cancers, with a national audit process for all cervical cancers. Varying methodologies for retrospective cytology review were employed; four include control samples, with a ratio varying from 1:1 to 1:2. Three conduct a blinded review. Most countries/regions do not discuss interval cancer audit with participants and 3/11 (27.3%) inform women when a cervical cancer audit takes place. Disclosure is limited and variable.

Conclusion

The responses suggest that there is no consistent approach to audit of interval cervical cancers or to disclosure of audit results.

Introduction

CervicalCheck, the National Cervical Screening Programme in Ireland, commenced in 2008. It achieved 80% population coverage and 7% annual reduction in the incidence of cervical cancer between 2010 and 2015.^1,2 Quality-assured laboratories located in the USA and Ireland were commissioned to provide cytology services. Programme-specified quality assurance parameters included turnaround times, external quality assurance and proficiency standards for all staff involved in reporting samples. ¹ Fifteen colposcopy units provided follow-up care for screen-positive women via service level agreements with the National Screening Service, the overarching organization for adult population screening.

In response to the UK's National Health Service Cervical Screening Programme (NHSCSP) audit of invasive cervical cancers, ³ CervicalCheck commenced design of a programmatic audit of invasive cervical cancers identified in screened women. The goal was to review the screening histories and previous cytology for all women who developed cervical cancer after previously participating in the programme. Consistent with international experience, ⁴ similar to the NHSCSP audit, ³ the retrospective slide review identified a discordance rate of 30%. In approximately half of these cases, this discordance had an impact on the clinical care. Open disclosure by clinicians for medical incidents was implemented on a discretionary basis into Irish healthcare in 2014. In response to this, the National Screening Service was advised by the Health Service Executive (HSE) management to retrospectively disclose results of the interval cervical cancer audit from 2008 onwards. The disclosure commenced in February 2016. There was a lack of clarity on who was responsible for the disclosure of the audit information to patients. There was also no formal feedback to the programme on whether the disclosure took place or not. It later became clear that disclosure of the audit results to the women involved did not occur in many cases.

Two high-profile court cases came to public attention in 2018 when women brought legal cases against the laboratories for failing to detect high-grade changes earlier and against the Irish HSE for failing to disclose the results of their interval cancer audit. The cases received a lot of public attention, and the women expressed their anger at how the audit results had not been communicated with them. This led to a media and political frenzy which fuelled public concerns that the programme was not operating as well as other cervical screening programmes internationally. ⁵ A public health specialist from the UK was asked to do a review of the audit. One of the conclusions of his report ⁶ was that despite issues regarding system-wide governance, “all the laboratories….are meeting the regulatory requirements current in their own country”. A subsequent review by the Royal College of Obstetricians and Gynaecologists ⁷ also concluded that the Irish Cervical Screening Programme was performing at international standards and had a similar rate of slide discordance on retrospective review to the NHSCSP. ⁸

In 2019, the HSE established an Expert Reference Group to define future audit and disclosure of interval cancers in Irish screening programmes. ⁹ The advice on the future management of interval cancers in the National Screening Service was based on review of international evidence and best practice. To inform the Expert Reference Group, a survey of international population-based cervical screening programmes was performed to determine if they undertook audit of invasive cervical cancers. Those countries/regions that perform reviews were asked (i) to describe how the audit was undertaken, including how the reviews were performed and how they controlled for retrospective bias, (ii) to describe how consent for the audit was obtained and how women were informed of the audit process, and (iii) to describe how the results of the audits were disclosed to patients.

Material and Methods

The inclusion criteria for the survey were (i) a national or regional population-based cervical screening programme; (ii) a country or region with a population ≥ population of Ireland; (iii) programmes located in Europe, Australia or Canada; (iv) an identifiable contact email.

The final questionnaire was developed using an online interactive GDPR-compatible survey tool (www.smartsurvey.co.uk). The questions were in English and most of the answers were in a pre-defined format; however, free text format was also used, providing respondents with an opportunity to enter explanatory text and/or append supporting evidence of their policies/procedures. The online survey was subject to pre-testing and validations before going live. The questionnaire was sent via a link embedded in an emailed invitation to participate. This included a letter outlining the rationale for the research and what it would be used for. Respondents were advised they could contact the authors to clarify any points. Permission for anonymised publication was sought. A reminder mail was sent to countries/programme representatives after 4 weeks to maximise response. The questionnaire was completed online by each country/region. Following receipt of questionnaires, discussions were held with some respondents to clarify queries arising.

Results

Seventeen out of 22 countries/regions invited completed the survey, giving a response rate of 77%. Of the 17 countries/regions that completed the survey, 11 have an audit process in place for invasive cervical cancers that develop in the screened population.

In these 11 countries/regions there are a number of different approaches taken (Tables 1 and 2). Five routinely perform individual patient-requested reviews (two of the five review all cervical cancers in screened and unscreened women); three routinely perform programme-wide review, with calculation of interval cancer detection rates; one routinely performs programme-wide review with calculation of interval cancer detection rates as well as offering individual patient-requested review, and one routinely performs local hospital level reviews. In the remaining country/region, hospital laboratories audit all cancers and, more recently, a national audit process for auditing all cervical cancers has been introduced.

Table 1.

Results by countries/regions who undertake audit.

Region or country	1	2	3	4	5	6	7	8	9	10	11
Audit of interval cancers	*	*	*	*	*	*	*	*	*	*	*
Use control samples	*					*	*		*
Reviewers blinded to cancer status	*						*		*
Different audit procedure for individuals						*	*		*
Inform patients that an audit is taking place			*		*						*
Choice to participate in audit								*
Consent specifically captured to take part in a clinical audit							*
Routine consent procedure for screening covers the interval cancer audit process		*		*	*			*	*	*	*
Results of audit communicated to the affected patients			*		*	*					*
Ask women if they want to know the outcome of the audit			*		*						*
Open disclosure policy for medical incidents	*	*	*			*		*	*
Mandatory open disclosure policy		*	*			*		*
Open disclosure policy extends to the results of audit of interval cancers			*			*		*
Legal protection for cancers arising post screening	*		*								*
Financial compensation for interval cancers		*				*		*			*
Interval cancer rates reported internally			*	*	*	*	*		*
Publish interval cancer rates		*	*			*	*		*

Table 2.

Summary responses (n = 11) to key questions.

	Yes	No
Informs women that audit is taking place	3 (27.3%)	8 (72.7%)
Routine consent covers audit processes	7 (63.6%)	4 (36.4%)
Asks women if they want to know the results of the audit	3 (27.3%)	8 (72.7%)
Blinded review for audit	3 (27.3%)	8 (72.7%)
Open disclosure	3 (27.3%)	8 (72.7%)
Financial compensation	4 (36.4%)	7 (63.6%)

The programmes employed different methodologies for retrospective review of cytology. Four of the eleven programmes include control samples with audit cases when sent for cytology review. The case: control ratio varied from 1:1 to 1:2. Three programmes conduct a blinded review where the reviewer was unaware that the woman subsequently went on to develop cancer.

Comments relating to dealing with hindsight bias are in Table 3 and reflect the challenges of hindsight bias. Comments relating to individual patient review versus programmatic review included “If an individual patient or her doctor requests a review of screening tests, it is national policy to do so and to provide the results to the “caretaker” doctor, usually gynaecologist or oncologist involved, who are deemed the most appropriate to explain the findings to the patient” and “No central procedure of reviewing slides on request of patient; a review is only done in a court case”.

Table 3.

Comments from programmes relating to hindsight bias.

“Good question”

“We are not trying to validate the entire screening programme. Our objective is to try and understand how any woman develops cervical cancer and could this have been prevented in her case”

“With difficulty but must be pragmatic and use common sense”

“One or two pathologists review the latest screening sample. There are no checks for hindsight bias. We trust the pathologists to do their job”

Of the 11 countries who undertake audit, only three of these inform patients that a cervical cancer audit is taking place (27%). This is done by the woman's treating clinician supported by an information leaflet. The remaining eight countries/regions do not inform women that an audit is taking place and do not have a process of disclosing audit results. One country/region includes information on interval cancer audit on its website and another includes this in screening leaflets. Further detail from programmes mentioned that “Information on screening leaflet to say samples and data may be used for audit. Clinician may inform patient that an audit will be done but is not required to” and “Not informed [that they are part of an audit] unless clinician undertaking care informs them or woman asks” (Tables 2 and 3). One country seeks specific consent from all women at the time of screening to be part of an audit if they develop an interval cancer. Four countries/regions responded that their routine consent procedure for screening covers the interval cancer audit process, and one country/region with exemption from disclosing audit results does not seek consent or implied consent.

Six countries/regions had an open disclosure policy for medical incidents (Tables 2 and 3). This is expressly applied to interval cancer audit disclosure in three of the six regions surveyed. In response to a question on who is responsible for disclosure of the audit results; in two countries/regions it is the screening programme in conjunction with the treating oncologist/consultant and in one country/region this responsibility is fulfilled by a multidisciplinary team. In one country the laboratories are notified by the programme when an interval cancer occurs but there is no standardised protocol for communicating the results to the women and the programme does not request confirmation that all women receive the audit result. Notably, three countries/regions have legal protection in place to prohibit disclosure of clinical audit results when interval cancers arise in a screened population. Two countries/regions provided further information on their legal protection as follows:

“There is indemnity from the Country risk pool. If there is a successful legal claim, then the risk pool proforma is completed and returned to recoup costs” and “Patients who might have been adversely affected still have to engage via medical legal processes.”

Four countries/regions have financial compensation for interval cancers when it is proven that the cancer occurred as a result of a screening incident and was not due to the known limitations of screening. One programme advised the average compensation was €10,000 when a case of interval cervical cancer is identified.

Discussion

This is the first international review of interval cancer audit processes in cervical cancer screening. The respondent countries/regions do not have a consistent approach to audit of interval cervical cancers or to disclosure of audit results. There are only three countries/regions in our survey where disclosure of interval cancer audit results is recommended. Two countries have adopted an approach of full programmatic review and recommend disclosure to the patient, while a third country conducts cytology review only but also recommends disclosure. The majority of jurisdictions do not disclose the results of interval cancer audits to patients, as the audits are considered to be quality assurance and the reviews are considered of educational benefit rather than disclosable patient safety incidents. In some countries/regions, there is legislative protection from disclosure of clinical audit results.

Open disclosure policy in many countries refers to medical incidents where an unanticipated harm occurs to a patient. Since it is known in advance that all screening programmes are associated with false positives and false negatives these are not “unanticipated events”. There is a lack of international consensus on whether open disclosure should apply in screening programmes. It is also important to protect the patient's right to decline to receive the audit results. In a study in northern England just 40% of women sent an invitation to discuss audit results chose to attend. ¹⁰ Public Health England commenced audit of invasive cancer to facilitate identification of good practice as well as areas for improvement in 2006. ¹¹ The treating gynaecologist or oncologist was asked to discuss the review findings with the patient. Sherman et al. surveyed lead colposcopists regarding disclosing the results of invasive cervical cancer reviews to patients in 2015. ¹² This study preceded Duty of Candour legislation but followed the introduction of an open disclosure policy. The authors found that despite NHS Cervical Screening Programme guidelines only 53% of all responding clinicians offered review meetings to patients and those that did offer meetings did not offer to all patients. Clinicians reported a mix of positive and negative aspects of meetings and, overall, 51% reported offering 15 or fewer review meetings over 5 years. For the 47% of respondents who did not offer review meetings, the reasons cited included lack of awareness of guidelines, time constraints, fear of causing distress, and fear of litigation.

A Danish study ¹³ found that over 40% of interval cancers were true interval cancers. The authors also found that 45% of interval cancers were due to lapsed attendance for cervical screening (defined as no cervical cytological screening performed within 3.5 years of cancer diagnosis for women up to 50 years of age and within 5.5 years of diagnosis for women between 50 and 65 years of age, and no screening test performed between 55 and 65 years of age for women over the recommended screening age). This emphasizes the importance of regular attendance and optimizing population coverage. ¹³ A case-control register based study in Sweden found that women who failed to attend the two previous screening rounds had four times the risk of developing cancer compared with women who were appropriately screened (odds ratio = 4.1, 95% confidence interval 3.8–4.5). ¹⁴ These studies demonstrate that some reviews will find that suboptimal attendance was a contributory factor in the development of cervical cancer. This is difficult information for patients to receive.

As is the case with other cancer screening programmes, audit in cervical screening programmes is largely considered to be an educational process. ¹⁵ Those programmes who audit cytology typically audit all invasive cervical cancers as there is no agreed international definition of an interval cervical cancer. Cervical cancer screening is different to breast cancer screening as it identifies a risk factor for the development of cancer (high grade cervical intraepithelial neoplasia) rather than identifying a cancer. Cervical screening is a preventative process rather than a diagnostic one. The inability to blind reviewers to the fact that the woman went on to develop cervical cancer is a major limitation of any cervical cancer audit. ⁴ Four of the 11 programmes include control samples with audit cases and three programmes conduct a blinded review in an attempt to correct for retrospective bias.

A 2014 literature review of informed consent found that patients’ recollection and understanding of the medical procedure, risks and complications is often low. Healthcare providers need to encourage patient participation to improve their recall of information and their understanding of planned treatment. For all screening programmes, education of the public regarding screening benefits and limitations is a key step in facilitating successful audit and disclosure.^16,17

Despite quality assurance and highly trained personnel, cytology still has limitations and will not detect and prevent all cancers. ⁴ The expert reference group ⁹ confirmed the considerable evidence that exists regarding the limitations of cytology. Several papers from the NHS England Cervical Screening Programme demonstrate this; a study of invasive cancer between April 2007 and March 2010 found a 55% discordance rate when 3759 slides originally deemed normal were reviewed retrospectively. ¹⁸ A later paper published by the same team in 2014, found a 41% discordance rate between the original and the review result. ¹⁹ The most recent published audit revealed that 42% of negative smears in women aged 25–49 were upgraded on review. ²⁰ When a woman develops cancer and a review of her previous slides is performed, it is expected that abnormalities will be detected in 30–55% of cases that were not seen at the time of the original screen. This highlights the effect of retrospective bias and the subjective nature of cervical cytology.

Like many countries, Ireland has recently introduced primary HPV screening. This will improve sensitivity, but it will still not detect and prevent 100% of cancers. ²¹ A Cochrane review in 2017 found that for every 20 women with precancerous changes, the HPV test will correctly identify 18 of these women where screening with primary cytology identifies 15 out of 20. ²²

The strengths of this study include its status as an original international study of this important topic in cervical screening, provision of an overview of international practice, a high response rate, and an opportunity to clarify responses directly if needed. However, we acknowledge that for those countries/regions where respondents may not have been aware of the Irish Cervical Screening Programme, this was a “cold-call” email which may have deterred some from responding.

The Expert Reference Group utilised these survey results and consultation with international experts to decide on recommendations, which include the following ⁹ :

Women should continue to be provided with all the information they require in order to help them make an informed choice to consent to participate in the cervical screening programme. Current informational materials should be revised in order to reinforce the information on the benefits and limitations of screening.

A process to conduct patient-requested reviews of all invasive cervical cancers (both interval and screen-detected cancers) is to be established, using a standard operating procedure for this purpose. Patient-requested reviews should only be undertaken following receipt of written consent from the patient. The findings of all patient-requested reviews should be fully disclosed, with responsibility for disclosure of the review outcome resting with the treating clinician.

The cervical screening programme should develop a new KPI, the interval cervical cancer rate. The programme should liaise with other international screening programmes and the International Agency for Research on Cancer (IARC), specifically with regard to: (1) definition of interval cancers, (2) methodology to calculate the interval cancer rate, and (3) benchmarking for participating programmes.

Conclusions

The results of this study confirm a lack of consistency in the approach to interval cervical cancer audit and disclosure. We know also that there is currently no agreed international definition of an interval cervical cancer. There is a lack of consensus regarding inclusion of microinvasive disease, whether to include screen-detected cancers as well as cancers that develop between screens, and the inclusion of cancers that develop in older women after their discharge from the screening programme. It is also unclear where the interval commences, i.e. at last normal smear or last mildly abnormal smear. This, added to inconsistent approach to audit and lack of publication of results, makes international benchmarking very difficult. Inconsistent approaches and lack of an agreed international definition of interval cervical cancer prevent the calculation of comparative statistics. To achieve comparative statistics and benchmarking, which would be beneficial for all cervical screening programmes, international collaboration and consensus are essential to ensure that trust can be maintained in cervical screening. The CervicalCheck screening programme in Ireland is currently working with the IARC on a project aimed at achieving international consensus regarding interval cancer definition and interval cancer rate calculation methods, to facilitate benchmarking between programmes.