Abstract
Objective
To determine the accuracy of information recorded regionally and locally on the screening classification of cervical cancer cases using the national invasive cervical cancer audit categories.
Methods
Comparison of the audit categorization of all cervical cancer cases diagnosed at the University Hospital of North Staffordshire (UHNS) between January 2003 and December 2006 with the classification assigned by the West Midlands Cervical Screening Quality Assurance Reference Centre (WMQARC).
Results
Eighty-seven cases of cervical cancer were diagnosed during the three-year study period. There was agreement between the UHNS and WMQARC classification of cases in 52 cases (59.7%), moderate agreement κ = 0.51 (95% CI 0.39–0.63). The greatest disparity was seen in the classification of lapsed attenders, with nine of the 26 cases categorized as ‘lapsed’ by the UHNS being assigned to the ‘lost to follow-up’ category by WMQARC. Three cases were deemed unclassifiable by WMQARC using the national classification since the women were over the age of 70 years but had previously been enrolled in the screening programme, and currently there is no national category for these women.
Conclusions
Accurate and consistent classification of invasive cervical cancer cases is essential in order to obtain useful information on the efficiency of the national screening programme at a local, regional and national level. The use of a national algorithm would provide reassurance that all data used in the national evaluation of the NHS Cervical Screening Programme are consistent, meaning that robust conclusions could then be drawn from the data.
INTRODUCTION
The ‘Audit of Invasive Cervical Cancer’ 1 document was published by the NHS Cervical Screening Programme (NHSCSP) in December 2006 in which a framework was set out for the systematic review of all cases of cervical cancer. The principal aims of the exercise were to examine the effectiveness of the screening programme and to try and identify areas where improvements could be made, as well as providing feedback and close monitoring of the workings of the programme. Although the audit outcomes are defined, the actual process for conducting the audit is determined locally by each region and therefore mechanisms for data collection differ between regions. In the West Midlands, although the responsibility for conducting the necessary local reviews and collection of data lies with the patient's local unit, all the data are submitted to the regional Quality Assurance Reference Centre (QARC). Each case is then assigned its national screening classification by the QARC using a computerized algorithm. The QARC also arranges any external reviews required and validates all the submitted information. Completed cases are then submitted by the QARC to Cancer Research UK who are responsible for analysing the information on behalf of the NHSCSP. Collection of the audit data began on 1 April 2007 and, although the national data have been used to inform other publications 2,3 the authors understand that a report of the national data is not due to be published until at least Autumn 2010.
A detailed investigation of cervical cancer cases diagnosed at the University Hospital of North Staffordshire (UHNS), undertaken prior to the introduction of the invasive cervical cancer audit guidelines, highlighted problems with data ascertainment and recording both locally and with the regional QARC. 4 As no systematic national computer-based algorithm currently exists for the classification of cervical cancers, for the national audit, the majority of services will be manually determining a patient's screening history at diagnosis as UHNS did prior to the implementation of the national audit. In addition, the West Midlands QARC is unique in classifying all of the regional cervical cancer cases using a computer-based algorithm and this is the classification provided to the national audit. Since the collation of inaccurate or inconsistent information could have a major impact on the quality of the data produced at a national level, a study was performed in order to determine the accuracy and consistency of information recorded both regionally and locally based on the published national screening classifications for cervical cancer cases.
METHODS
All invasive cervical cancer cases diagnosed at the UHNS between January 2003 and December 2006 were included in the study. Information on the patient's screening history was obtained from the laboratory database and all available cytology and histology specimens were reviewed. The cases were subjected to an in-depth local review at the hospital's clinico-pathological correlation multidisciplinary meeting and were retrospectively classified by the UHNS into one of the six nationally described categories 1 : screen-detected, interval cancer, lapsed attender, never invited, never attended and lost to follow-up.
The invasive cervical cancer data supplied from the UHNS colposcopy service were matched against the cervical cancer and screening status classification data collated at the West Midlands Cervical Screening Quality Assurance Reference Centre (WMQARC) again using the guidelines in the ‘NHSCSP Audit of Invasive Cervical Cancer’ document (Table 1). At the WMQARC, screening histories data for each case are obtained electronically from the call and recall system using a web browser system called ‘Open Exeter’ and these data are used to assign a category from an algorithm combining the patient's screening status and screening history classification as described by Bagnall et al. 5 The dates of diagnosis were obtained from the invasive cervical cancer cases recorded on the West Midlands Cancer Intelligence Unit (WMCIU) cancer registration database.
Definitions of screening classification 1
The classifications assigned to each case of cervical cancer by the UHNS and WMQARC were compared and analysed using StatXact-4 version 4.0.1 (1999), from Cytel Software Corporation, (Cambridge, MA, USA). The measure of agreement between the classifications assigned to each case was calculated using kappa (κ), as described by Altman. 6
RESULTS
Eighty-seven patients were diagnosed with cervical cancer at the University of North Staffordshire (UHNS) during the three-year study period and their cases reviewed. The median age at diagnosis was 43 years, range 24–91 years. Twenty-eight cases (32.2%) were Stage 2 or above at diagnosis.
Screening classification
Initial agreement between the UHNS and WMQARC classification of cases occurred in 50 cases (57.5%), however, in six out of the 37 cases where there was a lack of agreement between the UHNS and WMQARC; no screening classification was attainable by WMQARC due to the patient being recorded as only having CIN3 (1 case) or the case not being notified to the cancer registry (5 cases). Upon registration of the five additional cases not previously notified, two cases subsequently had matching screening classifications. Investigation of the CIN3 case revealed that the diagnosis had been made from a lymph node biopsy, not from the cervical biopsy, and a supplementary report had not been sent to the cancer registry. Consequentially there was agreement in 52 cases (59.7%) and a discrepancy in 35 cases, indicating a moderate level of agreement, κ = 0.51 (95% CI 0.39–0.63), P < 0.001(Table 2).
Classification of cervical cancer cases by the University Hospital of North Staffordshire (UHNS) and the West Midlands Quality Assurance Reference Centre (WMQARC) following the registration of all cases, n = 87
The greatest disparity was seen in the classification of lapsed attenders, with nine of the 26 cases categorized as ‘lapsed’ by the UHNS being assigned to the ‘lost to follow-up’ category by WMQARC. For the total 10 cases classified as ‘never attended’ by UHNS, WMQARC did not reach agreement on the classification in four cases. All of these cases were found to have screening histories by the WMQARC and as a result were categorized as ‘lost to follow-up’ (2 cases), one case as ‘screen-detected’ and one case as ‘lapsed’.
Date of diagnosis
The date of diagnosis agreement recorded by UHNS and WMQARC ranged between 0 and 723 days with a median of one day. In 10 cases there was a substantial difference between the two dates, six dates were recorded between 164 and 723 days later by the UHNS than WMQARC and four dates were recorded 134 and 614 days earlier. The reason for such a disparity in dates was due to the UHNS clinical team using a diagnosis date that was different from that recorded in the UHNS pathology laboratory. The laboratory date agreed with that recorded by the WMQARC and the WMCIU in nine of the 10 cases. The remaining case had an incorrect date of diagnosis recorded at the WMCIU, relating to the date of histological confirmation of CIN3, which occurred just prior to the diagnosis of the invasive cervical cancer.
Private tests
In one case the discrepancy between categorizations was due to the inclusion by UHNS of a test taken privately in the patient's screening history. This negative test was taken 22 months prior to the patient's next NHS screening test, which was 12 months overdue. She was diagnosed with cervical cancer one month after her NHS screening test, resulting in a classification of an ‘interval cancer’ by the UHNS. The WMQARC, however, excluded the private test as this is disregarded for the purposes of call and recall by the NHSCSP if it is negative. Since the diagnosis of malignancy occurred 48 months after the woman's last NHS screening test, which recommended routine recall after 36 months, the case was classified by the WMQARC as ‘lapsed’.
Unclassifiable cases
Three cases were deemed unclassifiable by WMQARC since the women were over the age of 70 years at the time of diagnosis but had previously been enrolled in the screening programme as currently there is no national category for these cases. UHNS classified one of these cases as ‘lapsed’ and two as ‘never invited’.
Start of screening
One case was classified as ‘never invited’ by UHNS but as ‘never attended’ by WMQARC. The patient was aged 64 years on 1 January 1988, the official start of the national call and recall system, and therefore would have been eligible and have been invited for screening, therefore placing her in the ‘never attended’ category.
Differences in classified populations
Despite the differences in the classification of cases by the UHNS and WMQARC the age and stage at diagnosis was similar in the categories of both classifications (Tables 3 and 4).
Median age at diagnosis and classification of cases according to the NHS Cervical Screening Programme invasive cancer audit guidelines, University Hospital of North Staffordshire (UHNS) and West Midlands Quality Assurance Reference Centre (WMQARC) classifications
Figures in brackets are range of ages
Advanced stage at diagnosis (stage 2 and above) and classification of cases according to the NHS Cervical Screening Programme invasive cancer audit guidelines, University Hospital of North Staffordshire (UHNS) and West Midlands Quality Assurance Reference Centre (WMQARC) classifications
Statistical significance calculated using chi-squared with Yates' correction
DISCUSSION
This is the first study to investigate the accuracy of information recorded regionally and locally for the national invasive cervical cancer audit. We have shown that the classification of cases at both the local and regional level can be inaccurate in a large proportion of cases and therefore the data collated at a national level must be interpreted with some degree of caution.
Although the UHNS and regional classifications of the invasive cervical cancers included in this study are based on the same national guidelines, errors in the classification of cases can occur at a local or regional level and strict adherence to the definitions needs to be reinforced in order to ensure accurate categorization. Although the UHNS cases occurred prior to the publication of the national guidelines, the cases were manually reclassified to be consistent with the published categories and definitions. 1 At WMQARC the classification of cases to the national specification is achieved by the use of the computer algorithm, which removes subjectivity in the classification and ensures that the definitions are always adhered to. As a result, the two sets of data are directly comparable and give a good indication of the potential issues that can arise. The importance of the computer algorithm was particularly seen with the ‘lapsed attender’ and ‘lost to follow-up’ categories where the only difference between the two classifications is in the last screening test result, whether it was negative or abnormal. The exclusion of negative tests taken in the private sector is also a requirement that needs to be reinforced because although the result may be correct it has been taken outside of the screening programme and therefore should not be included in the review of the patient's NHS screening history.
Failure to access a patient's full screening history can result in misclassification, as was seen when cases categorized locally as ‘never attended’ were documented as ‘lapsed attender’ regionally, since WMQARC identified screening tests taken previously. Despite the differences in absolute classification, all these cases feature patient compliance as a principal contributing factor, which could be considered a higher level category match. The reasons for patient non-compliance may not be the same in the different subgroups with women who have never attended holding different beliefs concerning screening and reporting different reasons for non-attendance compared with lapsed attenders. 7,8
The age and stage distribution of cases in the separate categories was similar implying that a certain level of inaccuracy in the categorization of cases may be tolerated and not have an impact on the overall national conclusions of the at-risk populations or disease survival with respect to audit classification. However, the amalgamation of data from numerous regions in the UK may magnify errors in classification and therefore the ability to correctly classify the cases is still important in order to produce reliable and meaningful data.
The study did highlight three cases that were unclassifiable under the current national system. The women's last test was over five years before their diagnosis of cervical cancer; however, since they were over the age of 70 years they could not be recorded as lapsed attenders. Currently there is no ‘post-eligible’ or ‘no longer eligible’ category which would encompass such cases since compliance with the screening programme is not the underlying aetiology for the extended time from the patient's last screening test. Rather it is due to a cessation of screening invitations at the age of 65. The authors are aware that discussion is underway at a national level to review the classification system to address this and ensure all women can be categorized.
Another cause of discrepancy between the classifications concerned whether a woman had received an invitation for screening or not. The National Cervical Screening Programme started in 1988, therefore all women born after 1 January 1923 should have been eligible at some point and sent an invitation to attend. The actual date that letters were sent to women may have differed from the official start date as the programme was being rolled out nationally. This may have resulted in women who were eligible for screening not receiving an invitation if they reached the age of 65 years in the time between the official start date and the local implementation of the programme. This situation will only relate to a small number of cases and with an ageing population this number will decrease further with time. We believe that the assumption in the case of women born after 1 January 1923 with no screening tests should be ‘never attended’ rather than ‘never invited’.
This study also highlights data ascertainment issues experienced by the regional cancer registries from the local hospital trusts. In six cases the registry had not been informed of the diagnosis of malignancy and as a result WMQARC could not designate a classification. It is hoped that the implementation of this national audit will improve cancer registry data quality over time as the parallel reporting of cases directly from hospitals to the QARCs will provide an invaluable cross check for case ascertainment with the cancer registries. Discrepancies in the date of diagnosis between the local and regional databases and failure to supply supplementary pathology reports are problems which need to be addressed. Although a difference in the date of diagnosis of a few days will not impact on the validity of the data, discrepancies of several months or years will have an effect. Coordination and checking procedures need to be reinforced to reduce the errors in the recording of such data. Again the invasive cervical cancer audit will help with this.
CONCLUSIONS
Accurate and consistent classification of invasive cervical cancer cases at a local and regional level is essential in order to obtain useful information on the epidemiology of cases and assess the efficiency of the national screening programme within subpopulations of women. The use of a national algorithm would provide reassurance that all data used in the national evaluation of the NHSCSP are consistent, meaning that robust conclusions could then be drawn from the data. If implemented, a retrospective re-classification exercise using the algorithm should be considered nationally as this study shows that in a high percentage of cases the classification assigned locally differs from the regional QARC classification and consequentially this is likely to impact on the accuracy of national audit data and its use in evaluating the NHSCSP. Obtaining screening histories at a regional level through access to all call and recall systems ensures that all screening tests are included to gain a full screening history so that an over or underestimation of the benefits of the National Cervical Screening Programme does not occur.