Abstract
Objectives
Cervical cancer elimination requires high-performance screening tests and high treatment rates, and thus high screening program performance is essential; however, Latin America lacks organized screening and quality assurance (QA) guidelines. We aimed to develop a core set of QA indicators suitable to the region.
Methods
We reviewed QA guidelines from countries/regions with highly organized screening programs and selected 49 indicators for screening intensity, test performance, follow-up, screening outcomes and system capacity. A regional expert consensus using the Delphi method in two rounds was implemented to identify basic indicators actionable within the regional context. The panel was integrated by recognized Latin American scientists and public health experts. They voted for the indicators blinded to each other based on feasibility and relevance. The correlation between both attributes was analyzed.
Results
In the first round 33 indicators reached consensus for feasibility but only 9 for relevance, without full coincidence. In the second round 9 indicators met the criteria for both (2 screening intensity, 1 test performance, 2 follow-up, 3 outcomes, 1 system capacity). A significant positive correlation was observed for test performance and outcomes indicators between the two attributes assessed (p < 0.05).
Conclusions
Cervical cancer control requires realistic goals supported by proper programs and QA systems. We identified a set of indicators suitable to improve cervical cancer screening performance in Latin America. The assessment by an expert panel with a joint vision from science and public health practice represents a significant progress towards real and feasible QA guidelines for countries in the region.
Keywords
Introduction
Screening coverage over 70% with a high-performance test and treatment of at least 90% of cervical lesions are key components of the WHO strategy for cervical cancer elimination. 1 Accordingly, effective screening programs are essential to achieve elimination objectives, but quality assurance (QA) is essential to effective programs. 2
The majority of high-income countries have significantly reduced cervical cancer incidence and mortality as a result of organized screening. 3 Although Latin American countries have a long tradition of screening programs, the region continues to bear the second-highest cervical cancer incidence and mortality worldwide, and the reported incidence for the region during 2020 is much higher than the elimination threshold defined by the WHO (17.9 and 4.0 per 100.000, respectively).1,4
Recommendations for cervical cancer screening significantly vary between countries in the region with different target populations, screening intervals and screening tests, and there is also a wide variability in program implementation. The reasons for such variability are not clearly established but it may be related to insufficient scientific background for guideline development (only three countries in the region have developed national evidence-based guidelines), low resource availability, and influence of competing interests. Accordingly, most Latin American countries lack organized screening and QA programs, as indicated by the absence of publicly available comprehensive reports on program performance. 5
Latin American countries have been focused on screening coverage and information about other program components is scarce, whereas most high-income countries in Europe, North America and Oceania have actually developed comprehensive QA guidelines for cervical cancer screening.6,7 Furthermore, the reports on coverage are mostly based on demographic and health surveys aimed at reproductive health rather than program management data, thus resulting in non-harmonized and non-standardized information. 7
Despite the availability of QA guidelines from high-income countries, those guidelines have been developed in the context of highly organized screening and consider an extensive number of indicators, making difficult their adoption by low- and middle-income countries. Therefore, we invited a group of Latin American experts to identify a set of indicators suitable to the regional context and essential for improving early detection to move forward in pursuing cervical cancer elimination.
Methods
The study was carried out in two phases between March and November of 2020 (Figure 1). Initially we built a list of indicators by reviewing QA guidelines for cervical cancer screening from the European Union, Canada and Australia, in order to have the perspective from organized programs in these three world regions, coupled with the WHO guidance on program monitoring.8–12 We selected indicators based on the reported evidence about impact on program performance by the corresponding guideline3,13,14; however, the majority of indicators in the reference guidelines were included (European Union 18 out of 20, Canada 12 out of 14, Australia 9 out of 20, and WHO 5 out of 8). A preliminary list was reviewed by a European expert, external to the research team, to provide input on the relevance and definition of the selected indicators. Also, addition of indicators not included in these guidelines (six indicators described in Supplemental Table 1) and adjustment of some indicators to the local context were done by the research team (denomination, definition and specification of target populations and type of test). The final list comprised five categories as follows: screening intensity, screening test performance, women's follow-up (diagnostic workup and treatment access), early detection outcomes and system capacity. We included an additional category for screen-and-treat approaches as proposed by the WHO guidelines given the characteristics of screening programs in the region (Supplemental Table 1).5,12

Phases of the study (see also Supplemental Table 1 for initial list of indicators).
During the second phase an expert consensus using the Delphi method was implemented with the aim of identifying a set of indicators (basic and extended) for monitoring cervical cancer screening programs suitable to the regional context. The expert panel members were selected by the research team, and included gynaecologists, pathologists, public health specialists and cytology lab experts (Table 1). In total, 14 out of 15 invited experts accepted to participate, representing 10 Latin American countries from Central and South America, and one expert representing a multilateral organization based in the USA. The selection was based on their expertise and publication record on cervical cancer screening. Some of these experts work in the clinical setting, some in public health programs, and others are dedicated to research. All participant experts voted in the first round but one did not participate in the second round.
Characteristics of the participating experts (N = 14).
PAHO: Pan American Health Organization.
The Delphi process was planned in two rounds: in the first round the most relevant indicators in each category were selected, and in the second round the expert panel voted on those indicators without agreement in the first round. The a priori objective was to define a core set of indicators (maximum 10) and an extended set of potential additional indicators (maximum 5).
The panel voted for the indicators based on two attributes: feasibility and relevance. The objective and the operational definition of every indicator were provided as well as the criteria for assessing both attributes; however, for the latter the panel were free to use their own criteria (Supplemental Material 4). The voting was done in an on-line survey (RedCap®), which was first piloted by individuals external to the project to review its structure and format to ensure understandability and completeness. The participation and the vote were blinded: no expert was aware of the identity of other panel members or votes by them; yet, for the second round the panel was informed of the first-round results. Feasibility was evaluated using a four-point Likert scale (Totally feasible; Feasible; Slightly feasible; Not feasible). Relevance was defined by a ranking process (highest to lowest) using the Borda count methodology 15 ; briefly, a score was assigned to each indicator according to the position defined by every expert starting with 0 for the lowest position, and the final score for each indicator is the result of adding individual scores by each expert. During the entire process questions and suggestions from the panel were addressed by the research team and shared with the panel members.
A database was built with the sum of the scores for each indicator with respect to the two attributes evaluated. Using STATA 16.1 ® a correlation analysis was done between feasibility and relevance scores within each category of indicators using the Pearson correlation coefficient; these results were also reported to the panel before the second round.
For analyzing the voting on feasibility, we grouped scores for each indicator into two classes (feasible or not feasible), and the percentage of votes for each indicator in each class was estimated. For analyzing the voting on relevance, we defined three classes according to the position given to each indicator by every expert within every indicator category (relevant, medium relevant, not relevant). Since only four categories have an even distribution of indicators in the three classes (screening intensity, follow-up diagnosis, outcome and screen-and-treat), we defined the classes a priori for the remaining categories (screening test performance 3-2-3, follow-up treatment 2-1-2, system capacity 2-0-2). Consensus was considered when at least 70% of experts agreed on the attributes either positively (feasible or relevant) or negatively (no feasible or not relevant).
We planned to select core indicators based on their final score from highest to lowest if they were qualified as feasible by 70% or more of the expert panel. Since fewer than 15 indicators met these criteria, a 65% cut-off point was set to conform the extended group of indicators.
Results
We provided a list of 49 indicators for the first round (Supplemental Table 1). For the feasibility attribute 33 indicators reached consensus but one was considered non-feasible (Supplemental Table 2). For the relevance attribute 16 indicators reached consensus: 9 as relevant, 2 medium relevant, and 5 not relevant, respectively.
In the second round 17 and 29 indicators were voted on for the feasibility and relevance attributes, respectively. Three indicators reached consensus as feasible, and four indicators reached consensus for the relevance attribute (two as relevant and two as not relevant).
In total, nine indicators reached negative consensus with only one as non-feasible (Supplemental Table 2). In addition, nine indicators met the criteria of relevance and feasibility and consequently were proposed as core indicators (Table 2). For the extended set we identified two indicators with consensus as medium-relevance and feasible, and two additional indicators that more than 65% of experts considered to be relevant and feasible (Table 3).
Core indicators for monitoring of cervical cancer screening programs in Latin America.
CIN: cervical intraepithelial neoplasia; ER: expert reviewer.
Extended indicators for monitoring of cervical cancer screening programs in Latin America.
HPV: human papilloma virus; VIA: visual inspection with acetic acid.
The categories of the selected core indicators are: two of screening intensity, one of screening test performance, two of women's follow-up (treatment access), three of program outcomes and one of system capacity. The extended set correspond to screening intensity (one indicator), women's follow-up (one indicator on diagnostic workup), program outcome (one indicator) and screen-and-treat approaches (one indicator) (Table 3).
There was a significant positive correlation between relevance and feasibility in the categories test performance and outcome, and not significant in women's follow-up (treatment access). A non-significant negative correlation was observed between the two attributes for the categories women's follow-up (diagnostic workup) and screen-and-treat approaches. No correlation was observed for the remaining categories (Figure 2).

Correlation between feasibility and relevance for each category of indicators for monitoring of cervical cancer screening programs, first round (see also Supplemental Table 3). *Statistically significant (p-value < 0.05).
Discussion
In the present study, a consensus was reached on crucial indicators for monitoring cervical cancer screening programs in Latin America. We consider this information vital to improve program performance because to our knowledge no comprehensive QA guidelines are in place, but also because the consensus considered not only the gold standard in QA from high-income countries but also the suitability of implementation within the regional context. High-quality programs will ensure a proper balance between benefits and risks of screening, and increase effectiveness of cervical cancer early diagnosis. Beyond the direct impact on program performance, we think having a common set of indicators will make possible benchmarking between Latin American countries instead of comparisons with high-income countries, and in consequence will allow a better assessment of program status towards cervical cancer elimination. 2
We pre-defined the objective to find 10 core indicators based on minimum number in our preliminary review of QA in high-income countries (Canada, 12 indicators), 11 but in fact the regional expert panel found a limited number of indicators to be both feasible and relevant, a point of view that significantly differs from QA programs in other regions. In contrast, Australia has 20 indicators 8 and Europe 20 indicators. 9 New Zealand has eight basic indicators but these can reach up to 13 including sub-indicators, or more considering the different reporting values based on age, ethnicity or other variables. 16 The agreement on a reduced number of indicators is of great relevance since QA assurance based on a large number could become unachievable in Latin America, where screening programs are not well organized and there is heterogeneity between countries regarding availability of resources and health system capacity. Accordingly, the lack of QA or the implementation of unsuitable QA guidelines could lead to improper definition of goals and priorities, and impractical cancer control plans.2,17 In that sense, the wide knowledge of the expert panel about regional health systems and screening programs, complemented by their technical and scientific skills, represents a strength of a methodology that has been previously used in other world regions for the same purpose. 18
Historically, coverage has been the most used indicator for screening program monitoring worldwide. Although early data from Europe showed an inverse correlation between screening coverage and cervical cancer incidence, 19 there is evidence from Latin America showing a lack of correlation between these variables, suggesting a lack of impact of cervical cancer screening or at least that screening coverage cannot fully explain the burden of disease.20–22 Although the most common indicator, the lack of population-based programs results in the lack of program coverage estimates; therefore, most Latin American countries rely on national household surveys. However, there is no harmonization between household surveys aimed at gathering data on reproductive health and national recommendations for cervical cancer screening, thus resulting not only in significant variability over time but also in lack of coherence regarding screening intervals, target populations and type of tests.5,20,21 There is no question about the relevance of screening coverage for successful screening and the expert panel clearly agrees on it; however, there is an urgent need for the region to harmonize sources of information regarding target populations and specific screening intervals for every country as recommended by the panel, while moving forward in program organization. Furthermore, Latin American countries have updated national recommendations to introduce new screening tests and algorithms following international recommendations (HPV tests, visual inspection, screen-and-treat, etc.)5,23; in consequence, greater variability is expected in terms of coverage data and for that reason the expert panel considers the most reliable but feasible information could be the history of screening during the previous year to the surveys, regardless of the type of test used (Table 2).
Although not frequently monitored, adherence or retention to diagnostic workup and treatment of precancerous lesions has shown a significant impact on reduction of cervical cancer burden.1,20 Hence, indicators on loss to follow-up are consistently associated with cervical cancer mortality, showing that a reduced loss to follow-up may have a great positive impact even with reduced screening coverage, 24 and some estimates show that screening algorithms with treatment provision in the same visit among women 35 years old and over could reduce their lifetime risk of cancer by up to 36% at a low cost per year of life saved, even with only one screening round in their lifetime. 25 Finally, treatment of 90% of pre-cancerous lesions is a strategic goal for the WHO elimination initiative. 2 Despite its relevance, no country in Latin America systematically collects data on this indicator. Most reports from the region correspond to non-population-based studies or studies restricted to local data. In addition, the available information is difficult to classify given the increasing variability already described in screening tests (HPV clinician collected, HPV self-collected, VIA), and follow-up strategies (screen-treat, screen-triage-treat, cyto triage, colpo triage, etc.).26–29 In this context, indicators on treatment rates included in the core set are complemented with indicators on compliance, with referral rates among positive screened women and treatment rates in the screen-and-treat approaches included in the extended set of indicators.
We found several indicators about test performance in our preliminary review of QA guidelines (Supplemental Table 1); however, none of them reached consensus from our expert panel apart from cytology specimen adequacy (Table 2). The latter is a traditional indicator of screening programs in the region; thus, we think the panel took into consideration the fact that, despite updated recommendations, cytology remains the predominant screening test all over Latin America. 5 The lack of consensus is mainly related to concerns around feasibility of data collection; however, given its relevance and the absence of indicators for tests other than conventional cytology, it is highly desirable to include some of the proposed indicators from international guidelines in the future.
Program performance indicators should correlate with screening outcomes. Reduction in cervical cancer incidence and mortality are the ultimate goals of cervical cancer screening and of the WHO elimination initiative.10,30,31 The expert panel considered both indicators relevant and feasible. Indeed, all Latin American countries have well-established vital statistics systems, and only El Salvador does not have reports on mortality data to the WHO database after 2015. 32 In contrast, only 13 out of 19 countries in the region have collected incidence data 33 and only six are included in the Cancer Incidence in Five Continents database. 34 Implementation of population-based cancer registries is critical for monitoring progress toward cervical cancer elimination and accordingly greater efforts in the region are necessary. Given the method of analysis, we ended up with two mortality indicators that might be redundant, in any case age-standardized rates allow comparisons between countries and between different periods within countries; thus, these should be prioritized over crude rates, reducing the total number of core indicators from 9 to 8 (Table 2). Since incidence and mortality would consistently change only in the long term, the expert panel recommends using the last year of available data to improve timeliness of program monitoring; however, great caution is advised given the potential variability for reasons other than real changes in the two indicators (data quality).
Our study is not exempt of limitations. The preliminary review of QA guidelines was not systematic but focused on countries or regions recognized as having highly organized screening; despite this limitation, we think our approach was appropriate to collect the most relevant indicators for QA of cervical cancer screening. Using virtual tools could have reduced the chance of open discussions and feedback of information among panel members; however, the methodology was designed to maintain the independence of panel members and therefore their objective appraisal of indicators. The panel was composed of experts from 10 out of 18 Latin American countries; thus, it could be argued that it was under-representative. Yet, we included experts from countries with different levels of income, population sizes, health systems and levels of program organization as indicated by the available data on screening coverage. 33
Conclusions
Cervical cancer control is a public health priority for low- and middle-income countries, where most cases and deaths occur. 35 Accordingly, the WHO elimination strategy should consider not only realistic goals but also realistic QA systems. Although not perfect, we deem the set of core and expanded indicators agreed from this study to be a suitable approach to improve the performance of cervical screening in the region. Some indicators perfectly align with the WHO elimination objectives (see indicators 1, 2, 4, 5 and 6 in Table 2), thus rendering them ideal for tracking the progress of this initiative by the Pan American Health Organization and WHO. The remaining indicators could be used for QA if included in national guidelines, as not only was their relevance considered but also their suitability for healthcare systems including availability of resources. In consequence, we consider that assessment of the indicators by an expert panel with a joint vision from science and public health practice represents significant progress towards the feasible strengthening of cervical cancer early detection in Latin America.
Supplemental Material
sj-docx-1-msc-10.1177_09691413231178253 - Supplemental material for Is it feasible to enhance quality assurance of cervical cancer screening in Latin America? A regional expert consensus
Supplemental material, sj-docx-1-msc-10.1177_09691413231178253 for Is it feasible to enhance quality assurance of cervical cancer screening in Latin America? A regional expert consensus by Valentina Rangel, Ginna Paola Fernández-Deaza, Juan Sebastián Castillo and Raúl Murillo in Journal of Medical Screening
Footnotes
Acknowledgements
This project would not have been possible without the help of the expert panel: Silvina Arrossi (Centro de Estudios de Estado y Sociedad y Consejo Nacional de Investigaciones Científicas y Técnicas – Argentina), Catterina Ferreccio (Pontificia Universidad Católica de Chile), Eduardo Lazcano (Instituto Nacional de Salud Pública -México), Mauricio Maza (Basic Health International - El Salvador), Carolina Wiesner (Instituto Nacional de Cancerología – Colombia), José Jerónimo (Liga Contra el Cáncer – Perú), Jaqueline Figueroa (Programa Nacional Contra el Cáncer – Honduras), Mauricio González (Instituto Nacional de Cancerología – Colombia), Guillermo Rodríguez (Comisión Honoraria de Lucha contra el Cáncer – Uruguay), Alejandro Calderón (Caja Costarricense de Seguro Social), Itamar Bento Claro (Instituto Nacional de Câncer – Brasil), Flavia de Miranda Correa (Instituto Nacional de Câncer – Brasil), Esperanza Teusaba (PatoLab – Colombia) and Silvana Luciani (Pan American Health Organization), and the contribution of Beatriz Serrano (Institut Catalá d’Oncologia – Spain) in the preliminary review of indicators.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Ethical approval
This study was approved by the Ethics Committee at Hospital Universitario San Ignacio and School of Medicine of the Pontificia Universidad Javeriana.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
