Abstract
Introduction
Lung cancer has one of the lowest 5-year survival rates of all cancers in Canada at 22%. 1 This can be attributed to the high proportion of lung cancers diagnosed at late stages, 2 which are associated with reduced survival rates. 3 Fortunately, earlier diagnosis with low-dose computed tomography (LDCT) screening programs reduce lung cancer mortality. 4
The Canadian Task Force on Preventive Healthcare (CTFPHC) recommends lung cancer screening with annual LDCT for three consecutive years in patients 55–74 years old with ≥ 30 pack-years of smoking history who continue to smoke or have quit within the past 15 years. 5 These recommendations were based on the National Lung Screening Trial (NLST), an American study which demonstrated a 20% reduction in lung cancer mortality when patients were screened with LDCT compared to chest radiography. 6 Despite the CTFPHC recommendations, Ontario is currently the only Canadian province with a formal lung cancer screening program, though this program is not yet province-wide. 7 Informal screening occurs on a limited basis in Canada, but many physicians are not following CTFPHC guidelines. 8
While NLST-based guidelines are prevalent in Canada and elsewhere, other proposed screening methods include risk calculators. Age and smoking are the sole criteria used for NLST-based screening; however, other important lung cancer risk factors include air pollution and exposures to industrial chemicals, asbestos, and radon, as well as low socioeconomic status, a personal history of chronic obstructive pulmonary disease (COPD) or cancer, and a family history of lung cancer. 9 Tammemägi et al 10 incorporated some of these factors into a prediction model entitled PLCOm2012, and determined that patients with a risk score ≥ 1.51% had consistently lower mortality rates in the LDCT arm compared to the chest radiography arm of NLST cohorts. Furthermore, when applied to the participants of the Prostate, Lung, Colorectal, and Ovarian (PLCO) Screening Trial, and more recently, an international prospective cohort study, 11 this ≥ 1.51% risk threshold achieved better sensitivity, specificity and positive predictive value than NLST-based criteria. 12 Tammemägi and colleagues also developed the PLCOall2014 risk calculator to incorporate never-smokers; however, as no PLCO never-smokers met a ≥ 1.51% risk threshold, they concluded that never-smokers should not be screened. 12
A prototype of the PLCOm2012 model was used in the prospective Pan-Canadian Early Detection of Lung Cancer (PanCan) Study that showed a risk threshold of ≥ 2.00% can lead to superior efficacy in lung cancer screening with earlier cancer detection compared to the NLST model. 13 While PanCan did not assess a ≥ 1.51% risk threshold, an Australian study on ever-smokers determined that sensitivity of a ≥ 1.51% risk threshold was 11% greater than a ≥ 2.0% threshold, while the specificity decreased by 8.5%. 14
The PLCOm2012 model is also being used for lung cancer screening in the UK 15 alongside the Liverpool Lung Project (LLPv2) model which was developed from a case-control study in Liverpool, UK. 16 However, the National Health Service (NHS) has proposed a higher risk threshold at ≥ 2.5% for the LLPv2 to limit unnecessary imaging. 17 LLPv2 ≥ 2.5% selected an equal number of patients for screening as PLCOm2012 ≥ 1.51% in a Manchester, UK cohort of ever-smokers, and both of these criteria selected more patients than NLST criteria. 18
Although there are rapid assessment review panels that expedite diagnosis and treatment of lung cancer, such as our Thoracic Triage Panel (TTP), 19 there are few formal screening programs and trials in Canada. While there are screening programs under development, there is not yet an official consensus on whether risk prediction models should be used in lieu of CTFPHC guidelines. Therefore, the purpose of this study was to retrospectively compare the sensitivities of three well-known lung cancer risk prediction models, (1) PLCOm2012, (2) PLCOall2014 and (3) LLPv2, against CTFPHC recommendations and determine which risk model and risk threshold selects the most patients in a cohort who have confirmed lung cancer.
Methods
We performed a retrospective chart review on patients diagnosed with lung cancer between 2015–2019 in the Eastern Health region of Newfoundland and Labrador (NL) to determine whether they would have been eligible for screening at time of diagnosis. We randomly selected 300 of 1000 patients with confirmed lung cancer from the pre-existing TTP database.
Three risk-prediction models were applied to these patients to retrospectively calculate the risk of developing lung cancer: (1) PLCOm2012, calculated through the online ‘Should I Screen’ webpage provided by the University of Michigan10,20; (2) PLCOall2014, a Canadian model developed by Tammemägi and colleagues 12 ; and (3) the Liverpool Lung Project (LLPv2) Risk Model, developed by Cassidy et al. 16 Each model relies on various predictive factors, and these variables were not available for all our patients. To ensure high-quality and reliable data, we included only 120 patients who had sufficient data for risk calculations using all three models.
Data were collected from the TTP database and electronic medical records. Patients’ smoking history, age at diagnosis, sex, weight, height, level of education and ethnicity were recorded. If the level of education was not clearly stated, it was assumed to be the lowest education required for the patient’s employment. Additional variables were collected for medical history such as a self-reported 1st- or 2nd-degree family history of lung cancer; history of malignancy or asbestos exposure; and previous diagnosis of pneumonia, emphysema, bronchitis, tuberculosis or COPD
The above information was used to determine the number of patients who would have been selected based on CTFPHC lung cancer screening guidelines. Additionally, the number of patients who had a risk prediction of ≥ 2.00% and ≥ 1.51% in all three models were tallied, representing patients who would have been eligible for screening according to the risk threshold of the PanCan study, 13 and the risk threshold recommended by Tammemägi et al, 12 respectively.
OncoSim was used to estimate the effect of changing risk thresholds on the number of patients screened. OncoSim is a microsimulation model that simulates cancer in the Canadian population using Canadian demography data (birth, immigration, migration and death) and cancer incidence and mortality data from the Canadian Cancer Registry.21,22 Briefly, the OncoSim-Lung model simulates lung cancer risk in smokers and non-smokers using the age- and smoking-related coefficients in the PLCO2014 risk equation. It also accounts for increased risk of lung cancer with radon exposure and has a residual term that takes into account other factors; the residual term was calibrated to match Canadian lung cancer incidence data by age, sex and province. 23 OncoSim simulates the smoking behaviour of the Canadian population (smoking initiation starting from age 15 years, quit rate and smoking intensity) using data from three large Canadian health surveys.24-26 Smoking trajectories were externally validated against other survey years and tobacco manufacturers’ data,27,28 and smoking initiation rates were also calibrated to match the smoking prevalence by 5-year age group, sex and province. 29 To project the impact of lung cancer screening, we assumed lung cancer screening started in 2022; 40% of individuals eligible would attend screening, and the target participation rate would be reached in the next 3 years. This analysis is based on the Canadian Partnership Against Cancer’s OncoSim model. OncoSim has been made possible through a financial contribution from Health Canada, through the Canadian Partnership Against Cancer. The assumptions and calculations underlying the simulation results were prepared by Eastern Health and the responsibility for the use and interpretation of these data is entirely that of the authors.
Data were presented as mean and standard deviations for continuous data and frequency and percentages for categorical data. Student t-tests were used for continuous data, χ2 tests for categorical data and Mann–Whitney U test for ordinal data. McNemar’s χ2 tests were used to determine whether there were significantly more patients that would have been selected for screening by one model compared to the others, or by the ≥ 1.51% threshold compared to the ≥ 2.00% threshold. Statistical analyses were performed using SPSS version 27.0 (IBM SPSS Statistics) and R version 3.6.1. A P-value < .05 was considered significant.
This study was approved by the NL Health Research Ethics Board (HREB) and the Research Proposals Approval Committee (RPAC) of Eastern Health.
Results
Patient characteristics which were used in the calculation of risk of lung cancer with any of PLCOm2012, PLCOall2014 and LLPv2 models.
Comparisons of the number of patients who would have been selected for screening, between each of three risk predication models, at each of two risk thresholds, and the CTFPHC guidelines (N = 120).a
aThe first value in each comparison cell corresponds to the selection criteria of that cell’s row, whereas the second value in each comparison cell corresponds to the selection criteria of that cell’s column.
bIndicates significance at the P < .05 level.
OncoSim projected impact of lung cancer screening in Canada in 2022–2024.
Differences in patient characteristics between those who were selected by at least one risk prediction model and those who were missed by all three risk prediction models, at a threshold of > 1.51%.
aP-value for Pearson’s chi-square test.
Discussion
We compared CTFPHC screening guidelines to three lung cancer risk prediction models and found that all three models were more sensitive in selecting known lung cancer patients for screening, with the ≥ 1.51% risk threshold selecting up to 80% of the study cohort. This risk threshold would increase the number of individuals screened but detect more cancers in our population. The 13.3% of patients who were not selected for screening by any model at the ≥ 1.51% risk threshold had higher BMI, were younger and smoked less.
The choice of a risk threshold is a balancing act between sensitivity and specificity. We recommend a ≥ 1.51% risk threshold, as all three models had approximately 80% sensitivity in our study population. This is concordant with the sensitivity (80.1% [95% CI, 76.8%–83.0%]) reported by Tammemägi et al 12 when applying a ≥ 1.51% threshold to PLCO patient data. As our cohort had already been diagnosed with lung cancer, we could not directly assess specificity of the models. However, Tammemägi et al 12 reported that with a ≥ 1.51% threshold, the PLCOm2012 model would have achieved greater specificity than the original criteria recommended by the United States Preventive Services Task Force (USPSTF) 30 (66.2% [95% CI, 65.7%–66.7%] vs 62.7% [95% CI, 62.2%-63.1%], P < .001), which were similar to the selection criteria of the NLST.
Our modelling analysis using OncoSim demonstrated how the clinical benefits (cancer detected and cancer death avoided), resources required (number of screens) and harms (false positives) vary between CTFPHC guidelines and different risk thresholds of the PLCOall2014 model. Required resources are similar between the ≥ 2.00% threshold and the CTFPHC criteria. Lowering the risk threshold from ≥ 2.00% to ≥ 1.51% would increase the number of individuals eligible for screening and the clinical benefits of lung cancer screening to detect more cancers, with the hope that earlier detection would increase survival and improve quality of life. However, it would also both increase the number of LDCT scans, thereby increasing costs and waiting times for other imaging, and increase the number of false positives that would increase patient anxiety, repeat imaging and unnecessary invasive procedures. Therefore, a cost–benefit analysis is necessary to determine the ideal risk threshold for lung cancer screening in Canada.
At a threshold of ≥ 1.51%, we did not observe a significant difference in the sensitivity of the PLCOm2012, PLCOall2014 and LLPv2 models. Thus, our results do not support the use of one model over another. However, the PLCOm2012 is already being implemented in some regions within Canada, including the screening program in Ontario 31 and in a clinical trial in Alberta. 32 These practical considerations might motivate the use of the PLCO models in Canadian provinces in preference to other risk models.
Notably, all 48 (40%) patients with a diagnosis of COPD were selected for screening by at least one risk model at a ≥ 1.51% threshold. This finding suggests that a diagnosis of COPD might be a particularly specific indicator of whether someone would be selected for screening by these models. Family doctors should become acquainted with the model’s input criteria, such as a diagnosis of COPD, to help them in referring appropriate patients to official screening centres for evaluation of eligibility for screening.
Our study cohort had a similar proportion of stage I (22.5% vs 21%) and II cancers (5.8% vs 8%), with a higher proportion of stage III (32.5% vs 20%) and lower proportion of stage IV cancers (39.2% vs 49%) compared to the national average. 3 Over half of lung cancers in Canada are being diagnosed at late stages, with 3-year survival rates of 5–22% compared to 71% and 49% for stage I and II lung cancers, respectively. 3 As the PanCan study showed that the PLCOm2012 risk prediction model can detect cancers at earlier stages, 13 implementing screening programs with risk prediction models is critical to increasing patient survival rates.
Thus far, lung cancer screening in Canada has mostly constituted clinical trials and pilots. These initiatives have not yet converged on shared selection criteria, and beyond these academic centres and trials, Canadian physicians might be more likely to follow the better-known, simpler CTFPHC guidelines. The CTFPHC initially published their lung cancer screening guidelines in 2016 based on the findings of the NLST, 5 as did many other professional associations. These guidelines have evolved independently over time between different healthcare groups, researchers and countries. In 2021, the USPSTF released expanded recommendations for LDCT lung cancer screening for patients between 50–80 years old who are current smokers or quit < 15 years ago and have a 20+ pack-year smoking history. 30 Considerable variation is apparent in Canadian approaches to screening. In Ontario’s screening pilot, patients had to be between 55–74 years old, a current or former smoker for ≥ 20 years’ duration and had to have a PLCOm2012 risk ≥ 2.00%. 31 The ongoing Alberta Lung Cancer Screening Program seeks to compare the effectiveness of screening between a group of patients who meet NLST criteria and a group of patients with a PLCOm2012 risk ≥ 1.51%, 32 which matches the comparison drawn by Tammemägi et al. 12 Risk prediction models have not been used in participant selection for the ongoing British Columbia Lung Screen Trial, in which eligible participants must be 55–80 years old, be a former or current smoker and have smoked for ≥ 20 years. 33 The same is true of Quebec’s Lung Cancer Screening Demonstration Program, where participants must be 55–74 years old, have smoked for ≥ 20 years duration, and are current smokers or have quit < 15 years ago. 34 Evidently, there is not yet a consensus among Canadian provinces regarding the use of risk prediction models in selecting candidates for lung cancer screening, nor which risk threshold to use. This lack of standardization may contribute to patient selection and quality issues. 8 Our results demonstrate the benefit which can be expected if the risk threshold is lowered from ≥ 2.00% to ≥ 1.51%, all other factors remaining equal.
Limitations of this study include a relatively small sample size, retrospective design and some missing data requiring assumptions. We collected data for patients who had already been diagnosed with lung cancer and thus could not calculate specificity. The data corresponded to the time of diagnosis and so there may be a slight skew of some risk factors such as age and pack-years, though each screening regimen was affected similarly, and so comparative value is retained. The diagnosis of COPD was based on physician records, as opposed to the results of standard pulmonary function tests, which limits the reliability of this important input variable. However, we feel this practice more accurately reflects real-world implementation of screening protocols. In the region of study, ethnic variation was low, and thus, our results have limited applicability for more ethnically diverse Canadian subpopulations and Canada’s indigenous peoples and should be validated in such groups. The PLCOm2012 and PLCOm2014 use an American-centric classification system of race/ethnicity, which presents a practical challenge when implementing these models in Canada, though Pasquinelli et al 35 have developed a risk model with the ethnicity parameter removed. A formal screening program applied to patients of all backgrounds would hopefully collect this key patient data.
Conclusion
This study demonstrates the effective application of risk prediction models in the selection of patients from a Canadian province for lung cancer screening. A risk threshold of ≥ 1.51% is more sensitive compared to a ≥ 2.00% threshold in identifying lung cancers. The PLCOm2012, PLCOall2014 and LLPv2 models were equally sensitive at the ≥ 1.51% risk threshold and were all superior to the CTFPHC guidelines. Patients who are younger, who have limited or no smoking history, or do not have COPD may be missed regardless of the model chosen.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
