Differential impact of test performance characteristics on burden-to-benefit tradeoffs for blood-based colorectal cancer screening: A microsimulation analysis

Abstract

Objectives

To inform the development and evaluation of new blood-based colorectal cancer (CRC) screening tests satisfying minimum United States (US) coverage criteria, we estimated the impact of the different test performance characteristics on long-term testing benefits and burdens.

Methods

A novel CRC-Microsimulation of Adenoma Progression and Screening (CRC-MAPS) model was developed, validated, then used to assess different screening tests for CRC. We compared multiple, hypothetical blood-based CRC screening tests satisfying minimum coverage criteria of 74% CRC sensitivity and 90% specificity, to measure how changes in a test's CRC sensitivity, specificity, and adenoma sensitivity (sizes 1–5 mm, 6–9 mm, ≥10 mm) affect total number of colonoscopies (COL), CRC incidence reduction (IR), CRC mortality reduction (MR), and burden-to-benefit ratios (incremental COLs per percentage-point increase in IR or MR).

Results

A blood test meeting minimum US coverage criteria for performance characteristics resulted in 1576 lifetime COLs per 1000 individuals, 46.7% IR and 59.2% MR compared to no screening. Tests with increased CRC sensitivity of 99% ( + 25%) vs. increased ≥10 mm adenoma sensitivity of 13.6% ( + 3.6%) both yielded the same MR, 62.7%. Test benefits improved the most with increases in all-size adenoma sensitivity, then size-specific adenoma sensitivities, then specificity and CRC sensitivity, while increases in specificity or ≥10 mm adenoma sensitivity resulted in the most favorable burden-to-benefit tradeoffs (ratios <11.5).

Conclusions

Burden-to-benefit ratios for blood-based CRC screening tests differ by performance characteristic, with the most favorable tradeoffs resulting from improvements in specificity and ≥10 mm adenoma sensitivity.

Keywords

adenoma cancer prevention colorectal cancer screening colorectal neoplasm decision analysis

Introduction

Colorectal cancer (CRC) incidence rates have decreased since the mid-1980s, yet CRC is still the second most common cause of cancer deaths in the United States.¹ Colonoscopy remains the gold standard screening method because the procedure couples direct visual examination of the entire colorectum with the ability to resect lesions in real time.^2–4 Non-invasive tests (e.g., stool-, blood-, or urine-based) are appealing to many individuals due to their convenience.^3,5,6 These tests vary widely in their ability to detect adenomas and early-stage CRC^7–11 as well as in their acceptance by patients.^5,6,12,13 Blood-based CRC screening offers the promise of higher adherence compared to other non-invasive methods^14,15 because blood testing is convenient¹⁶ and familiar to most patients.¹⁷ However, the US Food and Drug Administration (FDA) restricted the use of the only approved blood-based test (SEPT9) largely because of its clinical performance.¹¹ In 2021, the US Centers for Medicare & Medicaid Services (CMS) stipulated minimum clinical performance requirements for a blood-based screening test to secure Medicare coverage, in their National Coverage Determination (NCD)¹⁸: 90% specificity (based on fecal immunochemical test (FIT)-DNA performance in patients without any histopathological findings) and 74% CRC sensitivity (based on FIT performance).⁷ CMS also recommended a default screen interval of 3 years, based on FIT-DNA, but provided no guidance for minimum adenoma sensitivity.¹⁸ Higher adenoma detection at colonoscopy has been associated with reduced CRC incidence and mortality.^19–21 To our knowledge, no studies have carefully and systematically evaluated the impact of non-invasive test sensitivity for adenomas, how this varies by type or size of lesion, how it compares to sensitivity for CRC or specificity, and how all of these affect the colonoscopy burden for follow-up of positive results and post-polypectomy surveillance. A deeper understanding of which test performance characteristics yield the most favorable tradeoffs between benefits and burdens for novel blood-based CRC screening tests is important for a broad audience of stakeholders, including (but not limited to) test developers, regulators, payors, policymakers, and healthcare providers.

Simulation modeling offers a means to compare different hypothetical tests when it would be time- and/or cost-prohibitive to do so in randomized controlled trials. In particular, modeling can quantify any number of screening-related outcomes (e.g., number of screening tests, adverse events, and total colonoscopies) and clinical outcomes (e.g., CRC incidence and mortality reduction) across any combination of screening ages, test intervals, time horizons, and performance characteristics. Modeling routinely informs US Preventive Services Task Force recommendations and coverage and reimbursement decisions by CMS and private payers.^22–24 However, previous analyses focusing on existing tests do not provide a clear understanding of the differential impact of each of the test performance characteristics.

Here, a novel CRC microsimulation model was developed, validated, and then used to provide direct insights into the diagnostic performance characteristics to focus on when developing or evaluating new blood-based CRC screening tests. Multiple hypothetical tests meeting or exceeding NCD minimum performance requirements were compared to assess the implications of changes to specificity, adenoma sensitivity by size, and CRC sensitivity of the tests.

While the analyses focused on blood-based screening, they are applicable to any non-invasive screening modality.

Materials and methods

Model description and overview

CRC-MAPS is a semi-Markov state-transition microsimulation model of the adenoma-carcinoma sequence²⁵ implemented in TreeAge Pro (TreeAge Software, LLC, Williamstown, MA, USA). The high-level model structure (Appendix Figure 1) aligns with existing Markov²⁶ and Cancer Intervention and Surveillance Modeling Network (CISNET) models (SimCRC, CRC-SPIN, MISCAN-Colon).²⁷ The model simulates individuals from age 40 years to 100 years, with relevant health status updates in cycles of one month following best-practice recommendations.²⁸ Successive health states distinguished include small (1–5 mm), medium (6–9 mm), or large adenoma (≥10 mm); stage I-IV preclinical (i.e., asymptomatic, screen-detectable) cancer; stage I-IV clinical cancer; and death. The model simulates a single lesion at a time per individual. Adenomas of any size may transition to preclinical cancer, followed by diagnosis after a variable sojourn time. The stage of CRC and the associated rate of survival are determined at diagnosis. Death may occur from non-CRC or CRC causes, whichever comes first. A screening and colonoscopy surveillance overlay component may modify an individual's natural history, primarily through detection and removal of adenomas or early detection of asymptomatic CRC.

Details of the model, including all model parameters, pseudo-code, and source data references are provided in the Supplementary Appendix.

Source data

Age-based adenoma prevalence was calibrated to autopsy studies conducted prior to the wide availability of screening (see Appendix for all model-related references). The largest adenoma size present (i.e., most advanced) by age was calibrated to a large, multisite endoscopy study. Risk of progression to cancer by adenoma size and age of the individual was calibrated to a single-site endoscopy study. Direct estimates of the distribution of sojourn time by Zheng and Rutter (2012) (see Appendix) were used. Symptomatic cancer incidence and American Joint Committee on Cancer (AJCC) stage were based on Surveillance, Epidemiology, and End Results (SEER) registry data from 1975 to 1979, before screening was widely adopted. Death from non-CRC causes was modeled using the Centers for Disease Control and Prevention 2017 period life tables (National Center for Health Statistics). AJCC stage-specific mortality was estimated via parametric hazard models, including age-at-diagnosis as a covariate, generated from SEER data for CRC cases diagnosed from 2000 to 2003 with follow-up through 2015.

Model validation

Validity was evaluated by comparing outcomes to calibration targets (internal validity), other models (cross-model validity), meta-analyses, hold-out datasets, and randomized controlled trial data of non-invasive screening for CRC not used to inform the model (external validity).²⁸

Analyses

Analyses simulated the implementation of hypothetical, blood-based CRC screening tests among previously unscreened individuals free of diagnosed CRC. Unless otherwise specified, individuals were screened from age 45 to 85 at 3-year intervals, as recommended by CMS in the NCD¹⁸ and a recent proposition on the age to start screening.²⁹ Each test met or exceeded the minimum performance characteristics required by the NCD. Assumed adherence for screening, follow-up colonoscopy, and surveillance colonoscopy was 100%. Outcomes were aggregated from age 40 to death, and included CRC cases and deaths, total number of colonoscopies, complications, and the CRC incidence and mortality reductions compared to no screening.

Analysis 1

In this analysis, the first test (“NCD-Minimum”) assumed the minimum performance characteristics specified by the NCD: 74% CRC sensitivity and 90% specificity. Since the NCD did not explicitly specify any minimum adenoma performance, size-specific adenoma sensitivities were assumed to be at the false-positive rate of the test (i.e., all-size adenoma sensitivity of 10%). The second test (“NCD-CRC”) assumed the same performance characteristics as the first but had near-perfect (99%) CRC sensitivity (Table 1). A threshold analysis was performed to determine the increase in ≥10 mm adenoma sensitivity needed for NCD-Minimum to yield a CRC mortality reduction equivalent to NCD-CRC.

Table 1.

Test performance assumptions.

Performancecharacteristic	“NCD-Minimum”test^a	“NCD-CRC”test	“NCD-Adenoma”test	Follow-up and surveillanceCOL^b
Source	NCD 2021¹⁸	NCD 2021¹⁸	Potter 2014³⁰ Lin 2021³¹ D’Auria 2022³²	Zauber 2015³³
Sensitivity 1–5 mm adenomas	10%	10%	15%	75%
Sensitivity 6–9 mm adenomas	10%	10%	20%	85%
Sensitivity ≥10 mm adenomas	10%	10%	30%	95%
Sensitivity for CRC	74%	99%	74%	95%
Specificity	90%	90%	90%	86%

COL: colonoscopy; NCD: national coverage determination.

NCD minimum performance characteristics, with adenoma sensitivity assumed equal to the false positive rate.

Reported per-lesion sensitivity used. Colonoscopy specificity reflects the detection of non-adenomatous polyps, leading to unnecessary polypectomy and a risk of colonoscopy adverse events.

Analysis 2

This analysis examined the tradeoffs in benefits and burdens for a blood test with improved adenoma sensitivity (“NCD-Adenoma”), with modeled values in the range of publicly available estimates for blood-based CRC screening tests: 15% for 1–5 mm adenomas; 20% for 6–9 mm adenomas; and 30% for ≥10 mm adenomas.^30–32 Each test performance characteristic was independently varied around its base value (±5 percentage-point change) to evaluate the relative impact on clinical outcomes. Additionally, all-size adenoma sensitivity was varied jointly from the base value. The incremental number of colonoscopies, and the incidence and mortality reduction per percentage-point increase in each performance characteristic was calculated, as well as the ratio of incremental colonoscopies to incidence or mortality reductions (burden-to-benefit ratio).

Analysis 3

This analysis compares the “NCD-Adenoma” test to “NCD-Minimum” and “NCD-CRC” in an efficient frontier. An efficient frontier was constructed by plotting CRC mortality reduction versus number of colonoscopies for each test, assuming different potential screen intervals (1, 2, 3 years), start ages for screening (45, 50) and end ages for screening (75, 85). Each unique combination was identified as a separate screening strategy on the plot. For reference, screening with SEPT9 (Appendix Table 4) every 1, 2, or 3 years from age 45 to 75 years was also shown. A screening strategy was considered “dominant” if there was no other strategy or combination of strategies that provided more CRC mortality reduction with the same number of colonoscopies. The efficient frontier was represented as a line that connected all dominant strategies. Strategies representing NCD recommendations (45–85, 3 yr interval) were labeled with an asterisk. As the efficient frontier becomes flatter, there are diminishing returns to CRC mortality reduction from each additional colonoscopy.

Sensitivity analysis

Sensitivity analysis was performed for a hypothetical blood test assuming annual screening from age 45 to 75 and performance characteristics equivalent to FIT (Appendix Table 4), since that is a currently recommended strategy,² and benefit and burden tradeoffs may depend on the screening regimen and baseline test performance characteristics.

Results

Model validation

The model's adenoma prevalence, size, progression, and CRC incidence by age were consistent with source data and other models. The mortality reduction from screening matched RCT data. Refer to the Appendix for more detail.

Analysis 1: outcomes associated with test satisfying NCD requirements

Assuming perfect adherence, a blood test meeting minimum NCD performance characteristics resulted in 35.7 CRC cases compared to 66.9 cases without CRC screening, 11.2 versus 27.5 CRC deaths, and 163.1 life-years gained per 1000 individuals (Table 2, Appendix Table 16). The CRC incidence and mortality reductions compared to no screening were 46.7% and 59.2%, respectively. Per 1000 individuals, screening required 8847 blood tests, 899 follow-up colonoscopies, and 658 post-polypectomy surveillance colonoscopies (Table 2, Appendix Table 16). In contrast, a test with near-perfect sensitivity for CRC (NCD-CRC) increased the mortality reduction to 62.7% (Table 2, Figure 1) and life-years gained to 181.1 per 1000 individuals (Appendix Table 16). This same CRC mortality reduction was achieved with the NCD-Minimum test by increasing ≥10 mm adenoma sensitivity from 10.0% to 13.6% (Figure 1).

Figure 1.

Threshold analysis of NCD minimum performance test with improved adenoma versus CRC sensitivity.

Table 2.

Lifetime results for different test strategies among asymptomatic 40-year-olds (per 1000 individuals).

Scenario^a	Blood tests	Total COLs	COL complications	CRC cases	CRC deaths	CRC IR	CRC MR
Natural history	0	67	1.6	66.9	27.5	0.0%	0,0%
NCD-Minimum test	8847	1576	6.7	35.7	11.2	46.7%	59.2%
NCD-CRC test	8846	1576	6.7	35.9	10.3	46.4%	62.7%
NCD-Adenoma test	8372	1957	8.1	18.9	6.0	71.7%	78.1%
Adenoma sensitivity ±5% pts, all sizes (for the NCD-Adenoma test)	8675 to 8133	1728 to 2134	7.1 to 8.7	23.8 to 15.4	7.5 to 4.9	64.5 to 77.0%	72.7 to 82.0%
1–5 mm	8603 to 8186	1786 to 2093	7.3 to 8.6	20.3 to 17.7	6.4 to 5.6	69.6 to 73.5%	76.6 to 79.5%
6–9 mm	8419 to 8333	1922 to 1986	7.9 to 8.1	20.6 to 17.6	6.5 to 5.6	69.2 to 73.7%	76.2 to 79.6%
≥10 mm	8393 to 8355	1938 to 1972	8.0 to 8.1	20.4 to 17.8	6.5 to 5.7	69.5 to 73.5%	76.4 to 79.4%
CRC sensitivity ±5%pts (for the NCD-Adenoma test)	8372 to 8372	1957 to 1957	8.1 to 8.1	18.9 to 18.9	6.1 to 5.9	71.8 to 71.7%	77.7 to 78.5%
Specificity ±5%pts (for the NCD-Adenoma test)	7856 to 8970	2196 to 1674	8.3 to 7.7	19.0 to 18.8	6.0 to 6.0	71.7 to 71.8%	78.1 to 78.2%

COL: colonoscopy; IR: incidence reduction; MR; mortality reduction.

Unless otherwise specified, all non-invasive tests were evaluated from age 45 to 85 with 3-year intervals. For baseline test performance, see Table 1.

Analysis 2: impact of test performance characteristics on screening benefit and burden

A blood test with increased adenoma sensitivity in the range of published estimates (NCD-Adenoma) resulted in CRC incidence and mortality reductions of 71.7% and 78.1%, respectively, compared to no screening, and 214.7 life-years gained per 1000 individuals (Appendix Table 16), while requiring 1957 total colonoscopies per 1000 individuals (Table 2). Relative to this test, improvements in all-size adenoma sensitivity had the largest impact on CRC incidence and mortality (Table 3). For every percentage-point improvement in all-size adenoma sensitivity, the incidence reduction increased by 1.2% and the mortality reduction increased by 0.9%. Improvements were similar among size-specific adenoma sensitivities and were about 1/3^rd the size of the effect for all-size adenoma sensitivity. Changes in test specificity or CRC sensitivity had smaller effects of ≤0.1%. Colonoscopies increased with greater test adenoma sensitivity, decreased with greater specificity (Table 3), and were robust to the level of CRC sensitivity.

Table 3.

Impact of test performance on screening burden and benefit.

	Colonoscopy burdenChange per percentage-point improved test performance		CRC casesChange per percentage-point improved test performance			CRC deathsChange per percentage-point improved test performance			Burden-to-benefitIncremental colonoscopies per percentage-point benefit from improved test performance
Performancecharacteristic	Absolute (/1000)	Relative	Absolute (/1000)	Relative	IR	Absolute (/1000)	Relative	MR	IR	MR
Adenoma sensitivity (all sizes)	40.62	2.1%	−0.84	−4.4%	1.2%	−0.26	−4.3%	0.9%	32.5	43.5
1–5 mm	30.72	1.6%	−0.26	−1.4%	0.4%	−0.08	−1.3%	0.3%	79.0	105.4
6–9 mm	6.39	0.3%	−0.30	−1.6%	0.4%	−0.09	−1.5%	0.3%	14.3	19.0
≥10 mm	3.39	0.2%	−0.26	−1.4%	0.4%	−0.08	−1.3%	0.3%	8.6	11.5
CRC Sensitivity	0.01	0.0%	0.00	0.0%	0.0%	−0.02	−0.4%	0.1%	−1.4	0.1
Specificity	−52.18	−2.7%	−0.01	−0.1%	0.0%	0.00	−0.1%	0.0%	−2921.0	−3584.5

IR: incidence reduction; MR: mortality reduction.

Appendix Figure 5 highlights the burden-to-benefit ratios for changes to each performance characteristic, reflecting the inherent tradeoffs that occur at the base-case values. The CRC mortality reduction burden-to-benefit ratio for all-size adenoma sensitivity was 43.5, and the ratio varied for size-specific adenoma sensitivities (Appendix Figure 5, Table 3). The ratios for ≥10 mm adenomas (11.5) and 6–9 mm adenomas (19.0) were >5× more favorable than the ratio for 1–5 mm adenomas (105.4). Although the ratio for CRC sensitivity was small, indicating an efficient tradeoff, the magnitude of benefit was negligible. The ratio for specificity was −3584.5, due to a substantial decrease in total colonoscopies and slight increase in mortality reduction; the mortality reduction increased because false-positives result in an unnecessary colonoscopy and a 10-year break from screening, during which new lesions may emerge and progress. The rank-order for the impact of performance characteristics on CRC incidence was similar to that for CRC mortality, with CRC incidence reduction burden-to-benefit ratios varying from −2921.0 for specificity up to 79.0 for 1–5 mm adenoma sensitivity.

Analysis 3: efficient frontier for different blood-based testing strategies

Considering all strategies evaluated for the efficient frontier (Figure 2), strategies utilizing the NCD-Adenoma test were most efficient in terms of the required colonoscopies for most levels of mortality reduction. Nearly all strategies utilizing the NCD-Minimum test or NCD-CRC test were dominated, with the latter generally more efficient than the former. The lower-specificity SEPT9 test was also dominated.

Figure 2.

Efficient frontier of burden-to-benefit for non-invasive CRC test strategies

Sensitivity analysis

The sensitivity analysis with annual testing from age 45 to 75 and performance characteristics equivalent to FIT resulted in an identical rank-order and similar magnitude of burden-to-benefit ratios for each performance characteristic (Appendix Table 17).

Discussion

Randomized clinical trials are the gold standard for evidence on comparative effectiveness of different CRC screening tests. However, trial-based evaluation of all possible combinations of test characteristics and aggregated lifetime outcomes is neither feasible nor practical. Simulation models enable such assessments, offering insights into the potential benefits and burdens of tests with different performance characteristics, and informing test development, resource allocation, clinical decision-making, regulatory authorization, and reimbursement decisions. To our knowledge, this study was the first to use methodology similar to the US Preventive Services Task Force, to examine how the performance characteristics of new non-invasive, blood-based CRC screening tests differentially impact key clinical outcomes.

A novel microsimulation model of the adenoma-carcinoma pathway was developed to investigate the impact of different screening strategies on clinical outcomes. The natural history and screening outcomes produced by the CRC-MAPS model were consistent with those from other models³⁴ and a reference randomized controlled trial,^35,36 supporting the validity of the study findings. This analysis of multiple, hypothetical blood-based tests investigated test burden and benefit tradeoffs as a function of test performance characteristics to better understand the impact of these characteristics on clinical outcomes. Specifically, we assessed one test barely meeting the performance requirements stipulated by CMS in the 2021 NCD (NCD-Minimum), one with near-perfect CRC-sensitivity (NCD-CRC), and one emphasizing adenoma sensitivity (NCD-Adenoma), while also independently varying the detailed test performance characteristics.

CRC incidence reduction and mortality reduction were most sensitive to improvements in all-size adenoma sensitivity, compared to CRC sensitivity or specificity, underscoring the benefit of CRC prevention over earlier diagnosis. Restricting the analysis only to clinical benefits (i.e., CRC incidence reduction and mortality reduction) suggested that detecting even the smallest adenomas can be beneficial to patients, since most CRCs were small adenomas at some point (Table 3).³⁷ The approximately 12-fold greater CRC mortality reduction observed for increases in all-size adenoma sensitivity versus CRC sensitivity generally is consistent with findings from other modeling studies focusing only on the impact of CRC sensitivity. For example, Haug et al. reported that increasing the CRC sensitivity of a non-invasive annual screening test from 80% to 100% increased CRC mortality reduction by only one percentage point.³⁸

While this may seem counter-intuitive, a blood-based test with good adenoma sensitivity has less benefit from high CRC sensitivity, by preventing CRCs via the detection and subsequent removal of their precursors (i.e., adenomas) at follow-up colonoscopy, or via post-polypectomy colonoscopy surveillance.³⁹ Because of this, CRC sensitivity may be more important for one-time test takers, especially older individuals with no prior screening history for whom asymptomatic cancer rates may be comparatively high. CRC sensitivity, for early-stages particularly, may also be more important for a blood-based test with poor adenoma sensitivity and high specificity. Nonetheless, given two tests with equivalent CRC sensitivity, the one with better adenoma detection will yield more favorable clinical outcomes. For a blood-based CRC screening test that would be used programmatically, there is considerable utility in the test's ability to detect adenomas and triage individuals to post-polypectomy colonoscopy surveillance.

Although detecting all adenomas effectively reduces the risk of CRC, analyses focused only on benefits without an equally robust evaluation of associated burdens and harms are incomplete and potentially misleading. Burden-to-benefit tradeoffs have important implications for patient care and resource management,^40–42 which is why they play a critical role in medical decision-making and care management.^2,4 While some key burdens and harms to consider include costs, complications from the procedure, patient time, and decrements in quality of life,⁴³ total colonoscopies were used here as a measure to assess test burden. It is a cost-agnostic outcome favored by the US Preventive Services Task Force that reflects the impact and potential costs of screening on patient care and the health system.²⁷

Improvements to all-size adenoma sensitivity and each size-specific adenoma sensitivity resulted in both higher colonoscopy burden and an increase in CRC incidence reduction and mortality reduction; this can be an acceptable tradeoff depending on resource capacity and on patient willingness to complete follow-up and surveillance colonoscopies. The results support optimization of screening tests for ≥10 mm adenomas,^40,44–46 as the gains in incidence reduction and mortality reduction were associated with relatively few additional colonoscopies. Detection of 6–9 mm adenomas had similar gains in CRC incidence reduction and mortality reduction with only moderately more colonoscopies, suggesting a potentially acceptable tradeoff. Improvements to sensitivity for 1–5 mm adenomas resulted in burden-to-benefit ratios that were much less favorable (i.e., >5-fold) than those for ≥10 mm and 6–9 mm adenomas. While an evaluation of benefits alone suggested similar benefits for removing 1–5 mm adenomas versus larger adenomas, an assessment of both burdens and benefits yields a different and more nuanced conclusion that it may be overly burdensome for a blood-based (and more generally a non-invasive) CRC screening test to detect 1–5 mm adenomas, let alone non-adenomatous hyperplastic polyps.⁴⁷ These results are consistent with observations from the Clinical Outcomes Research Initiative consortium showing that 6–9 mm adenomas exhibit advanced histology ∼3× more frequently than 1–5 mm adenomas,⁴⁴ and observations from CT colonography studies.⁴⁰

The CRC-MAPS model also demonstrated a favorable burden-to-benefit ratio for improvements in specificity. As specificity increases, CRC incidence reduction and mortality reduction increase slightly while the total number of colonoscopies decreases. The increase in these benefits occurs because colonoscopies with no findings are followed by a 10-year break from screening, during which an individual has a small chance of developing disease that could otherwise have been detected (i.e., had the individual continued with their screening test had the false positive result not occurred in the first place). The decrease in total colonoscopies occurs because a blood-based screening test with a low false positive rate will result in fewer referrals for unnecessary follow-up colonoscopy. Although an improvement in specificity drastically reduces colonoscopy burden with a small increase to benefit, it also greatly increases the total number of lifetime blood-based tests performed (Table 2), potentially increasing overall costs.

By contrast, the burden-to-benefit ratios of changes to CRC sensitivity were small, and at least for CRC mortality reduction, represents a potentially acceptable tradeoff; however, the magnitude of mortality benefit was negligible. The burden-to-benefit ratios for CRC incidence reduction were negative, as greater CRC sensitivity increases CRC diagnoses. Burden-to-benefit frontier analysis comparing the NCD-Minimum, NCD-CRC, and NCD-Adenoma tests further corroborated the relative importance of adenoma sensitivity versus CRC sensitivity.

Limitations and future research

Several limitations of the model should be noted. First, the CRC-MAPS model simulates the adenoma-carcinoma pathway and does not model serrated lesion progression to CRC. However, this is consistent with other models. A recent report by the CRC CISNET working groups summarized the evidence for modeling the serrated pathway and determined that there was insufficient evidence about prevalence and multiplicity of serrated polyps by age and location, their malignant potential, and the size-based sensitivity of screening tests to accurately model the pathway.²⁷ Second, the CRC-MAPS model does not explicitly simulate subtypes of adenomas by histology. Adenoma size and advanced histology strongly correlate, and the CRC-MAPS model utilizes a similar approach to the CRC CISNET models by using adenoma size as a proxy for advanced histology to simulate adenoma progression. Lastly, the CRC-MAPS model is based on one index lesion (i.e., the largest lesion) and does not model lesion multiplicity (which impacts surveillance interval decisions after polypectomy) or location. However, since this study did not evaluate direct visualization methods, lesion location was less critical. Despite these limitations, comparison of the CRC-MAPS model against results from CISNET models and the Minnesota fecal occult blood test (FOBT) study for existing screening tests supports the validity of the results. Moreover, despite the focus on blood-based tests, the results of this study are applicable to any non-invasive CRC screening modality.

Future modeling studies will explore additional tradeoffs of a hypothetical blood-based CRC screening test under alternative test implementation scenarios such as realistic adherence (initial and longitudinal rates), and alternative outcomes such as life-years gained.^48–50 Further, similar questions exist for multi-cancer early detection tests, and may be addressed using simulation.

Conclusions

This microsimulation study of hypothetical blood-based CRC screening tests provides insights into the diagnostic performance characteristics to focus on when developing or evaluating new noninvasive tests. Our findings demonstrate the need to consider both the burdens and benefits of a test and highlight a test's sensitivity for adenomas ≥ 10 mm in size and its specificity as the features that best balance these tradeoffs. Future studies with the CRC-MAPS model will explore different test implementation scenarios, including real-world programmatic adherence.

Supplemental Material

sj-docx-1-msc-10.1177_09691413231175056 - Supplemental material for Differential impact of test performance characteristics on burden-to-benefit tradeoffs for blood-based colorectal cancer screening: A microsimulation analysis

Supplemental material, sj-docx-1-msc-10.1177_09691413231175056 for Differential impact of test performance characteristics on burden-to-benefit tradeoffs for blood-based colorectal cancer screening: A microsimulation analysis by Andrew Piscitello, Lauren N Carroll, Signe Fransen, Ben Wilson, Tarun Chandra, Reinier Meester and Girish Putcha in Journal of Medical Screening

Footnotes

Acknowledgements

Karen Ventii, PhD, provided medical writing and editorial support.

CRC-MAPS is a trademark of Freenome Holdings, Inc.

Author contributions

Study concept and design: GP, SF, TC, AP

Model development and analysis: AP

Interpretation of data: GP, SF, LC, BW, TC, AP, RM

Drafting of the manuscript: LC, AP, RM

Critical review of the manuscript: GP, SF, BW, TC, LC, AP, RM

Data availability statement

Data produced by simulations are available upon request from reinier.meester@freenome.com.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: RM is employee of Freenome Holdings, Inc. LC, SF, BW, and GP were employed by Freenome Holdings, Inc. at the time of this analysis and hold options in Freenome Holdings, Inc. TC and AP are contractors of Freenome Holdings, Inc and were previously contractors of Exact Sciences, BlueStar Genomics, and Medtronic.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article

ORCID iD

Reinier Meester

Supplemental material

Supplemental material for this article is available online.

References

Siegel

Miller

Fuchs

, et al. Cancer statistics, 2021. CA Cancer J Clin 2021; 71: 7–33.

US Preventive Services Task Force, Davidson

Barry

Mangione

, et al. Screening for colorectal cancer: US preventive services task force recommendation statement. JAMA 2021; 325: 1965–1977.

Wolf

AMD

Fontham

ETH

Church

, et al. Colorectal cancer screening for average-risk adults: 2018 guideline update from the American Cancer Society: ACS colorectal cancer screening guideline. CA-Cancer J Clin 2018; 68: 250–281.

Rex

Boland

Dominitz

, et al. Colorectal cancer screening: recommendations for physicians and patients from the U.S. multi-society task force on colorectal cancer. Am J Gastroenterol 2017; 112: 1016–1030.

Mansfield

Ekwueme

Tangka

FKL

, et al. Colorectal cancer screening: preferences, past behavior, and future intentions. Patient 2018; 11: 599–611.

Shokar

Carlson

Weller

. Informed decision making changes test preferences for colorectal cancer screening in a diverse population. Ann Fam Med 2010; 8: 141–150.

Imperiale

Ransohoff

Itzkowitz

, et al. Multitarget stool DNA testing for colorectal-cancer screening. N Engl J Med 2014; 370: 1287–1297.

Imperiale

Gruber

Stump

, et al. Performance characteristics of fecal immunochemical tests for colorectal cancer and advanced adenomatous polyps: a systematic review and meta-analysis. Ann Intern Med 2019; 170: 319–329.

Deng

Chang

Foshaug

, et al. Development and validation of a high-throughput mass spectrometry based urine metabolomic test for the detection of colonic adenomatous polyps. Metabolites 2017; 7: E32.

10.

Lin

Perdue

Henrikson

, et al. Screening for Colorectal Cancer: An Evidence Update for the U.S. Preventive Services Task Force. Evidence Synthesis No. 202. AHRQ Publication No. 20-05271-EF-1. Rockville, MD: Agency for Healthcare Research and Quality; 2021. http://www.ncbi.nlm.nih.gov/books/NBK570913/

11.

U.S. Food and Drug Administration. FDA Summary of Safety and Effectiveness Data for Epi ProColon. 2016. https://www.accessdata.fda.gov/cdrh_docs/pdf13/P130001B.pdf

12.

Green

BlueSpruce

Tuzzio

, et al. Reasons for never and intermittent completion of colorectal cancer screening after receiving multiple rounds of mailed fecal tests. BMC Public Health 2017; 17: 531. doi:10.1186/s12889-017-4458-6

13.

Lecky

Hawking

McNulty

. Patients’ perspectives on providing a stool sample to their GP: a qualitative study. Brit J Gen Pract 2014; 64: e684–e693.

14.

Taber

Aspinwall

Heichman

, et al. Preferences for blood-based colon cancer screening differ by race/ethnicity. Am J Health Behav 2014; 38: 351–361.

15.

Ioannou

Sutherland

Sussman

, et al. Increasing uptake of colon cancer screening in a medically underserved population with the addition of blood-based testing. BMC Cancer 2021; 21: 966.

16.

Xie

Jiang

, et al. Diagnostic value of methylated Septin9 for colorectal cancer detection. Front Oncol 2018; 8: 247.

17.

Osborne

Flight

Wilson

, et al. The impact of sample type and procedural attributes on relative acceptability of different colorectal cancer screening regimens. Patient Prefer Adher 2018; 12: 1825–1836.

18.

National Coverage Determination (NCD): Colorectal Cancer Screening Tests. 210.3. Centers for Medicare & Medicaid Services. https://www.cms.gov/medicare-coverage-database/view/ncd.aspx?NCDId=281&ver=5

19.

Schottinger

Jensen

Ghai

, et al. Association of physician adenoma detection rates with postcolonoscopy colorectal cancer. JAMA 2022; 327: 2114–2122.

20.

Corley

Jensen

Marks

, et al. Adenoma detection rate and risk of colorectal cancer and death. N Engl J Med 2014; 370: 1298–1306.

21.

Meester

RGS

Doubeni

Lansdorp-Vogelaar

, et al. Variation in adenoma detection rate and the lifetime benefits and cost of colorectal cancer screening: a microsimulation model. JAMA 2015; 313: 2349–2358.

22.

Knudsen

Rutter

Peterse

EFP

, et al. Colorectal cancer screening: an updated modeling study for the US preventive services task force. JAMA 2021; 325: 1998–2011.

23.

Peterse

EFP

Meester

RGS

Siegel

, et al. The impact of the rising colorectal cancer incidence in young adults on the optimal age to start screening: microsimulation analysis I to inform the American Cancer Society colorectal cancer screening guideline. Cancer 2018; 124: 2964–2973.

24.

Meester

RGS

Peterse

EFP

Knudsen

, et al. Optimizing colorectal cancer screening by race and sex: microsimulation analysis II to inform the American cancer society colorectal cancer screening guideline. Cancer 2018; 124: 2974–2985.

25.

Muto

Bussey

Morson

. The evolution of cancer of the colon and rectum. Cancer 1975; 36: 2251–2270.

26.

Silva-Illanes

Espinoza

. Critical analysis of Markov models used for the economic evaluation of colorectal cancer screening: a systematic review. Value Health 2018; 21: 858–873.

27.

Knudsen

Rutter

Peterse

EFP

, et al. Colorectal Cancer Screening: An Updated Decision Analysis for the U.S. Preventive Services Task Force. Technical Report, No. 202s. AHRQ Publication No. 20-05271-EF-2. Rockville, MD: Agency for Healthcare Research and Quality; 2021. http://www.ncbi.nlm.nih.gov/books/NBK570833/

28.

Siebert

Alagoz

Bayoumi

, et al. State-transition modeling: a report of the ISPOR-SMDM modeling good research practices task force–3. Value Health 2012; 15: 812–820.

29.

Proposed Rule 87 FR 45860. Centers for Medicare & Medicaid Services (CMS), Health and Human Services (HHS); 2022. federalregister.gov/d/2022-14562

30.

Potter

Hurban

White

, et al. Validation of a real-time PCR-based qualitative assay for the detection of methylated SEPT9 DNA in human plasma. Clin Chem 2014; 60: 1183–1191.

31.

Lin

Ariazi

Dzamba

, et al. Evaluation of a sensitive blood test for the detection of colorectal advanced adenomas in a prospective cohort using a multiomics approach. Conference poster presented at: ASCO-GI; January 15, 2021. https://www.freenome.com/blood-based-detection-of-advanced-adenomas

32.

D'Auria

Chang

Multhaup

, et al. Validation of a multi-modal blood-based test for the detection of colorectal cancer with sub single molecule sensitivity. Conference poster presented at: ASCO; 2022; Chicago. https://guardanthealth.com/clinical-studies/key-publications/

33.

Zauber

Knudsen

Rutter

, et al. Evaluating the benefits and harms of colorectal cancer screening strategies: a collaborative modeling approach. AHRQ Publication No. 14-05203-EF-2. Rockville, MD: Agency for Healthcare Research and Quality; 2015. https://www.uspreventiveservicestaskforce.org/home/getfilebytoken/4Et7DL3SqpYqVkUxd-VM8K

34.

Knudsen

Zauber

Rutter

, et al. Estimation of benefits, burden, and harms of colorectal cancer screening strategies: modeling study for the US preventive services task force. JAMA 2016; 315: 2595.

35.

Mandel

Bond

Church

, et al. Reducing mortality from colorectal cancer by screening for fecal occult blood. Minnesota Colon Cancer Control Study. N Engl J Med 1993; 328: 1365–1371.

36.

Mandel

Church

Bond

, et al. The effect of fecal occult-blood screening on the incidence of colorectal cancer. N Engl J Med 2000; 343: 1603–1607.

37.

Church

JM.

Clinical significance of small colorectal polyps. Dis Colon Rectum 2004; 47: 481–485.

38.

Haug

Knudsen

Lansdorp-Vogelaar

, et al. Development of new non-invasive tests for colorectal cancer screening: the relevance of information on adenoma detection. Int J Cancer 2015; 136: 2864–2874.

39.

Gupta

Lieberman

Anderson

, et al. Recommendations for follow-up after colonoscopy and polypectomy: a consensus update by the US multi-society task force on colorectal cancer. Gastroenterology 2020; 158: 1131–1153.e5.

40.

Pickhardt

Hassan

Laghi

, et al. Cost-effectiveness of colorectal cancer screening with computed tomography colonography: the impact of not reporting diminutive lesions. Cancer 2007; 109: 2213–2221.

41.

Kothari

Huang

Shaukat

, et al. ASGE review of adverse events in colonoscopy. Gastrointest Endosc 2019; 90: 863–876.e33.

42.

Peterse

EFP

Meester

RGS

de Jonge

, et al. Comparing the cost-effectiveness of innovative colorectal cancer screening tests. J Natl Cancer Inst 2021; 113: 154–161.

43.

Zauber

. Cost-effectiveness of colonoscopy. Gastrointest Endosc Clin N Am 2010; 20: 751–770.

44.

Lieberman

Moravec

Holub

, et al. Polyp size and advanced histology in patients undergoing colonoscopy screening: implications for CT colonography. Gastroenterology 2008; 135: 1100–1105.

45.

Pickhardt

Pooler

Kim

, et al. The natural history of colorectal polyps: overview of predictive static and dynamic features. Gastroenterol Clin North Am 2018; 47: 515–536.

46.

Dubé

Yakubu

McCurdy

, et al. Risk of advanced adenoma, colorectal cancer, and colorectal cancer mortality in people with low-risk adenomas at baseline colonoscopy: a systematic review and meta-analysis. Am J Gastroenterol 2017; 112: 1790–1801.

47.

Meester

RGS

Anderson

. Long-term risk for colorectal cancer in patients with index serrated polyps. Gastroenterology 2022; 162: 2108–2110.

48.

Naber

Knudsen

Zauber

, et al. Cost-effectiveness of a multitarget stool DNA test for colorectal cancer screening of medicare beneficiaries. PLoS ONE 2019; 14: e0220234.

49.

Ladabaum

Mannalithara

. Comparative effectiveness and cost effectiveness of a multitarget stool DNA test to screen for colorectal neoplasia. Gastroenterology 2016; 151: 427–439.e6.

50.

Randel

Schult

Botteri

, et al. Colorectal cancer screening with repeated fecal immunochemical test versus sigmoidoscopy: baseline results from a randomized trial. Gastroenterology 2021; 160: 1085–1096.e5.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.55 MB