Clinical performance and utility: A microsimulation model to inform the design of screening trials for a multi-cancer early detection test

Abstract

Objectives

Designing cancer screening trials for multi-cancer early detection (MCED) tests presents a significant methodology challenge, as natural histories of cell-free DNA-shedding cancers are not yet known. A microsimulation model was developed to project the performance and utility of an MCED test in cancer screening trials.

Methods

Individual natural history of preclinical progression through cancer stages for 23 cancer classes was simulated by a stage-transition model under a broad range of cancer latency parameters. Cancer incidences and stage distributions at clinical presentation in simulated trials were set to match the data from Surveillance, Epidemiology, and End Results program. One or multiple rounds of annual screening using a targeted methylation-based MCED test (Galleri ^Ⓡ ) was conducted to detect preclinical cancers. Mortality benefit of early detection was simulated by a stage-shift model.

Results

In simulated trials, accounting for healthy volunteer effect and varying test sensitivity, positive predictive value in the prevalence screening round reached 48% to 61% in 6 natural history scenarios. After 3 rounds of annual screening, the cumulative proportions of stage I/II cancers increased by approximately 9% to 14%, the incidence of stage IV cancers was reduced by 37% to 46%, the reduction of stages III and IV cancer incidences was 9% to 24%, and the reduction of mortality reached 13% to 16%. Greater reductions of late-stage cancers and cancer mortality were achieved by five rounds of MCED screening.

Conclusions

Simulation results guide trial design and suggest that adding this MCED test to routine screening in the United States may shift cancer detection to earlier stages, and potentially save lives.

Keywords

Cancer screening clinical utility multi-cancer early detection test microsimulation randomized controlled trial trial design

Introduction

In the United States, screening tests are available for just five cancers: colorectal, breast, lung in high-risk individuals, cervical, and prostate cancer on an individual basis.^1–5 For many other cancers, single-organ cancer screening in the general population is not cost-effective because of low incidence and/or lack of an appropriate test. Yet, in 2020, approximately 76% of estimated cancer-related deaths in the US population aged 50–79 were from cancers without population screening programs.⁶

Significant progress has been made recently to develop blood-based, multi-cancer early detection (MCED) tests, with some tests utilizing features of cell-free DNA (cfDNA), such as cancer methylation signatures or fragmentation patterns.^7–10 The performance of these MCED tests has been evaluated in retrospectively identified cancers and controls. For example, one MCED test (Galleri ^Ⓡ ; GRAIL, LLC) was reported to detect cancer signals from more than 50 types of cancer, with a low false-positive rate of 0.5%.⁹ By aggregating detection rates over multiple cancers in a single blood draw, MCED is a promising approach to improve the cost-effectiveness over single-organ cancer screening approaches, potentially shifting the paradigm for cancer early detection and population screening.¹¹

Development of a biomarker for cancer screening is a lengthy endeavor. The Early Detection Research Network has proposed a five-phase pipeline to facilitate the development of screening tools for early cancer detection.¹² In “phases 1–3” studies, a biomarker is developed and evaluated in retrospective studies; “phases 4–5” prospective studies will then be conducted to investigate feasibility, clinical utility, and potential harms in an asymptomatic, intent-to-screen population. Compared to single-cancer screening modalities, diagnostic resolution of an MCED test positive result (“cancer signal detected”) is more challenging due to the complexity of determining the cancer signal origin. Therefore, the goal of a “phase 4” MCED study is to evaluate the feasibility, effectiveness, and safety of diagnostic work-up procedures. Other endpoints include positive predictive value (PPV) in a screening population, characteristics of screen-detected cancers, and impact of negative MCED test results on participants’ adherence to existing cancer screening programs. Examples of “phase 4 studies” include PATHFINDER (NCT04241796) and PATHFINDER 2 (NCT05155605).^13,14

Ultimately, the gold-standard design for evaluating clinical utility of a cancer early detection test is the randomized controlled trial (RCT) with a cancer-specific mortality endpoint (“phase 5” study). However, a cancer-specific mortality endpoint requires a large trial population and many years, often over a decade of follow-up, as observed in previous clinical utility trials for cancer screening.^15–17 In lieu of the long trial duration needed for a cancer-specific mortality endpoint, reduction of late-stage cancers has been proposed as a clinically meaningful surrogate endpoint for an early evaluation of clinical benefit in breast and colorectal cancer screening trials.^18–20 Recently, the proportion of early-stage cancers was used as the primary endpoint for the Early Detection Initiative trial for pancreatic cancer screening.²¹ The NHS-Galleri trial (ISRCTN91431511) was designed to detect a significant reduction in the incidence rate of late-stage (stages III and IV) cancers targeted by Galleri^Ⓡ.²²

The design of an MCED screening trial presents significant methodological challenges because the performance and utility of an MCED test in a screening population depend on the natural history of cfDNA-shedding cancers, which are unknown. Specifically, sojourn time—the length of a cancer's preclinical detectable period—has not been studied for MCED-detectable cancers. Existing knowledge of the natural histories for a few cancers was obtained through single-cancer screening studies, examining the numbers of screen-detected cancers and interval cancers missed by screening.²³ Few studies have reported sojourn times by cancer stage,^24,25 which, as we show in this work, will dictate the degree of stage shift triggered by early detection. While there are mathematical formulas for determining sample size for an RCT with cancer mortality as the primary endpoint,^26–29 it is difficult to determine the timing of mortality analysis: depending on natural history and frequency of screening, mortality reduction typically appears after a delay and eventually dissipates after screening stops.^30,31

In this article, we propose a microsimulation model, coupled with a broad spectrum of preclinical latency scenarios to enhance robustness, for planning a screening study for the Galleri ^Ⓡ test. Microsimulation models dynamically simulate a population of individuals and their potential transitions between health states by using risk factors and determinants to predict health outcomes, with the ability to assess the impact of policy and interventions.^32,33 Repeating simulations of such populations allows for estimation of the potential variation in outcomes due to chance. A simple backward simulation approach was taken to generate preclinical cancer stages. The model simulates one or multiple screening rounds on individual cancer trajectories with stochastic variation, providing insight on effect sizes of screening trial endpoints and directly estimating power. To illustrate clinical utility, we consider two types of screening studies: (1) a single-arm “phase 4” study to evaluate feasibility, performances, and harms of MCED screening; (2) a “phase 5” RCT comparing three or five rounds of annual MCED screening (intervention arm) to no MCED screening (control arm) to evaluate clinical utility.

Methods

Natural history of 23 cancer classes and stage-specific sojourn times

We define 23 stageable cancer classes, with additional refinement into subtypes for breast (hormone receptor HR+, HR−, and HR unknown) and lung (adenocarcinoma, squamous cell carcinoma, small cell, and all other) cancers using ICD-O-3 site and histology code definitions (Supplemental Table S1). Cancer stages were categorized according to the American Joint Committee on Cancer (6th edition) as I, II, III, or IV. Individual cancer progression trajectories through stages were characterized by a state-transition model (Figure 1(a)). The time span a preclinical cancer would stay in a stage before progressing to the next more-advanced stage was defined as stage-specific dwell time.³⁴

Figure 1.

Microsimulation model for simulating cancer natural history and MCED. (a) An illustration of the natural history of a clinically diagnosed stage IV cancer and hypothetical MCED screening based on cfDNA shedding. Note that clinical symptoms can arise at any stage, but are most likely at later stages, particularly stage IV. (b) Six scenarios of dwell time by cancer stage. Cancers are combined into five groups based on SEER survival probabilities (Figure S1 in Supplemental material). The first two scenarios represent faster progression in later stages; the middle two scenarios have a longer dwell time for stage I but equal dwell times for stages II, III, and IV; the last two scenarios have equal dwell times in all four stages. (c) Mechanics of simulating cancer natural history and the MCED screening schedule. The time of clinically diagnosed cancer at a particular stage is first generated from SEER cancer incidence and stage distribution. The times for preclinical cancer stages were determined by back subtracting the hypothesized dwell times illustrated in (b). Survival and mortality were simulated from SEER survival data. Prevalence and incidence screening schedules were subsequently superimposed on the cancer natural history. cfDNA: cell-free DNA; MCED: multi-cancer early detection; SEER: Surveillance, Epidemiology, and End Results.

Because of the lack of knowledge on the natural histories of many cancers, including cfDNA-shedding cancers, we cast a wide net on potentially plausible sojourn time scenarios (Figure 1(b)). In a previous work,³⁴ three sets of hypothetical tumor growth scenarios were constructed to define mean dwell time per cancer type and stage. These scenarios assume shorter mean dwell times in more advanced stages, reflecting the common view of accelerated late-stage cancer growth. In this work, we expand plausible dwell time scenarios, now including the possibility that dwell times may not accelerate in later stages. Figure 1(b) shows graphically the dwell time at each stage for 23 cancer classes (also shown numerically in Supplemental Table S2), clustered into five groups based on their survival probabilities (Supplemental Figure S1). In scenarios 1 and 2, dwell time shortens with the progressing stage, representing the conventional understanding of cancer progression. In scenarios 3 and 4, dwell time decreases from stages I to II, then remains constant for stages II, III, and IV. This pattern of dwell time by cancer stage is motivated by the literature on lung and colorectal cancers.^24,25 Further information on constructing these dwell time scenarios and additional scenarios is included in the Supplemental material (Figure S3 and Table S5). In scenarios 5 and 6, dwell times remain equal across four stages, which may be less plausible but of interest for testing the model's robustness.

Figure 1(c) shows an illustrative example of simulating the natural history of a stage IV cancer at clinical diagnosis. After enrollment, each individual received a time of cancer diagnosis that follows an exponential distribution. The mean rate of the diagnosed cancer for each cancer type and stage was set to be the corresponding cancer incidence in Surveillance, Epidemiology, and End Results (SEER) data for the population aged 50–79 years when diagnosed with invasive primary cancer from 2006 to 2015. The simulation accounted for the healthy volunteer effect commonly observed in cancer screening trials,³⁵ by applying a pre-specified percentage of reduction to the SEER cancer incidence rates. For all simulated trials in this article, we used a 15% reduction of the cancer incidences to model the healthy volunteer effect. This level of healthy volunteer effect was observed in the first 4 years of the Prostate, Lung, Colorectal, and Ovarian Cancer Screening Trial.³⁵ The preclinical natural history was connected to the time of clinical diagnosis in a backward fashion: the time a preclinical cancer dwelled at the stage of clinical diagnosis before clinical detection was drawn randomly from the uniform distribution; then the dwell time of each preclinical cancer stage was sequentially generated from an exponential distribution at each cancer stage (Figure 1(c)). This backward simulation strategy is similar to a microsimulation model for ovarian cancer screening.³⁶ From clinical diagnosis, survival and mortality were simulated for each individual by sampling the empirical cancer type and stage-specific survival probabilities from SEER.

Screening the study population prospectively by the MCED test

As illustrated in Figure 1(a), a preclinical invasive cancer sheds cfDNA into blood, which becomes detectable by the MCED test during the preclinical detectable phase. Screening by MCED test can be a one-time blood draw, or several rounds of annual blood draws to assess the cumulative effect of a screening program. In the microsimulation, a key assumption was that a cancer detectable at a certain stage will also be detectable at a more advanced stage, whereas a cancer that is undetectable at a certain stage is undetectable at all earlier stages, reflecting that cfDNA shedding amount increases by cancer stage.

MCED test sensitivity by cancer type and stage and an overall specificity were estimated using data from the case-control Circulating Cell-free Genome Atlas (CCGA; NCT02889978) study.⁹ The simulation program can apply a prespecified percentage reduction (e.g. 15% reduction for all trials simulated in this article) to the CCGA-derived sensitivity to reflect the potentially decreased sensitivity when detecting preclinical cancers in an asymptomatic population. A 15% to 20% reduction in test sensitivity is a conservative estimate for screening preclinical cancers; although this is pending confirmation by screening trial data. The overall specificity across all cancer types and stages was set to be 99.5%. We did not model the diagnostic journey, and assumed the diagnostic workup for each positive test result either resulted in a definitive resolution of cancer or a false positive.

We used a stage-shift model to attribute improved survival to screen detection at a stage earlier than clinical diagnosis^37–39 for a fraction of participants. Cancer survival time after MCED test detection was added to the original clinical diagnosis time to avoid lead-time bias and the implausible result of worse survival by screening, using the stage at the time of detection by MCED test and the corresponding stage-specific survival probabilities extracted from SEER. We also modeled worse stage-specific survival among MCED-positive participants compared to MCED-negative participants (Supplemental material, Figure S4).⁴⁰

Validation of the modeling approach

Our microsimulation model was validated by simulating lung cancer data from the National Lung Screening Trial (NLST) and comparing it to the published data on lung cancer stages.¹⁵ The estimated stage-specific natural history parameters and sensitivity for low-dose computed tomography (LDCT) were obtained from previous modeling work.²⁵ Using the cancer incidence and stage distribution in the control arm of the NLST, simulated trials yielded stage distribution in the LDCT arm consistent with the reported distribution: simulated stages I–IV proportions were 46%, 8%, 24%, and 23%, while observed stages I–IV proportions were 50%, 7.1%, 21.2%, and 21.7%. The reduction of stage III/IV lung cancers in simulated trials was 18%, close to the observed 20% reduction in NLST. Further details are provided in Supplemental material.

Computation and algorithm implementation

The proposed simulation model was executed by the computationally efficient programming language, Julia, which affords both speed and straightforward parallelization for a prospective screening study with a large sample size.⁴¹ One thousand simulated trials were generated in parallel on an Amazon EC2 M5 Instance. Trial endpoints for hypothetical trials were summarized and details are provided in Supplemental material.

Code/data availability

Requests for code and data should be emailed to jdai@grailbio.com.

Results

Hypothetical phase 4 prospective single-arm study: Evaluating safety and performance of an MCED test in an asymptomatic screening population

When evaluating an MCED test in an asymptomatic US population aged 50 years or older, a test positive result (“cancer signal detected”) would trigger a diagnostic work-up pathway, which could include imaging and invasive biopsy procedures. Therefore, the primary safety endpoints would be the number and the type of invasive procedures performed among false positive participants. Regulatory agencies might require a minimal number of MCED test positive (“cancer signal detected”) results returned to healthcare providers to evaluate the safety of MCED screening, which would have implications for the sample size of a phase 4 trial.

MCED performance metrics in a prospective asymptomatic population would be evaluated as co-primary endpoints, including PPV and cancer detection rate (CDR; the proportion of screened participants who receive a test positive result and are diagnosed with cancer). Other performance metrics, such as sensitivity, specificity, and negative predictive value, could not be directly evaluated because diagnostic workup would only be triggered for MCED test positives, and MCED test negatives could not be determined to be false negatives or true negatives.

Table 1 shows a summary of the key endpoints for the MCED test in a hypothetical prospective screening study with 20,000 participants. There was a prevalence screening round followed by two annual incidence screening rounds. In 1000 simulated datasets and across six different dwell time scenarios, accounting for the healthy volunteer effect and 15% reduced test sensitivity in a screening population, PPV estimates in the prevalence round ranged from 48.3% to 60.6%. The PPV estimates in the incidence rounds were lower (31.7%–34% for the first incidence round and 30.1%–32.7% for the second incidence round), reflecting the lowered rate of cancers after the prevalence round. Similarly, the number of test positives and CDR both decreased after prevalence screening. Across the six dwell time scenarios, we observed that the number of test positives, PPV, and CDR increased with the overall sojourn time, since a longer preclinical latency would increase the chance of a preclinical cancer being detected in a prevalence screen. The relative length of stage-specific dwell time had a minor effect on these endpoints.

Table 1.

Summary of key endpoints and performance metrics for an MCED test in simulated prevalence screening and incidence screening.

Dwell time scenario	Prevalence screen			Incidence screen 1			Incidence screen 2
Dwell time scenario	#Positives(95% CI)	PPV(95% CI)	CDR (per 1000)	#Positives(95% CI)	PPV(95% CI)	CDR (per 1000)	#Positives(95% CI)	PPV(95% CI)	CDR (per 1000)
1	198 (169,225)	0.502 (0.435,0.573)	5.0	134 (113,156)	0.317 (0.242,0.398)	4.6	131 (109,153)	0.301 (0.226,0.375)	4.6
2	246 (213,279)	0.606 (0.626,0.734)	7.5	137 (116,161)	0.339 (0.261,0.419)	7.0	134 (111,159)	0.323 (0.246,0.406)	7.0
3	190 (163,217)	0.484 (0.411,0.558)	4.6	134 (112,157)	0.318 (0.239,0.398)	4.3	131 (108,154)	0.301 (0.292,0.383)	4.3
4	234 (206,269)	0.584 (0.524,0.648)	6.8	138 (116,162)	0.340 (0.257,0.424)	6.4	134 (112,158)	0.325 (0.246,0.406)	6.5
5	190 (163,218)	0.483 (0.414,0.551)	4.6	134 (113,156)	0.318 (0.245,0.404)	4.3	131 (109,155)	0.303 (0.229,0.381)	4.3
6	237 (207,271)	0.587 (0.525,0.652)	7.0	138 (114,161)	0.340 (0.258,0.418)	6.5	135 (113,158)	0.327 (0.246,0.414)	6.5

CDR: cancer detection rate; CI: confidence interval; MCED: multi-cancer early detection; PPV: positive predictive value.

Hypothetical phase 5 randomized controlled screening trial: Evaluating clinical utility of MCED in an asymptomatic population

One hundred thousand participants aged 50–79 were randomly allocated at a 1:1 ratio to the intervention and control arms. The hypothetical sample size of N = 50,000 in each arm was chosen to resemble the sample size in the two active screening arms in UKCTOCS for ovarian cancer screening.¹⁷ Participants in the intervention arm underwent three or five annual blood draws, and MCED tests were performed. MCED test positive results would be returned to participants and healthcare providers for diagnostic confirmation. Participants in the control arm underwent annual blood draws, and blood samples were stored. An annual dropout rate of 6% was assumed.

Using the microsimulation model, and accounting for healthy volunteer effect and a 15% reduced test sensitivity, we investigated the projected effect size for the three clinical utility endpoints: stage shift (proportion of stage I/II cancers), reduction of late-stage (stages III and IV) cancer incidence, and reduction of cancer-specific mortality. In the context of MCED testing, cancer-specific mortality is defined as the mortality caused by any of the cancer classes screened by the MCED test. Other design considerations can also be evaluated, such as whether five rounds of repeated MCED screening could achieve a greater reduction in late-stage cancers and cancer mortality. Another important consideration for a trial designed to assess mortality is to predict the temporal pattern of mortality benefit and determine the follow-up window.

Shifting cancer stage distribution to stage I/II

Table 2 shows the proportions of stage I/II cancers among all cancers that were diagnosed after three or five rounds of annual MCED screenings. In the control arm, the proportion of stage I/II cancers was approximately 62% on average, reflecting the proportion in SEER data. In the intervention arm, sojourn time parameters affected stage shift to some degree, though not substantially. The proportion of stage I/II cancers in the intervention arm ranged from 71% to 76% and 72% to 76% for three and five annual MCED screening rounds, respectively. In dwell time scenario 1, where tumor growth is accelerating in late stages, the absolute increase in proportion of stage I/II cancers between the intervention arm and the control arm reached as high as 12%–13%, a relative increase of 20%. The power to test whether there is a difference in the proportion of stage I/II cancers between the two arms was uniformly greater than 0.99 across all dwell time scenarios because this test is highly efficient, conditional on the number of cancers being diagnosed during the trial follow-up. Additional simulation scenarios to further explore the impact of dwell times on stage shift are described in the Supplemental material (Figure S4).

Table 2.

Cumulative proportion of stages I and II cancers diagnosed after three or five rounds of annual MCED screening.

Dwell time scenario	3 Rounds of annual screening				5 Rounds of annual screening
	Number of stage I/II (proportion in all cancers)		Relative increase in proportion	Power	Number of stage I/II (proportion in all cancers)		Relative increase in proportion	Power
	MCED	Control	Relative increase in proportion	Power	MCED	Control	Relative increase in proportion	Power
1	987 (74%)	733 (62%)	20% (13%, 27%)	>0.99	1482 (75%)	1140 (62%)	20% (15%, 26%)	>0.99
2	1083 (76%)	731 (62%)	22% (15%, 28%)	>0.99	1588 (76%)	1139 (62%)	22% (17%, 28%)	>0.99
3	948 (72%)	731 (62%)	17% (10%, 24%)	>0.99	1436 (73%)	1139 (62%)	18% (12%, 23%)	>0.99
4	1032 (73%)	731 (62%)	18% (12%, 25%)	>0.99	1531 (74%)	1140 (62%)	19% (14%, 24%)	>0.99
5	935 (71%)	730 (62%)	15% (9%, 22%)	>0.99	1423 (72%)	1139 (62%)	17% (11%, 22%)	>0.99
6	1009 (71%)	733 (62%)	15% (9%, 22%)	>0.99	1509 (73%)	1142 (62%)	17% (12%, 23%)	>0.99

MCED: multi-cancer early detection.

Reducing the incidence of late-stage cancers

Table 3 and Supplemental Figure S2 present the comparison of incidence rates of late-stage cancers in the two arms diagnosed cumulatively during the follow-up period (from randomization to 1 year after the final round of MCED screening) for three or five annual screening rounds. Late-stage cancers could either be stage III/IV cancers or stage IV cancers alone. As illustrated in Supplemental Figure S2, the incidence of stage III/IV cancers in the MCED arm decreased as the relative length of stage I/II sojourn time increased (e.g. dwell time scenarios 1–2, 3–4, vs 5–6) because a longer dwell time for stage I/II cancers permits a higher probability for the MCED test to intercept them during their progression to late-stage cancers. On the other hand, for a fixed ratio between dwell times in stage I/II and dwell times in stage III/IV (e.g. dwell time scenario 2 vs 1), the longer dwell time in stage III/IV led to a higher stage III/IV incidence and a smaller reduction in the intervention arm. This composite effect led to a greater reduction of late-stage cancers and greater power to detect the difference between the two arms in dwell time scenario 1 (24% reduction and 97% power), while the relative reduction and the power were the lowest (9% reduction and 29% power) in dwell time scenario 6 for three rounds of screening. When only stage IV (metastatic) cancers were counted as late-stage cancers, three rounds of annual MCED screening resulted in >35% reduction of stage IV incidence, which can be tested with power >99% for all scenarios. Five rounds of annual MCED screening further reduced the incidence of late-stage cancers. For instance, the percentage of reduction for stage III/IV incidence reached >19% for all scenarios after five rounds of screening.

Table 3.

Reduction of late-stage cancer incidence after three or five rounds of annual MCED screening.

Dwell time scenario		3 Rounds of annual screening				5 Rounds of annual screening
		Incidence per 1000 person years		Relative reduction	Power	Incidence per 1000 person years		Relative reduction	Power
		MCED	Control	Relative reduction	Power	MCED	Control	Relative reduction	Power
1	Stage III + IV	2.52	3.31	24% (12%,34%)	0.97	2.40	3.30	27% (19%,36%)	>0.99
1	Stage IV	1.09	1.93	43% (30%,55%)	>0.99	1.07	2.28	53% (44%,60%)	>0.99
2	Stage III + IV	2.61	3.31	21% (9%,32%)	0.91	2.41	3.31	27% (18%,35%)	>0.99
2	Stage IV	1.03	1.93	46% (35%,57%)	>0.99	0.99	2.26	56% (48%,63%)	>0.99
3	Stage III + IV	2.68	3.31	19% (6%,30%)	0.82	2.54	3.30	23% (12%,33%)	0.99
3	Stage IV	1.19	1.93	38% (33%,55%)	>0.99	1.19	2.27	48% (38%,56%)	>0.99
4	Stage III + IV	2.81	3.30	15% (2%,26%)	0.63	2.57	3.31	22% (13%,31%)	0.99
4	Stage IV	1.18	1.93	39% (26%,50%)	> 0.99	1.12	2.27	50% (42%,58%)	>0.99
5	Stage III + IV	2.79	3.31	16% (3%,27%)	0.68	2.60	3.31	21% (12%,30%)	0.99
5	Stage IV	1.19	1.93	38% (25%,50%)	>0.99	1.17	2.27	48% (40%,56%)	>0.99
6	Stage III + IV	3.00	3.31	9% (−3%,21%)	0.29	2.68	3.31	19% (9%,30%)	>0.99
6	Stage IV	1.22	1.93	37% (24%,49%)	>0.99	1.14	2.27	50% (42%,58%)	>0.99

MCED: multi-cancer early detection.

Reducing cancer-specific mortality

Table 4 shows that the relative reduction of cancer-specific mortality reached 13%–16% for all dwell time scenarios at year 5 for three rounds of annual MCED screening, and 16%–19% at year 7 for five rounds of annual MCED screening. Among the six dwell time scenarios, the trend of mortality reduction was consistent with the pattern of reduction in stage III/IV incidence: a greater reduction of mortality was observed when the dwell time in stage III/IV was shorter. The effect sizes of mortality reduction for other years are summarized in Supplemental Tables S3 and S4. Figure 2 shows the temporal pattern (from years 3 to 10) of power to detect a statistically significant reduction in cancer-specific mortality. When there were three rounds of annual MCED screening at years 0, 1, and 2, the power continued to increase after screening had ended, reaching the maximum at year 5 for most of the dwell time scenarios. However, none of the dwell time scenarios had >80% power for detecting mortality reduction. Adding two more rounds of annual MCED screening substantially increased the power, which reached the maximum (>80%) for most dwell time scenarios at year 7.

Figure 2.

The temporal trend of the power to detect a reduction of cancer-specific mortality in the intervention arm with three or five rounds of annual MCED screening for six dwell time scenarios. Randomization occurred in year 0; for three rounds, annual MCED screening occurred in years 0, 1, and 2; for five rounds, annual MCED screening occurred in years 0, 1, 2, 3, and 4. MCED: multi-cancer early detection.

Table 4.

Reduction of cancer-specific mortality after three or five rounds of annual MCED screening.

Dwell time scenario	3 Rounds of annual screening			5 Rounds of annual screening
	5-year cancer mortality per 1000 person years		Relative reduction	7-year cancer mortality per 1000 person years		Relative reduction
	Control	MCED	Relative reduction	Control	MCED	Relative reduction
1	1.60	1.36	15% (2%,27%)	1.60	1.32	18% (5%,29%)
2		1.34	16% (3%,27%)		1.30	19% (6%,30%)
3		1.39	13% (0%,26%)		1.34	16% (3%,28%)
4		1.37	15% (3%,26%)		1.33	17% (5%,28%)
5		1.39	13% (0%,25%)		1.34	16% (2%,27%)
6		1.38	14% (2%,25%)		1.34	16% (5%,27%)

MCED: multi-cancer early detection.

Discussion

We developed a novel microsimulation model for projecting effect sizes of key study endpoints and informing the design of cancer screening studies for an MCED test (Galleri ^Ⓡ ). Several modeling approaches have been developed to project downstaging by MCED, including the interception model motivating this work.^34,42 To our knowledge, this is the first microsimulation model developed for designing MCED screening studies. The model is flexible for single-arm studies or RCTs, as well as one or multiple rounds of annual MCED screening, accounting for individual-level stochastic variations in cancer incidence, stage distribution, and test sensitivity. Validated with data from the NLST, the backward simulation approach provided a simple, yet flexible way to model MCED of over 20 cancer classes.

When compared to the prior mathematical interception model,³⁴ the structure and assumptions of this microsimulation model remain consistent. Several differences in the features of the two modeling approaches led to different projected reductions in late-stage cancers and cancer mortality. First, a broader range of dwell time scenarios were considered in this work, varying both the total sojourn time and the dwell time in preclinical cancer stages. Second, the simulation in this work also considered reduced sensitivity in a screening population (a 15% reduction) when compared to the sensitivity estimated from the CCGA3 case-control study, and applied a crude healthy volunteer effect that reduced the SEER cancer incidences by 15%. We expect that this microsimulation model will be improved as we learn more about natural histories for cfDNA-shedding cancers from ongoing MCED screening studies (e.g. PATHFINDER2 and NHS-Galleri).

In a hypothetical phase 4 single-arm screening study, the PPV estimates in simulated datasets ranged from 48% to 61% in the prevalence screening round, and 30% to 34% in the incidence rounds (Table 1). This is much higher than the typical <10% PPV estimates of existing single-cancer screening tests^43–45 because of the low false positive rate of the MCED test and aggregated detection of many cancer types in one blood test.

We observed that the relative distribution of stage-specific dwell times had little impact on the number of test positives and various performance metrics, appearing to mainly affect the relative reduction of late-stage cancer incidence (Tables 2 and 3; Supplemental Figure S2). The reduction of stage III/IV incidence appears to be more affected by variation of late-stage dwell times, particularly in the scenario with equal dwell times across four stages (16% and 9% in dwell time scenarios 5 and 6). Compared to three rounds of annual MCED screening, five rounds considerably increased the reduction of stage III/IV cancer incidence (>19% in all scenarios). Compared to stage III/IV cancers, the power and the effect size for the reduction of stage IV cancers (>35%) were consistently higher across all scenarios. This is because stage IV is the end stage before clinical symptomatic diagnosis, when the MCED test has the highest sensitivity and, therefore, is more likely to be shifted to an earlier stage, including stage III.

The choice of stage shift or reduction of late-stage cancer incidence as the primary endpoint of a clinical utility trial has been supported but remains a topic of debate. The proportion of early-stage cancers is subject to the issue of overdiagnosis,^46,47 that is, screen-detected preclinical early-stage cancers that would otherwise neither progress nor be diagnosed clinically in the absence of screening. This phenomenon has been frequently observed after implementing a screening program, such as those for breast and prostate cancers.^46,47 In contrast, meta-analyses of breast cancer screening trials consistently showed a strong correlation between a larger degree of reduction of late-stage cancer incidence and a greater reduction of cancer mortality.^19,20

The degree of reduction in cancer mortality in a screening trial depends on the natural history of the cancer, test sensitivity, and the delivery of cancer care and treatment in the particular clinical trial setting for the enrolled sites. It is therefore more difficult to predict mortality benefit for a clinical trial via a microsimulation model. The stage-shift model we used to predict mortality reduction makes certain assumptions, for example, that screen-detected cancers have the same survival probabilities as the clinically detected cancers at the same stage, which we relaxed in extended models provided in the Supplemental material. In addition, the model adds prolonged survival from the time of clinical detection in the absence of screening to avoid lead-time bias, which can be arbitrary. Moreover, the cancer incidence and survival probabilities used in the model are based on SEER data from the general US population from 2005 to 2016, which may not exactly match cancer incidence and survival during a long follow-up window for a future RCT. As such, results in Table 4 and Figure 2 should be interpreted within this context.

There are limitations in the current version of the model. Aimed to inform study design, our model is relatively simple compared to single-cancer CISNET models for informing screening policies and recommendations,³³ without considering age, gender, and other risk factors. The extent of overdiagnosed cancers detected by cfDNA-based MCED tests has not been systematically studied and, therefore, was not explicitly modeled in this work. For this reason, we focused on the reduction in late-stage cancers in Table 3. The presence of overdiagnosis may increase the number of stage I/II cancers and increase stage shift. Several lines of evidence suggest that cfDNA-based MCED testing may be less prone to overdiagnosis of slow-growing cancers. This MCED test has lower sensitivity for early-stage breast and prostate cancers detected by standard-of-care screening, so it appears to not detect these often overdiagnosed, possibly indolent cancers. Furthermore, more rapidly growing and aggressive tumors tend to shed more cfDNA, and are therefore more likely to be detected by cfDNA-based MCED screening tests.^48–50 In addition, stage-shift models for attributing mortality benefit make strong assumptions and may not capture the entire path from cancer diagnosis to mortality.³⁸ Finally, diagnostic resolution depends on the capacity of healthcare providers and may not be achieved for every screen-detected cancer. Future work should include refinements of the microsimulation model to address these limitations.

Conclusion

The proposed microsimulation model provides a flexible and robust tool to inform the design of screening studies for an MCED test. The simulation results showed that, with a small number of rounds of annual screening, the blood-based MCED test (Galleri ^Ⓡ ) may significantly improve PPV over single-cancer screening tests, shift cancer detection to earlier stages, and potentially save lives.

Supplemental Material

sj-docx-1-msc-10.1177_09691413241228041 - Supplemental material for Clinical performance and utility: A microsimulation model to inform the design of screening trials for a multi-cancer early detection test

Supplemental material, sj-docx-1-msc-10.1177_09691413241228041 for Clinical performance and utility: A microsimulation model to inform the design of screening trials for a multi-cancer early detection test by James Y Dai, Jing Zhang, Jerome V Braun, Noah Simon, Earl Hubbell and Nan Zhang in Journal of Medical Screening

Footnotes

Acknowledgements

We thank Christina A Clarke (GRAIL, LLC) for generating the SEER incidence and survival statistics. The authors thank Amy Sinor-Anderson, Rachel McMullan, and Randall Janairo (GRAIL, LLC) for help with manuscript development. Project management assistance was provided by Prescott Medical Communications Group (Chicago, IL, USA).

Author Contributions

JYD, JZ, and NZ conceived the study. All authors planned the study. JYD, JZ, JVB, EH, and NZ performed the analysis and interpretation of the data. JYD, JZ, and NZ drafted the manuscript and all authors participated in the critical review and editing of the work, as well as approval of the final version. JYD is the guarantor. The corresponding authors attest that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

The Corresponding authors (JYD and NZ) affirm that the manuscript is an honest, accurate, and transparent account of the study being reported, and that no important aspects of the study have been omitted.

Declaration of conflicting interests

JYD, EH, and NZ are employees of GRAIL, LLC. JZ is a former employee of GRAIL, LLC and is a current employee of Adela, Inc. JVB is an independent statistical consultant at GRAIL, LLC. NS has received consulting fees from GRAIL, LLC.

Funding

The authors disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This project was funded by GRAIL, LLC.

ORCID iDs

James Y Dai

Earl Hubbell

Supplemental material

Supplemental material for this article is available online.

References

Curry

Krist

Owens

, et al. Screening for cervical cancer: U.S. preventive services task force recommendation statement. JAMA 2018; 320: 674–686.

US Preventive Services Task Force . Screening for colorectal cancer: U.S. preventive services task force recommendation statement. JAMA 2016; 315: 2564–2575.

US Preventive Services Task Force . Screening for prostate cancer: U.S. preventive services task force recommendation statement. JAMA 2018; 319: 1901–1913.

US Preventive Services Task Force , Krist

Davidson

, et al. Screening for lung cancer: U.S. preventive services task force recommendation statement. JAMA 2021; 325: 962.

Siu

. Screening for breast cancer: U.S. preventive services task force recommendation statement. Ann Intern Med 2016; 164: 279–296.

Hackshaw

Cohen

Reichert

, et al. Estimating the population health impact of a multi-cancer early detection genomic blood test to complement existing screening in the US and UK. Br J Cancer 2021; 125: 1432–1442.

Cristiano

Leal

Phallen

, et al. Genome-wide cell-free DNA fragmentation in patients with cancer. Nature 2019; 570: 385–389.

Liu

Oxnard

Klein

, et al. Sensitive and specific multi-cancer detection and localization using methylation signatures in cell-free DNA. Ann Oncol 2020; 31: 745–759.

Klein

Richards

Cohn

, et al. Clinical validation of a targeted methylation-based multi-cancer early detection test using an independent validation set. Ann Oncol 2021; 32: 1167–1177.

10.

Cohen

Javed

Thoburn

, et al. Combined circulating tumor DNA and protein biomarker-based liquid biopsy for the earlier detection of pancreatic cancers. Proc Natl Acad Sci 2017; 114: 10202–10207.

11.

Liu

. Transforming the landscape of early cancer detection using blood tests—commentary on current methodologies and future prospects. Br J Cancer 2021; 124: 1475–1477.

12.

Pepe

Etzioni

Feng

, et al. Phases of biomarker development for early detection of cancer. JNCI J Natl Cancer Inst 2001; 93: 1054–1061.

13.

Beer

McDonnell

Nadauld

, et al. A prespecified interim analysis of the PATHFINDER study: performance of a multicancer early detection test in support of clinical implementation. J Clin Oncol 2021; 39: 3070–3070.

14.

GRAIL, LLC . The PATHFINDER 2 Study: Evaluating the Safety and Performance of the GRAIL Multi-Cancer Early Detection Test in an Eligible Screening Population. Clinical Trial Registration NCT05155605, clinicaltrials.gov, https://clinicaltrials.gov/ct2/show/NCT05155605 (11 January 2022, accessed 27 February 2022).

15.

National Lung Screening Trial Research Team , Aberle

Adams

, et al. Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med 2011; 365: 395–409.

16.

de Koning

van der Aalst

de Jong

, et al. Reduced lung-cancer mortality with volume CT screening in a randomized trial. N Engl J Med 2020; 382: 503–513.

17.

Menon

Gentry-Maharaj

Burnell

, et al. Ovarian cancer population screening and mortality after long-term follow-up in the UK Collaborative Trial of Ovarian Cancer Screening (UKCTOCS): a randomised controlled trial. Lancet 2021; 397: 2182–2193.

18.

Cuzick

Cafferty

Edwards

, et al. Surrogate endpoints for cancer screening trials: general principles and an illustration using the UK Flexible Sigmoidoscopy Screening Trial. J Med Screen 2007; 14: 178–185.

19.

Tabár

Yen

, et al. Insights from the breast cancer screening trials: how screening affects the natural history of breast cancer and implications for evaluating service screening programs. Breast J 2015; 21: 13–20.

20.

Autier

Héry

Haukka

, et al. Advanced breast cancer and breast cancer mortality in randomized controlled trials on mammography screening. J Clin Oncol 2009; 27: 5919–5923.

21.

Chari

Maitra

Matrisian

, et al. Early detection initiative: a randomized controlled trial of algorithm-based screening in patients with new onset hyperglycemia and diabetes for early detection of pancreatic ductal adenocarcinoma. Contemp Clin Trials 2022; 113: 106659.

22.

Neal

Johnson

Clarke

, et al. Cell-free DNA-based multi-cancer early detection test in an asymptomatic screening population (NHS-Galleri): design of a pragmatic, prospective randomised controlled trial. Cancers (Basel) 2022; 14: 4818.

23.

Walter

Day

. Estimation of the duration of a pre-clinical disease state using screening data. Am J Epidemiol 1983; 118: 865–886.

24.

Loeve

Brown

Boer

, et al. Endoscopic colorectal cancer screening: a cost-saving analysis. JNCI J Natl Cancer Inst 2000; 92: 557–563.

25.

ten Haaf

van Rosmalen

de Koning

. Lung cancer detectability by test, histology, stage, and gender: estimates from the NLST and the PLCO trials. Cancer Epidemiol Biomarkers Prev 2015; 24: 154–161.

26.

Van Iersel

De Koning

Draisma

, et al. Risk-based selection from the general population in a screening trial: selection criteria, recruitment and power for the Dutch-Belgian randomised lung cancer multi-slice CT screening trial (NELSON). Int J Cancer 2007; 120: 868–874.

27.

Hackshaw

Berg

. An efficient randomised trial design for multi-cancer screening blood tests: nested enhanced mortality outcomes of screening trial. Lancet Oncol 2021; 22: 1360–1362.

28.

Moss

Draper

Hardcastle

, et al. Calculation of sample size in trials of screening for early diagnosis of disease. Int J Epidemiol 1987; 16: 104–110.

29.

Prorok

Marcus

. Cancer screening trials: nuts and bolts. Semin Oncol 2010; 37: 216–223.

30.

Hanley

. Measuring mortality reductions in cancer screening trials. Epidemiol Rev 2011; 33: 36–45.

31.

Liu

Hanley

Saarela

, et al. A conditional approach to measure mortality reductions due to cancer screening: measuring mortality reductions due to screening. Int Stat Rev 2015; 83: 493–510.

32.

Çağlayan

Terawaki

Chen

, et al. Microsimulation modeling in oncology. JCO Clin Cancer Inform 2018; 2: 1–11.

33.

Cancer Intervention and Surveillance Modeling Network (CISNET) . https://cisnet.cancer.gov/ (accessed 11 December 2023).

34.

Hubbell

Clarke

Aravanis

, et al. Modeled reductions in late-stage cancer with a multi-cancer early detection test. Cancer Epidemiol Biomarkers Prev 2021; 30: 460–468.

35.

Pinsky

Miller

Kramer

, et al. Evidence of a healthy volunteer effect in the prostate, lung, colorectal, and ovarian cancer screening trial. Am J Epidemiol 2007; 165: 874–881.

36.

Skates

Singer

. Quantifying the potential benefit of CA 125 screening for ovarian cancer. J Clin Epidemiol 1991; 44: 365–380.

37.

Connor

Chu

Smart

. Stage-shift cancer screening model. J Clin Epidemiol 1989; 42: 1083–1095.

38.

Wever

Draisma

Heijnsdijk

EAM

, et al. How does early detection by screening affect disease progression? Modeling estimated benefits in prostate cancer screening. Med Decis Making 2011; 31: 550–558.

39.

Birnbaum

Gadi

Markowitz

, et al. The effect of treatment advances on the mortality results of breast cancer screening trials: a microsimulation model. Ann Intern Med 2016; 164: 236–243.

40.

Sasieni

Smittenaar

Hubbell

, et al. Modelled mortality benefits of multi-cancer early detection screening in England. Br J Cancer 2023; 129: 72–80.

41.

Julia: A Fresh Approach to Numerical Computing . SIAM Review, https://epubs.siam.org/doi/10.1137/141000671 (accessed 11 December 2023).

42.

Lange

Gogebakan

Gulati

, et al. A general two-stage progressive model of cancer natural history to project downstaging due to multi-cancer screening tests. Epub ahead of print 30 June 2023. DOI: 10.48550/arXiv.2307.00092.

43.

Lehman

Arao

Sprague

, et al. National performance benchmarks for modern screening digital mammography: update from the Breast Cancer Surveillance Consortium. Radiology 2017; 283: 49–58.

44.

Pinsky

Gierada

Black

, et al. Performance of lung-RADS in the National Lung Screening Trial: a retrospective assessment. Ann Intern Med 2015; 162: 485–491.

45.

Imperiale

Ransohoff

Itzkowitz

, et al. Multitarget stool DNA testing for colorectal-cancer screening. N Engl J Med 2014; 370: 1287–1297.

46.

Srivastava

Koay

Borowsky

, et al. Cancer overdiagnosis: a biological challenge and clinical dilemma. Nat Rev Cancer 2019; 19: 349–358.

47.

Welch

Black

. Overdiagnosis in cancer. JNCI J Natl Cancer Inst 2010; 102: 605–613.

48.

Chen

Dong

Hubbell

, et al. Prognostic significance of blood-based multi-cancer detection in plasma cell-free DNA. Clin Cancer Res 2021; 27: 4221–4229.

49.

Bredno

Lipson

Venn

, et al. Clinical correlates of circulating cell-free DNA tumor fraction. PLoS ONE 2021; 16: e0256436.

50.

Abbosh

Frankell

Harrison

, et al. Tracking early lung cancer metastatic dissemination in TRACERx using ctDNA. Nature 2023; 616: 553–562.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.13 MB