Abstract
Background
A well-designed cancer control plan is an important tool for a nation, state, or community to address the burden of cancer. Furthermore, it provides the opportunity to devise and implement measurable objectives. However, there has been little to no assessment of the success rates of such objectives.
Methods
I compared the success rate of objectives between US states’ current plan and most recent past plan to determine the proportion of success in the United States overall. I also tested possible reasons for low success rates.
Results
The mean success rate was 20% for stringent successes (only exact matches between plans) and 28% for loose successes (exact and similar matches between plans). The magnitude of change in percentage between the baseline and target for loose objectives significantly predicted success (P = .0347). Higher change resulted in lower success. However, neither the number of objectives nor the level of overlap significantly predicted success rate. Nor was population size, region, or rural–urban status significantly related to success rate. The most successful states had high proportions of objectives that were measurable and a high number of overlapping objectives.
Conclusion
I found that objective success rates were low for cancer control plans. To improve success rates, I suggest that future cancer control plans ensure each objective has a measurable baseline and realistically attainable target.
Introduction
Cancer control plans help address the impact of cancer on populations. In 1998, the US Centers for Disease Control and Prevention (CDC) established the National Comprehensive Cancer Control Program (NCCCP) to assist US states in creating and implementing such plans. 1 Most plans include an overview of state cancer rates; cover relevant topics such as prevention, screening, treatment, and survivorship; and include goals that are measured by objectives. There are currently 66 coalitions with a cancer control plan, which includes all fifty US states, the District of Columbia, eight tribal organizations, and seven US territories/jurisdictions. 2 As of June 2021, most US states have created at least two versions of a cancer control plan, with plans typically spanning 5 years.
Despite the utility of a successfully implanted plan, there appears to be little research on the effectiveness of state-level cancer control plans. Several articles have addressed general topics, such as the implementation of evidence-based practices or the inclusion of specific material such as genomics, capacity/sustainability, or end-of-life content.3-9 Other articles have focused on general outcomes such as better quantified goals coupled with associated costs of implementation, comparing state-level recommendations to federal recommendations, identifying keys to success, 2 and developing core domains.10-12
The only reports I was able to find on explicitly measuring success rates of cancer control objectives are as follows. The first was a broad-level US review that found 52% and 63.8% of cancer control plans had met at least one established annual objective based on an environmental or system change in years 4 and 5 of the plan. 13 The second was a study that found that breast cancer objectives that had a measurable baseline and target in US state plans for incidence, mortality, and mammogram prevalence were at 2%, 21%, and 79%, respectively. 14 The third was a study that found that South Carolina’s cancer control plan had 33% of objectives that were clear and measurable, 38% that were not met, 38% that were partially met, and 24% that were met fully. 15 Apart from those three articles, I was unable to locate any further research on cancer control objective success rates. More importantly, I were unable to find any research that explored the mechanism behind cancer control objective success.
Such a lack of research on successful objective achievement is a glaring omission. The inclusion of specific, measurable, attainable, realistic, and time-bound (SMART) objectives has been emphasized by several resources.16-18 For its part, the NCCCP’s self-assessment tool for cancer control plans suggests having cancer control objectives: (1) present, (2) clearly labeled, (3) specific, (4) measurable, (5) attainable, (6) result-oriented, and (7) time-phased. 19 Finally, NCCCP’s priorities include (1) emphasizing primary prevention of cancer, (2) promoting early detection and treatment, (3) supporting survivors and caregivers, and (4) promoting cross-cutting priorities. 20 The cross-cutting priority of note is demonstrating outcomes through evaluation. It seems necessary, therefore, to evaluate the success rates of cancer control plan objectives.
Methods
State cancer control plan documents were obtained from the CDC comprehensive cancer control plan publication list (https://ftp.cdc.gov/pub/publications/CANCER/ccc/). If a state did not have at least two plans present on the list, that state’s website was searched for any additional cancer control plan documents. For states with more than two plans, only the current plan and most recent past plan were used. For each state, “Plan Past” is defined as is the most recent past plan, while “Plan Current” is the current plan.
Terminology for Methods.
I generated three plausible reasons for low objective success rate. The first was a large difference between the baseline measurement and target measurement of objectives. An unrealistically high goal (e.g., hit a target of 80% screening rates from a baseline of 40%) is expected to limit success. The second was an excessive number of objectives. Cancer coalitions have limited resources, so putting insufficient effort into too many avenues is also expected to limit success. The third was low linkage between objectives across time. If an objective in an earlier cancer plan is not included in a later cancer plan, success cannot be measured.
To test the first reason—a large difference between the baseline and target—each linked objective was re-coded as a binary variable called success status (success = 1, non-success = 0). Then, the percentage change between the base measurement and the target measurement was calculated. Measurements that were not percentage-based were excluded from the analysis because they could not be properly scaled. For stringent and loose objectives separately, the success status (1 or 0) was modeled as a function of the percentage change with logistic regression using a binary distribution.
To test the second reason—a high number of objectives —success rates for each state’s objectives were transformed to fit a beta distribution by adding or subtracting a marginal value (.001) to ensure the values did not include exactly 0 or 1. The proportion of stringent successful objectives was then modeled as a function of the number of the measurable objectives for Plan Past and the measurable objectives for Plan Current with a generalized linear model using a beta distribution. The same was done for loose successful objectives. The proportions of successful objectives (stringent and loose) were also modeled as a function of the change in measurable objectives between Plan Current and Past.
To test the third reason—low linkage between objectives across plans—stringent success rates were modeled as a function of the number of stringent matched objectives with a generalized linear model using a beta distribution. The same was done for loose success rates and loose-matched objectives.
Finally, to explore if geographic or demographic data affected success rates, state-level urban population percentages for 2010 were obtained from the Iowa Community Indicators Program (https://www.icip.iastate.edu/tables/population/urban-pct-states). States with urban percentages less > 70% were designated as urban, otherwise they were designated as rural. Next, state population size was taken from the 2010 US Census and log-transformed to approximate a normal distribution. After that, states were also categorized as belonging in one of five regions (Northeast, Midwest, West, Southwest, and Southeast). Finally, stringent and loose success rates by state were each modeled as a function of log population size (numerical) with a generalized linear model using a beta distribution. The same was done for region (categorical), urban percentage (numerical), and rural–urban status (binary).
Results
State Summaries of Cancer Plan Objectives. Past Plan Indicates the Most Recent Past Plan if There Were More than One Past Plan.
Number of Successful Objectives by Category.
The mean stringent success rate was 20% (SD = .22). The mean loose success rate was 28% (SD = .23). Success rates varied by state from 0 to 100% (Figure 1). Other than Wyoming, which had a single linked stringent objective and two linked loose objectives, no state had a success rate greater than 50%. Map of success rates (A) Stringent objectives and (B) loose objectives. For both, Alaska has zero successful objectives and Hawaii had missing objectives.
The mean number of total objectives was significantly higher in Plan Past compared to Plan Current (55.06 vs 32.66, t-value = 3.29, P = .0014), while the number of measurable objectives was lower, though not significantly (24.00 vs 30.40). The proportion of measurable objectives (measurable objectives/total objectives) was significantly lower in Plan Past compared to Plan Current (.577 vs .730, t-value = −3.43, P = .0010).
The magnitude of difference between an objective’s baseline and target measurement values did not predict success for stringent objectives. However, it did for loose objectives (F = 4.51, P = .0347). As the magnitude of difference increased, the probability of success decreased (Figure 2). Logistic regression results for loose objectives. Objective success was coded as 1 or 0, and the change in objective was the difference between the baseline measurement and the target measurement. Only objectives with percentage measurements were included in the model.
The proportion of successful objectives was not significantly associated with the number of objectives for Plan Past, the number of objectives for Plan Current, or the change in objectives across plans. Furthermore, neither stringent nor loose proportions of successful objectives were significantly associated with the number of linked objectives. Finally, none of the other potential demographic or geographic factors (log of state population size, region, urban percentage, rural–urban status) was significant in predicting objective success rates.
Discussion
Across states, the mean stringent success rate for cancer control plans was under 25%, while the mean loose success rate was under 30%. Unfortunately, 34% of states either had no comparable objectives across plans or did not have two plans to compare, so not all states could be represented. By using logistic regression, I found that one of my suspected underlying reasons—a large difference between the baseline and target —appeared to be a plausible contributor; loose objectives had lower success rate probabilities with higher differences between the baseline and target. My other suspected reasons—too many objectives, and little overlap between objectives—showed no significant contribution. Furthermore, there did not appear to be a difference in success rate across population size, rural–urban status, or region.
After looking at the results, some general comments can be made on what makes a state successful in its cancer control objectives. For this, I will consider the number of successful objectives rather than the success rates, as the actual number of successful objectives is more useful in analyzing successes when looked at individual states as opposed to the overall average (proportion is a better unifying metric that does not depend on the number of objectives). For example, Wyoming had the highest success rate but it only had one stringent linked objective. Similarly, the next highest states—Rhode Island and Nevada—only had two.
South Dakota had the highest number of stringent successes (n = 12), followed by Iowa (n = 5). Arkansas and New York had 4; Kentucky and Idaho had 3; and Missouri, Kansas, and Alabama had 2, while the rest of the states had 1 or less. Correspondingly, South Dakota had the highest number of linked objectives (n = 34). Its number of total objectives and measurable objectives was the same, both for Plan 1 (n = 48) and Plan 2 (n = 53). Similarly, Iowa had the third-highest number of linked objectives (n = 17). Its number of total objectives and measurable objectives was the same for Plan 1 (n = 67) and similar for Plan 2 (n = 45, 40). Therefore, a successful tactic for cancer control plan creators to use appears to be including clearly measurable objectives that remain consistent across plans.
This study had several caveats. First, while each cancer control plan typically followed a similar format, each was unique and required manual entry and decision making that, despite a consistent and rigorous approach as possible, still required personal judgment. None of the numerical measurements, however, were a matter of subjectivity. Second, because over a third of states either did not have multiple cancer plans, did not have measurable objectives, or did not have linked objectives, I was not able to include them in the analysis. Third, there are many other reasons that an objective may not have been met that I could not address in this analysis. For example, states differ in the amounts of money available to implement objectives, number and type of hospitals, approaches to educating the public, county-level metrics, etc.
Fourth, I only used objectives that matched across Plan Past and Plan Current. There were other measurable objectives in Plan Past that could be theoretically determined as successful or non-successful by using survey data like the Behavioral Risk Factor Surveillance System (BRFSS) or a state-level equivalent. Such data collection went beyond the resources I had available in this study. A future study that considers all such measurements could be used to determine if the low success rates I calculated here are valid for non-linked objectives as well. If the success rates were to be significantly higher using that additional data collection, my conclusions would be limited to success rates for only the objectives that had been carried over between plans. However, I believe that my results should match well with the unobserved set of measurement results as objectives that carry through from Plan Past to Plan Current are presumably of high importance, thus necessitating a concerted effort to reach the target. Conversely, it is certainly possible that non-linked objectives could have higher success rates because once objectives are met, they may be dropped from later plans.
The low success rates I found here was surprising given that many cancer control plan reports stress the need for realistic and measurable objectives with planned evaluation.21-26 While the role of evaluation in cancer control plans is growing, 27 measuring objective success does not appear to be part of standard evaluations. For example, the CDC requires an evaluation plan from all their grantees that includes an indicator of plan objective implementation in data collection methods, but there is no requirement or suggestion on measuring objective success. 28
For future cancer control plan drafters, I have some suggestions based on the results of this study. First, to ensure a successful cancer control plan, I suggest implementing a measurable outcome for every objective. Second, for each objective, I also suggest carefully selecting the target of the objective to be within an achievable level. A modest but realistic target seems a more sensible approach than an impressive but unrealistic target. To support this suggestion, I compared the success rate between ambitious (top 25th percentile difference between baseline and target) and non-ambitious (bottom 25th percentile difference between baseline and target) objectives and found that non-ambitious objectives, compared to ambitious objectives, were twice as successful for loose objectives (23% vs 11%) and three times as successful for stringent objectives (19% vs 8%). The analyses were not statistically significant, but the trend is supportive of such a suggestion.
Third, while the number of objectives did not appear to be related to success rate, selecting fewer objectives may be beneficial. Focusing on fewer, more important objectives may provide better health outcomes that than many, variously important objectives. Indeed, some of this advice has already been suggested. In a study of three cancer control plan revisions, 29 two of the key factors for successful revision were feasibility and prioritization. Finally, in the evaluation portion of the cancer control plan, the objective success rate should be measured.
Conclusion
Much remains to be done on assessing the success of cancer control plan objectives. As noted earlier, survey data could be used in a future study to capture more information on objective success rate statuses. In addition, a future study could test in detail if the general category of the objective (prevention, detection, survivorship, etc.) predicted success rate. In the longer term, many of the current state cancer control plans only run through 2020 or 2021 and are thus due for renewal. In the next few years, as such plans are made public, they would be ideal targets for an updated analysis of objective success rate. The analysis would build on the initial lessons learned in this report to implement a more systematic and standardized process a priori rather than a posteriori. Furthermore, this initial approach could be refined for use in national and regional cancer control plans outside the United States. Collaboration to develop international guidelines for accessing cancer control objective success rates is a worthy goal and would improve the robustness of these findings.
Cancer control is a difficult problem that requires a well-constructed, realistic, and relevant plan. To date, states have made great strides in constructing and implementing such plans. As states continue to update and improve such plans, learning from earlier iterations will help future constructions and demonstrate evidence-based research both in action and design.
Supplemental Material
sj-pdf-1-ccx-10.1177_10732748211041504 – Supplemental Material for Success Rates for the Objectives of US State Cancer Control Plans: A First Look
Supplemental Material, sj-pdf-1-ccx-10.1177_10732748211041504 for Success Rates for the Objectives of US State Cancer Control Plans: A First Look by Mark R. Williamson in Cancer Control
Footnotes
Acknowledgments
The author would like to thank Dr. Gary Schwartz for helpful feedback.
Declaration of Conflicting Interests
The author declares no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Research reported in this publication was supported by the National Institute of General Medical Sciences of the National Institutes of Health under Award Number U54GM128729.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
