Sage Journals: Discover world-class research

Abstract

Over 10 million uninsured individuals are eligible for subsidized health insurance coverage through the Affordable Care Act (ACA) marketplaces, and millions more were projected to become eligible with the end of the federal COVID-19 Public Health Emergency in 2023. Individual studies on behaviorally informed interventions designed to encourage enrollment suggest that some are more effective than others. This study summarizes evidence on the efficacy of these interventions and suggests which administrative burdens might be most relevant for potential enrollees. Published and unpublished studies were identified through a systematic review of studies assessing the impact of behaviorally informed interventions on ACA marketplace enrollment from 2014 to 2022.

Thirty-four studies comprising over 18 million participants were included (32 randomized controlled trials and 2 quasiexperimental studies). At the time of data extraction, 8 were published. Twenty-seven of the studies qualified for inclusion in a meta-analysis, which found that the average rate of enrollment was about 1 percentage point higher for those who received an intervention (0.009, P < 0.001), a 24% increase relative to control households; for every 1000 people who receive an intervention, that would correspond to about 9 additional enrollees. When stratifying by intervention intensity, support-based interventions increased enrollment by 2 percentage points (0.020, P = 0.004), while information-based interventions increased enrollment by 0.6 percentage points (0.006, P < 0.001).

The meta-analysis found that behaviorally informed interventions can increase ACA marketplace enrollment. Interventions aimed at alleviating compliance costs by providing enrollment support were about three times as effective as information alone.

Introduction

The passage of the American Rescue Plan Act in 2021 and Inflation Reduction Act in 2022 dramatically expanded the Patient Protection and Affordable Care Act’s (ACA) subsidies through the end of 2025. These policies have contributed to record-high marketplace enrollment and record-low uninsured rates.¹ Despite these coverage gains, an estimated 10.9 million individuals eligible for subsidized health insurance remain uninsured, likely due in part to a lack of awareness of low-cost coverage options available through the marketplaces.² In addition, an estimated 18 million individuals are projected to lose Medicaid coverage with the end of the federal COVID-19 Public Health Emergency in 2023 and will be at risk of gaps in coverage.³

Given the dual challenges of addressing enrollment frictions for the remaining uninsured and for those transitioning from Medicaid, state and federal marketplace administrators need to understand what strategies are effective at preserving—and expanding upon—the coverage gains in recent years. This systematic review and meta-analysis evaluate and synthesize the evidence from published and unpublished studies on the effects of behaviorally informed interventions on ACA marketplace enrollment.

Methods

The authors conducted a systematic review and meta-analysis, following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.⁴

Search strategy and data sources

The study selection protocol is detailed in Supplementary Appendix SA1. It involved English language searches of three databases in September 2022 (PubMed, CINAHL, and Google Scholar) and four registries from September to November 2022 (PROSPERO systematic review registry, Open Science Framework, the American Economic Association Randomized Controlled Trials registry, and ClinicalTrials.gov). The following search terms were used: (“The ACA” OR “The Affordable Care Act” OR Obamacare OR “The Patient Protection and Affordable Care Act”) AND (enrollment OR “take up” OR uptake OR takeup OR “take-up”) AND “health insurance” AND (intervention OR increas* OR strateg* OR nudg* OR impact OR evidence).

The search strategy also included efforts to mitigate potential publication bias and to reflect the understanding that practitioners may conduct studies for internal use without publishing the findings. Specifically, owing to issues such as p-hacking, reviewing only published work on ACA interventions may lead us to draw misleading conclusions.⁵ To identify gray literature and unpublished studies, the authors hand-searched reference lists and conducted outreach to researchers and practitioners. They met with staff from four US federal government organizations and four nongovernmental research organizations and placed a call for studies in a relevant organization’s newsletter and reached out to State-based marketplaces and All-payer Claims Databases in 6 states; the authors heard from all 6 and met with 4. In particular, one author (A.F.) extracted data from unpublished randomized controlled trial studies by Covered California, California’s ACA marketplace. In addition, the authors discussed the project with representatives from two private health insurance brokers and with three teams of academic researchers.

Study selection

For the systematic review, selection criteria required studies to be credibly causal⁶ empirical evaluations—randomized controlled trials or quasiexperimental studies—of interventions designed to increase ACA marketplace enrollment. This means studies focused on coverage for children or elderly adults through the Children’s Health Insurance Program or Medicare were excluded. Only studies focused on marketplace enrollment were included, setting aside research related to continuity of care or efforts to support people with finding an optimal plan. Studies that were conducted after January 2014, when most major provisions of the ACA were in force, were included.

One author (L.M.) screened the identified studies, removed duplicates, and reviewed the title and abstract to assess whether each met the inclusion criteria, in consultation with other authors. If a study’s abstract met the criteria, the full paper was reviewed. To validate these decisions, two authors (A.K.C. and A.F.) coded a random sample of 20 studies each, with full concurrence on final inclusion decisions. A.F. screened 20 unpublished studies by Covered California to identify which met the eligibility criteria.

To qualify for inclusion in the meta-analysis, studies identified in the systematic review: (1) compared marketplace enrollment outcomes between a treatment group and a group that did not receive any intervention as a result of the study; (2) measured both treatments and outcomes at the individual/household level, such that all of the constituent studies have similar outcomes (a binary measure of ACA enrollment); and (3) reported a sample size, an effect size, and a measure of variance, either directly or by providing enough information that they could be calculated.

One author (L.M.) screened studies identified in the systematic review for inclusion and created study covariates related to intervention types (eg, information-only treatments vs. treatments that included a support component or personalized vs. generic interventions), in consultation with other authors (A.F., E.S., and A.K.C.). An independent reanalyst validated this process. This narrowed the list of 34 studies identified in the systematic review to 27 that were eligible for inclusion in the meta-analysis.

Data extraction and synthesis

For each eligible study, information was extracted regarding the study design and methods, population of interest, intervention type, outcomes, and variance of measured effects. The full list of variables the team attempted to extract for each study is included in the appendix. To identify study-level risk of bias, studies were coded using the JAMA Quality Rating Scheme for Studies and Other Evidence, which rates studies from 1 (properly powered and conducted randomized clinical trial) to 5 (opinion of respected authorities; case reports).⁷

Studies identified in the systematic review tested interventions aimed at people looking for health insurance through state-based marketplaces. The authors coded these interventions as falling into two categories: information-based interventions and interventions that provided some enrollment assistance. These categories align with two types of administrative burdens that might affect prospective enrollees: learning costs (eg, understanding eligibility, benefits of enrollment, and the steps involved) and compliance costs (eg, navigating a marketplace, submitting an application, and completing the enrollment process).⁸ Information-only interventions could help address learning costs; interventions that involve enrollment support might alleviate compliance costs as well. For many consumers, navigating the many choices offered on the marketplaces is not only confusing, but overwhelming.⁹ Among these consumers, it is possible that support-based interventions also address a third category of administrative burdens, psychological costs.⁸

To explore effect size heterogeneity across subgroups of studies in the meta-analysis, the authors conducted analyses to see if findings differed by study context, including location and timing relative to the January 31, 2020, federal determination of the COVID-19 Public Health Emergency; characteristics of the intervention, including whether it addressed learning or compliance costs, and personalization of content; and whether study participants had previously interacted with the marketplace or not. Intervention cost was closely aligned with whether the intervention included a support component, and there was no variation in study quality among the studies that qualified for the meta-analysis, so it was not possible to test for heterogeneity of effects across these categories.

Statistical Analysis

A random effects meta-analysis model was used to estimate an average pooled treatment effect across outreach interventions.^10,11 Weighting effect sizes by the inverse of their variance ensured that more precise effect sizes contributed relatively more to estimates.¹² The authors accounted for within-study statistical dependence using a correlated and hierarchical effects (CHE) approach.¹³ Note that because all effect sizes in the sample are based on similar comparisons between a randomized outreach intervention and a true control group, and because the inclusion criteria required studies to be closely aligned in terms of outcome measurement—a household-level, binary indicator of ACA enrollment—unstandardized effect sizes were chosen for ease of interpretation.

Below, this article provides estimates for the average effect of ACA outreach across the studies included in this meta-analysis, summarizes various estimates of the degree of treatment effect heterogeneity, and reviews results from the subgroup analyses. All analyses were performed in R Statistical Software.¹⁴ Supplementary Appendix SB1 outlines the meta-analysis strategy in more detail, presents results for all findings reported in the text, reviews the relationships between some of the subgroup variables, and discusses a series of robustness checks.

Systematic Review Results

Description of the evidence

The search process, outlined in Figure 1, identified 703 studies through databases and registries and 45 studies through hand searches and individual outreach; 34 met the inclusion criteria for the systematic review. Of these 34, 32 were randomized controlled trials, covering over 18 million participants. Two were quasi-experimental studies: one used matching to compare enrollment rates across states with and without outreach and one used a regression discontinuity approach to identify the effects of advertisements. At the time of data collection in 2022, 8 were published studies and 26 were unpublished. Eight were studies of national (or near-national) interventions, 22 were of interventions in California, and 1 each were of interventions in Kentucky, Massachusetts, Maryland, and Colorado. Data collection years ranged from 2014 to 2022.

FIG. 1.

Most studies that met the inclusion criteria for the systematic review and meta-analysis were unpublished.

Interventions aimed at learning costs

Most of the studies reviewed the effects of interventions aimed at learning costs and are described in Table 1. The interventions in this category were diverse, including information shared via text message, e-mail, direct mail, television advertisements, and website banners. Interventions were largely delivered during enrollment periods and generally contained information about the enrollment process, such as upcoming deadlines, marketplace web or contact information, estimated costs, and recommended enrollment options. In many cases, these interventions included some components informed by behavioral science, such as action language or social norm messaging. In some cases, these interventions included some element of personalization, such as a premium estimate tailored to the recipient. Three examples of these types of interventions are described in more detail, with additional information about the decision to code them as information-based interventions, in Supplementary Appendix SD1.

Table 1.

Twenty-Four Studies of Interventions Aimed at Learning Costs

Author, year of data collection	Setting	Population	Sample size	Intervention mode	Intervention type	Control group mean	Treatment effect(standard error), withranges reported formulti-arm studies
Covered California, 2020	California	Households who had started an application	4753	Text message and email	Special Enrollment Period reminder	0.0219	0.0305** (0.014)
Covered California, 2022	California	Households who submitted an application and were eligible for $0 CSR Silver coverage	4095	Email	Personalized email about CSR Silver coverage eligibility	0.16	0.023* (0.013)
Covered California, 2022	California	Households who submitted an application and were eligible for $0 CSR Silver coverage	11,245	Direct mail plus email reminder	Personalized net premium estimate	0.05	0.014*** (0.004)
Domurat et al., 2016^a ^, ²⁵	California	Californians who had an active determination of eligibility	87,394	Direct mail	Letters designed to address behavioral frictions	0.081	0.01* (0.003) to 0.015* (0.003)
Covered California, 2021	California	Households who submitted an application and were eligible for $1 CSR Silver coverage	27,500	Direct mail plus email reminder	Personalized net premium estimate	0.022	0.011*** (0.002)
Goldin et al., 2018^a ^,e ^,26	National	Taxpayers who had paid an income tax penalty for lacking coverage in 2015	4,526,717	Direct mail	Letters designed to address behavioral frictions	0.034	−0.0005 (0.0006) to 0.019 (0.001)^d
Covered California, 2021	California	Subsidy-eligible households who submitted an application	60,192	Direct mail	Deadline letter with implementation prompt	0.065	0.006*** (0.002)
Covered California, 2021	California	Subsidy-eligible households who submitted an application and reported receiving unemployment insurance compensation	43,358	Email	Email about CSR Silver eligibility	0.062	0.005* (0.003)
Covered California, 2022	California	Californians who reported being uninsured when filing their 2021 taxes	20,851	Direct mail plus email reminder	Semi-personalized net premium estimate	0.011	0.003 (0.002) to 0.006*** (0.002)
Covered California, 2022	California	Californians who reported being uninsured when filing their 2021 taxes	39,786	Direct mail plus email reminder	Semi-personalized net premium estimate	0.01	0.004*** (0.001)
Covered California, 2020	California	Likely uninsured Californians	1,226,560	Direct mail	Information co-branded with FTB about sign-up deadlines and new tax penalty	0.009	0.004*** (0.0003)
Covered California, 2019	California	Eligible households who submitted an application	475,981	Postcard	Deadline and sign-up information	0.064	0.004*** (0.001)
Epstein et al., 2019^e, ²⁸	Maryland	Uninsured identified through the Maryland Easy Enrollment Health Insurance Program	16,477	Postcard and follow-up email	Reminder with affordability, responsibility, and community message frames	0.065	0.008 (0.005) to 0.011** (0.005)
Covered California, 2022	California	Californians who reported being uninsured when filing their 2020 taxes	220,960	Direct mail	Either a reminder letter in November, a late letter in December or two letters with information about the state’s mandate penalty and enrollment information	0.021	0.002* (0.001) to 0.006*** (0.001)
Covered California, 2022	California	Californians who reported being uninsured when filing their 2021 taxes	6763	Direct mail plus email reminder	Semi- personalized net premium estimate	0.003	0.003* (0.002)
Yokum et al., 2015^a ^, ²¹	37 states on HealthCare.gov	Individuals who started the enrollment process on HealthCare.gov but had yet to complete it	744,510	Direct mail	Letters designed to address behavioral frictions	0.04	0.0007 (0.0009) to 0.009 (0.0009), with statistically significant effects in other treatment arms
Cox et al., 2014^a ^, ^c ^, ²⁷	National	Visitors to a private sector exchange website	305,720	website banners	A deadline reminder 4–10 days prior to the deadline	0.055	0.0006 (0.0006) to 0.002** (0.0009)
Covered California, 2022	California	Californians who reported being uninsured when filing their 2020 taxes	64,843	Email	Either financial help or penalty emails	0.017	0 (0.001) to 0.002* (0.001)
Epstein et al., 2020²⁸	Colorado	Uninsured identified through Colorado’s Health Care Coverage Easy Enrollment Program	128,283	Postcard and follow-up email	Advertisements with affordability, responsibility, and community message frames	0.01	−0.0005 (0.0008) to 0.008 (0.0009)
OES (family motivation), 2015^c	National	Individuals who had registered for a HealthCare.gov account but had not yet enrolled in a plan	944,167	Email	Messages about potential motivations for enrollment	0.0001	0.00006*** (0.003)
OES (deadline), 2015^c	National	Individuals who had registered for a HealthCare.gov account but had not yet enrolled in a plan	7,318,780	Email	Messages framing the deadline in various ways	0.0003	No detectable effect
OES (goal progress), 2015^c	National	Individuals who had registered for a HealthCare.gov account but had not yet enrolled in a plan	95,311	Email	Messages about potential motivations for enrollment	0.002	No detectable effect
OES (personal motivation), 2015^c	National	Individuals who had registered for a HealthCare.gov account but had not yet enrolled in a plan	1,487,131	Email	Personal motivation	0.0003	No detectable effect
Shafer et al., 2016^a ^, ^b ^, ^c ^, ²⁹	Kentucky	Observational study of state-wide enrollment rates	122 state-weeks	TV ads	Televised messages about the ACA	n/a	No significant association with enrollment

*P < 0.1.

**P < 0.05.

***P < 0.01.

indicates studies published at the time of data collection in 2022.

indicates a quasiexperimental study, rated 4 on the JAMA Quality Rating Scheme. All other studies are randomized controlled trials, rated 1.

indicates a study that qualified for inclusion in the systematic review but not in the meta-analysis. All other studies were included in the meta-analysis.

indicates a study that did not report p-values for estimated effects.

indicates a study on the effects of an intervention on enrollment in any plan, including Medicaid.

Of these 24 studies, 7 were national studies (including one that covered the 37 states on HealthCare.gov in 2015), 14 were based in California, and 1 each took place in Kentucky, Maryland, and Colorado. Twenty-three were randomized controlled trials covering sample populations totaling 17,678,399; 1 was a quasiexperimental study. Effect sizes ranged from 0.006 to 2.95 percentage points, and 19 were unpublished at the time of data collection for the meta-analysis in 2022.

Interventions aimed at learning and compliance costs

Ten studies evaluated the effects of interventions aimed at alleviating burdens associated with learning and compliance costs and are described in Table 2. The most crucial difference between these support-based interventions and the information-based interventions described above is that these included some element of assistance beyond the passive provision of information. For example, many of these interventions included a phone call through which a representative could provide a potential enrollee with personalized support as they began to consider and navigate the process. Others included mechanisms to support them as they began the process, such as a streamlined enrollment process. Supplementary Appendix SD1 outlines two examples.

Table 2.

Ten Studies of Interventions Aimed at Learning and Compliance Costs

Author, year of data collection	Setting	Population	Sample size	Intervention mode	Intervention type	Control group mean	Treatment effect (ranges reported for multi-arm studies)
Covered California, 2022	California	Californians who reported being uninsured when filing their 2020 taxes	3000	Phone call plus letter(s)	Phone call plus either an early letter in November, a late letter in December or two letters	0.02	0.047^{ (0.02) to 0.059*} (0.021)
Ravel et al., 2022^a ^, ¹⁹	California	Households who recently lost Medicaid eligibility and were eligible for subsidized marketplace coverage	2173	Emails, phone call, or both	Phone call with an enrollment reminder	0.145	−0.005 (0.021) to 0.069^{*** (0.026); treatment effects for treatments including a phone call ranged from 0.044* (0.025) to 0.069***} (0.026)
Ericson et al., 2021²²	Massachusetts	Households that were determined eligible for financial assistance	62,400	Direct mail	Generic enrollment reminders, reminders with personalized information, and streamlined enrollment	0.304	0.013^{ (0.005) to 0.032*} (0.005) in the streamlined enrollment arm
Covered California, 2020	California	Subsidy-eligible households who submitted an application	6900	Phone call	Phone call offering enrollment assistance emphasizing state-level penalty for going uninsured	0.069	0.028^*** (0.01)
Myerson et al., 2019^a ^, ¹⁸	California	People who had applied but had not selected a plan	79,522	Phone call	Phone call offering personalized information about plan options and assistance in choosing	0.12	0.027^*** (0.005)
Covered California, 2021	California	Households who submitted an application	100,000	Phone call	Phone call offering enrollment assistance	0.081	0.023^*** (0.003)
Covered California, 2020	California	Subsidy-eligible households who submitted an application	6900	Phone call	Phone call offering enrollment assistance emphasizing affordability amid state subsidy program	0.068	0.016 (0.01)
Covered California, 2022	California	Californians who reported being uninsured when filing their 2020 taxes	40,232	Phone call plus letter(s)	Phone call plus either an early letter in November, a late letter in December, or two letters	0.025	0.008^{* (0.003) to 0.013*} (0.003)
Covered California, 2020	California	Households who submitted an application	9000	Phone call	Phone call offering enrollment assistance	0.083	0.004 (0.007) to 0.013^* (0.007)
Orzol et al., 2014^a ^, ^b ^, ^c ^, ³⁰	National	People in non-Medicaid expansion states	2001	Outreach from an assistance program	Enroll America outreach	n/a	No percentage point increase reported (15.1% increase in enrollment)

P < 0.1.

P < 0.05.

***

P < 0.01.

indicates studies published at the time of data collection in 2022.

indicates a quasiexperimental study, rated 4 on the JAMA Quality Rating Scheme. All other studies are randomized controlled trials, rated 1.

indicates a study that qualified for inclusion in the systematic review but not in the meta-analysis. All other studies were included in the meta-analysis.

indicates a study that did not report P-values for estimated effects.

Of these 10 studies, 9 were randomized controlled trials and 1 was a quasi-experimental study; the randomized studies covered a total sample population of 310,127. One of these studies was nationwide, 1 in Massachusetts, and 7 in California. Seven were unpublished at the time of data collection. Most effect sizes were larger than those attributable to the information-only interventions described above; effect sizes in this category ranged from 0.4 to 5.9 percentage points. The meta-analysis that follows explores this difference in effects by intervention strategy more systematically.

Meta-analysis results: Average estimated effects and heterogeneity

Twenty-seven of the studies identified in the systematic review qualified for inclusion in the meta-analysis, covering 58 treatment arms; 22 studies were unpublished at the time of data collection. Across interventions, the average enrollment rate in the control groups, weighted by sample size, was 3.8%. For those who received an intervention, the meta-analysis indicates that the enrollment rate was 0.9 percentage points higher, a 24% increase (0.009, P < 0.001; 95% confidence interval [CI; 0.006, 0.012]).

The 95% prediction interval in which the effects of a new intervention might fall, if its target population were selected at random from the combined population of the meta-analysis, was −0.006 to 0.024. Prediction intervals are analogous to confidence intervals, incorporating effect size heterogeneity as an additional source of uncertainty. The prediction interval was wider than the confidence interval by approximately 1.2 percentage points in either direction, suggesting noteworthy heterogeneity in the impacts of ACA outreach interventions.

Further heterogeneity estimates are provided in Figure 2, a forest plot. Cochran’s Q test rejects the null hypothesis of no effect size heterogeneity (Q = 1648.02, df = 57, P < 0.01). The authors also estimate τ = 0.007, which can be thought of as the standard deviation of the distribution of possible true effect sizes. This value is almost as large as the pooled effect estimate. Lastly, the I-squared estimate of 96.4% suggests that an overwhelming majority of the effect size variance across the sample stems from effect size heterogeneity rather than sampling error.

FIG. 2.

Interventions with support components aimed at learning and compliance costs increased enrollment more than information alone.

The forest plot illustrates that treatments that involved support, ie, those aimed at alleviating compliance costs, were more effective than information-only interventions, which aim to alleviate learning costs. Support-based interventions increased enrollment by 2 percentage points (0.020, P = 0.004; 95% CI [0.010, 0.030]), while information-based interventions increased enrollment by 0.6 percentage points (0.006, P < 0.001; 95% CI [0.004, 0.008]). A cluster-robust Wald test rejected the hypothesis that average effects are equal across these subgroups (P = 0.010).

Personalized interventions were also more effective. Personalized interventions increased enrollment by 1.2 percentage points (0.012, P < 0.001; 95% CI [0.008, 0.016]), while interventions with no personalized component increased enrollment by 0.5 percentage points (0.005, P = 0.011; 95% CI [0.002, 0.009]). A cluster-robust Wald test rejected the hypothesis that average effects are equal across these subgroups (P = 0.045).

In addition, outreach interventions may be more effective among people with some prior interaction with the marketplaces, such as those who have initiated the enrollment process by submitting an application. Among those with some prior interaction, interventions increased enrollment by 1.2 percentage points (0.012, P < 0.001; 95% CI [0.007, 0.017]), compared with 0.6 percentage points (0.006, P = 0.004; 95% CI [0.002, 0.010]) among those with no prior interaction. A cluster-robust Wald test rejected the hypothesis that average effects are equal across these subgroups (P = 0.014).

In contrast, there was no evidence of statistically significant differences in estimated effect sizes across study setting, year, or timing relative to the declaration of the federal COVID-19 Public Health Emergency.

Discussion

As policymakers seek to help the remaining uninsured obtain health insurance and facilitate coverage transitions for those losing Medicaid, this systematic review and meta-analysis found that interventions aimed at addressing administrative burdens caused statistically significant increases in ACA take-up. Although information-based interventions targeting learning costs increased ACA enrollment, support-based interventions targeting learning and compliance costs were significantly more effective. Outreach with personalized elements also more significantly increased ACA enrollment. Finally, this study found that outreach was more effective among people who had already interacted with the marketplace in some way, such as by starting or submitting an application. These findings underscore the benefits of behaviorally informed outreach interventions while also documenting variation in their relative efficacy.

Many states are conducting outreach, creating resources, training assisters, and taking other proactive steps to increase enrollment.¹⁵ Prior research has found that informational interventions tend to be more effective among those less in need,^16,17 whereas support-based interventions tend to help more disadvantaged subpopulations.^18,19 For example, in California’s ACA marketplace, informational nudges delivered via mail and email were most effective among penalty payers and healthier individuals, while personalized enrollment assistance was most effective among older consumers, low-income consumers and those with a Spanish language preference. With that in mind, policymakers concerned about equitable take-up will need to deploy more support-based interventions.²⁰

One of the key considerations in implementing these more effective and more equitable support-based interventions is that they may be more resource-intensive than alternatives. Cost estimates were not available for all the interventions covered in this systematic review and meta-analysis, so additional research on the cost effectiveness of providing support in addition to information would be constructive. However, a prior randomized evaluation of support-based outreach indicated that even more resource-intensive outreach interventions can be cost-effective for agencies, providing a two-to-one return on investment.¹⁸

In addition, the cost difference between providing support or personalization and information alone may be relatively minimal. In one study, providing nonpersonalized letters was associated with an overall cost per new enrollee of $191.²¹ In comparison, an evaluation of personalized telephone outreach tested an intervention that cost approximately $224 per new member acquired.¹⁸ Personalization may be an even lower cost alternative: a study testing personalized messages found that they yielded an overall cost per new enrollment of approximately $50 during a special enrollment period and $41 during an open enrollment period.¹⁷

Yet even when states offer enrollment assistance, take-up remains incomplete. Structural approaches, such as autoenrollment, may be necessary to connect people to benefits for which they are eligible. An evaluation of a streamlined enrollment process, which required a marketplace to manually process paper forms returned by mail, benchmarked effects to the reduction in premiums needed for a commensurate increase in enrollment: a decrease in premiums of $39 per year would be necessary to increase enrollment as much as streamlined enrollment.²²

Limitations

Systematic reviews and meta-analyses have risks, which the authors worked to mitigate. Publication bias is a concern, so the systematic review process included a thorough review of unpublished studies. This was intended to both broaden the knowledge base around interventions to support health insurance enrollment and reduce the chance of overestimating effects in the meta-analysis, as published studies are often more likely to report statistically significant findings than unpublished studies. The value of mitigating publication bias via the inclusion of unpublished evaluations outweighs any potential risks due to methodological differences between published and unpublished evaluations; in addition, including these unpublished evaluations offers an opportunity to extend the knowledge base beyond prior systematic reviews of published studies. Effect sizes also tend to be larger in quasiexperiments than randomized experiments,²³ so the meta-analysis included only experiments with true controls. It is possible that the systematic review search missed relevant studies, and excluding quasi-experimental studies limits the types of interventions covered.

Another challenge for meta-analyses is the potential for study-level biases, which lead to inaccurate averages. However, this study focuses on exploring sources of heterogeneity, rather than simply estimating an overall pooled effect size, makes this less of a concern. In addition, the authors’ assessments of study-level quality suggest that there is minimal risk-of-bias among the included studies.

All 27 of the studies included in the meta-analysis are randomized controlled trials, which require high organizational capacity to plan, execute, and analyze. It is possible that high research capacity is correlated with stronger tools overall or a focus on innovation at the organizational level, which would limit the generalizability of these results to settings in which organizations are working with fewer resources or different orientations toward innovation.

In addition, most of the studies included took place in California, which could raise questions around generalizability. However, effect sizes were statistically similar across California and non-California studies, eligibility criteria for marketplace coverage are broadly similar across states such that marketplaces serve similar types of consumers, and the reported interventions sought to address commonly cited barriers to take-up such as limited awareness and the complexity of the application and enrollment process. Furthermore, California’s demographics reflect predicted demographic trends in the rest of the United States,²⁴ so findings in California could have future relevance elsewhere. For these reasons, these results are well-positioned to inform outreach efforts across the ACA marketplaces.

Conclusions

This systematic review and meta-analysis demonstrate that outreach interventions can increase ACA marketplace enrollment; even with small effect sizes, the implications at a population level are still meaningful. Interventions aimed at alleviating compliance costs by providing enrollment support were about three times as effective as information alone. These support-based interventions are likely to be more resource-intensive, but they have been found to be cost-effective.¹⁸ Amid the end of the federal COVID-19 Public Health Emergency and the availability of enhanced marketplace subsidies, these findings have implications for policymakers looking for effective strategies to preserve the coverage gains in recent years.

Author Disclosure Statement

This publication does not represent the views of the General Services Administration (GSA) or the federal government. The co-authors of this article granted unlimited and unrestricted rights to GSA to use and reproduce all materials in connection with the co-authorship. This publication was written independently by the authors and did not go through internal Office of Evaluation Sciences (OES) review beyond analyses presented in the OES abstract. The publication’s analyses comply with the journal’s peer review process, rather than the OES project process, for validation.

Footnotes

Supplementary Material

Supplementary Appendix SA1

Supplementary Appendix SB1

Supplementary Appendix SC1

Supplementary Appendix SD1

References

Keith

. Record-High marketplace enrollment, but failure to extend subsidies could impact consumers. Health Affairs Forefront; doi: 10.1377/forefront.20220328.489767

distribution of eligibility for aca health coverage among the remaining uninsured. KFF. Published April 7, 2021. Accessed [Last accessed: May 29, 2023 ]. Available from: https://www.kff.org/health-reform/state-indicator/distribution-of-eligibility-for-aca-coverage-among-the-remaining-uninsured/

Buettgens

, Green

. The Impact of the COVID-19 Public Health Emergency Expiration on All Types of Health Coverage. Urban Institute; 2022. Accessed [Last accessed: May 29, 2023 ]. Available from: https://www.urban.org/research/publication/impact-covid-19-public-health-emergency-expiration-all-types-health-coverage

Page

, McKenzie

, Bossuyt

, et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. Syst Rev, 2021; 10(1):89; doi: 10.1186/s13643-021-01626-4

Sutton

, Duval

, Tweedie

, Abrams

, Jones

. Empirical assessment of effect of publication bias on meta-analyses. BMJ, 2000; 320(7249):1574–1577; doi: 10.1136/bmj.320.7249.1574

Jackson

, Mackevicius

. The Distribution of School Spending Impacts. February 12021 Accessed May 29, 2023. https://papers.ssrn.com/abstract=3795037

Instructions for Authors. Ratings of the quality of the evidence. Available from: https://jamanetwork.com/journals/jamanetworkopen/pages/instructions-for-authors

Moynihan

, Herd

, Harvey

. Administrative Burden: Learning, Psychological, and Compliance Costs in Citizen-State Interactions. Journal of Public Administration Research and Theory, 2015; 25(1):43–69; doi: 10.1093/jopart/muu009

Mulligan

. The problem of choice: From the voluntary way to Affordable Care Act health insurance exchanges. Soc Sci Med, 2017 May;181:34–42; doi: 10.1016/j.socscimed.2017.03.055

10.

Hedges

. A random effects model for effect sizes. Psychological Bulletin, 1983; 93(2):388–395; doi: 10.1037/0033-2909.93.2.388

11.

Hartung

, Knapp

, Sinha

. Statistical Meta-Analysis with Applications. Wiley; 2008. Accessed [Last accessed: May 30, 2023 ]. Available from: https://www.wiley.com/en-us/Statistical+Meta+Analysis+with+Applications-p-9780470290897

12.

Marín-Martínez

, Sánchez-Meca

. Weighting by inverse variance or by sample size in random-effects meta-analysis. Educational and Psychological Measurement, 2010; 70(1):56–73; doi: 10.1177/0013164409344534

13.

Pustejovsky

, Tipton

. Meta-analysis with robust variance estimation: Expanding the range of working models. Prev Sci, 2022; 23(3):425–438; doi: 10.1007/s11121-021-01246-3

14.

R Core Team. R: A language and environment for statistical computing. Published online 2021. Available from: https://www.R-project.org/

15.

States of Unwinding . April 28, 2023. State health and value strategies; 2023. Accessed [Last accessed: May 30, 2023 ]. Available from: https://www.shvs.org/states-of-unwinding-april-28-2023/

16.

Finkelstein

, Notowidigdo

. Take-Up and Targeting: Experimental evidence from SNAP. * The Quarterly Journal of Economics, 2019; 134(3):1505–1556; doi: 10.1093/qje/qjz013

17.

Feher

, Menashe

, Miller

, Wolf

. Personalized letters and emails increased marketplace enrollment among households eligible for zero-premium plans. Health Aff (Millwood), 2023; 42(4):585–593; doi: 10.1377/hlthaff.2022.01301

18.

Myerson

, Tilipman

, Feher

, Li

, Yin

, Menashe

. Personalized telephone outreach increased health insurance take- up for hard-to-reach populations, but challenges remain. Health Aff (Millwood), 2022; 41(1):129–137; doi: 10.1377/hlthaff.2021.01000

19.

Ravel

, Ahrary

, Avakian

, Feher

, Menashe

. Effect of personalized outreach on medicaid to marketplace coverage transitions: A randomized clinical trial. JAMA Health Forum, 2022; 3(10):e223616; doi: 10.1001/jamahealthforum.2022.3616

20.

Myerson

, Dague

. Identifying scalable strategies to maintain coverage as medicaid continuous enrollment ends. Health Affairs Forefront; doi: 10.1377/forefront.20230504.122888

21.

Yokum

, Hopkins

, Feher

, Safran

, Peck

. Effectiveness of behaviorally informed letters on health insurance marketplace enrollment. JAMA Health Forum, 2022; 3(3):e220034; doi: 10.1001/jamahealthforum.2022.0034

22.

Ericson

, Layton

, McIntyre

, Sacarny

. Reducing administrative barriers increases take-up of subsidized health insurance coverage: Evidence from a field experiment. National Bureau of Economic Research, 2023:w30885; doi: 10.3386/w30885

23.

Cheung

ACK

, Slavin

. How Methodological Features Affect Effect Sizes in Education. Educational Researcher, 2016; 45(5):283–292; doi: 10.3102/0013189X16656615

24.

Jain

, Reno

, Cohen

, Bassey

, Master

. Building a culturally-responsive, family-driven early childhood system of care: Understanding the needs and strengths of ethnically diverse families of children with social-emotional and behavioral concerns. Children and Youth Services Review, 2019; 100:31–38; doi: 10.1016/j.childyouth.2019.02.034

25.

Domurat

, Menashe

, Yin

. The role of behavioral frictions in health insurance marketplace enrollment and risk: Evidence from a field experiment. American Economic Review, 2021; 111(5):1549–1574; doi: 10.1257/aer.20190823

26.

Goldin

, Lurie

, McCubbin

. Health insurance and mortality: Experimental evidence from taxpayer outreach. The Quarterly Journal of Economics, 2021; 136(1):1–49; doi: 10.1093/qje/qjaa029

27.

Cox

, Handel

, Kolstad

, Mahoney

. Messaging and the mandate: The impact of consumer experience on health insurance enrollment through exchanges. Am Econ Rev, 2015; 105(5):105–109; doi: 10.1257/aer.p20151080

28.

Epstein

, Ko

, Robertson

, Wilson

, Yokum

. Can moral framing drive insurance enrollment in the us? A randomized field experiment. Published Online May, 2023; 17.

29.

Shafer

, Fowler

, Baum

, Gollust

. Television advertising and health insurance marketplace consumer engagement in kentucky: A natural experiment. J Med Internet Res, 2018; 20(10):e10872; doi: 10.2196/10872

30.

Orzol

, Hula

. Impact of enroll america on the number of individuals covered through the federally facilitated marketplace. Health Serv Res, 2018; 53(1):341–365; doi: 10.1111/1475-6773.12630

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.39 MB

Interventions to Increase Affordable Care Act Marketplace Enrollment: A Systematic Review and Meta-Analysis

Abstract

Introduction

Methods

Search strategy and data sources

Study selection

Data extraction and synthesis

Statistical Analysis

Systematic Review Results

Description of the evidence

Interventions aimed at learning costs

Interventions aimed at learning and compliance costs

Meta-analysis results: Average estimated effects and heterogeneity

Discussion

Limitations

Conclusions

Author Disclosure Statement

Footnotes

Supplementary Material

References

Supplementary Material