Abstract
Medicare Advantage (MA) has grown rapidly since the Affordable Care Act; nearly one-third of Medicare beneficiaries now choose MA. An assessment of the comparative value of the 2 options is confounded by an apparent selection bias favoring MA, as reflected in mortality differences. Previous assessments have been hampered by lack of access to claims diagnosis data for the MA population. An indirect comparison of mortality as an outcome variable was conducted by modeling mortality on a traditional fee-for-service (FFS) Medicare data set, applying the model to an MA data set, and then evaluating the ratio of actual-to-predicted mortality in the MA data set. The mortality model adjusted for clinical conditions and demographic factors. Model development considered the effect of potentially greater coding intensity in the MA population. Further analysis calculated ratios for subpopulations. Predicted, risk-adjusted mortality was lower in the MA population than in FFS Medicare. However, the ratio of actual-to-predicted mortality (0.80) suggested that the individuals in the MA data set were less likely to die than would be predicted had those individuals been enrolled in FFS Medicare. Differences between actual and predicted mortality were particularly pronounced in low income (dual eligibility), nonwhite race, high morbidity, and Health Maintenance Organization (HMO) subgroups. After controlling for baseline clinical risk as represented by claims diagnosis data, mortality differences favoring MA over FFS Medicare persisted, particularly in vulnerable subgroups and HMO plans. These findings suggest that differences in morbidity do not fully explain differences in mortality between the 2 programs.
Background
In 2015, 31% of current Medicare beneficiaries were enrolled in a private managed care plan (Medicare Advantage [MA]) rather than the default option of federally administered traditional fee-for-service (FFS) Medicare. 1 The original proponents of MA saw the potential for improved quality of care, which could lead to better health outcomes when compared with those of FFS Medicare. 2 However, the ability of MA to achieve these goals has not been fully demonstrated. A recent systematic review published by the Kaiser Family Foundation described the evidence regarding the relative impact of MA on actual objective health outcomes and mortality as outdated and characterized by insufficient control for selection bias. 3 Some studies comparing mortality and/or switching rates between MA and FFS Medicare have concluded that differences suggest selection bias favorable to MA, but authors have also acknowledged that the lack of publicly available claims diagnosis data for the MA population limits comparisons between the 2 populations. In other words, an adjustment for underlying morbidity might improve comparisons of mortality between FFS Medicare and MA.4-7
Only 1 comparison of FFS Medicare and MA published since 2000 has addressed mortality differences with control for selection bias due to differences in morbidity. 8 This study concluded that a favorable selection bias does not explain the lower rates of mortality observed in MA. The study relied on self-reported morbidity data collected through the 1996 and 1998 Medicare Current Beneficiary Survey.
The present study compared mortality between the MA population covered by Humana Inc, a large health and wellness company with more than 1.9 million MA enrollees as of December 31, 2012, the end of the study period, and an FFS Medicare sample with correction for selection bias through adjustment for chronic conditions and demographic factors, with consideration of the potential for bias due to greater coding intensity in MA plans. Further analysis calculated ratios for subgroups. The study builds on previous research by conducting an evaluation from a recent time frame and with adjustment for morbidity.
Methods
Modeling Design
An indirect comparison of mortality between MA and FFS Medicare was conducted by modeling person-level mortality on an FFS Medicare data set, applying the model to an MA data set, and then evaluating the ratio of actual-to-predicted mortality rates in the MA data set. The predicted MA mortality rate would represent the probability of mortality if the individuals were enrolled in FFS Medicare, that is, the rate that one would expect to observe if enrollment in MA had no impact on mortality.
Data Source and Populations
The FFS Medicare data comprised 5% Limited Data Set samples from the Centers for Medicare & Medicaid (CMS), which are randomly selected, for the years 2010-2012. These samples represented 4 313 885 person-years. The MA data came from the insurer for the years 2011-2013. The variables used from the MA data set were equivalent to those available in the FFS Medicare data set. The MA sample represented 5 477 976 person-years. Both data sets represented the most recent data available at the time the study was initiated. Mortality data were derived from CMS denominator files for the FFS Medicare data set and from CMS Monthly Membership Reports for the MA data set. Individuals with end-stage renal disease (ESRD), non-US residence, Medicare secondary coverage, a group Medicare plan, or recognizably missing data were excluded.
As this study was conducted as part of the insurer’s normal business operations, it did not meet the Department of Health and Human Service’s regulatory definition of research under 45 Code of Federal Regulations 46.102(d), and thus did not require institutional review board approval or a Health Insurance Portability and Accountability Act (HIPAA) exemption—either for analysis of the insurer’s MA data or for analysis of the CMS 5% Limited Dataset. The authors have access to patient identifying information in the insurer’s MA claims and enrollment databases through the course of their daily job responsibilities and have accessed such data to complete this work. Corporate policy forbids analysts from accessing more than the minimum protected health information (PHI) data necessary to fulfill specific business needs. Patient confidentiality is further protected by avoiding the reporting of data for subgroups with 10 or fewer individuals. The FFS data, provided in a CMS 5% Limited Dataset, were de-identified. In addition, the data use agreement with CMS additionally required that aggregate data be reported for no fewer than 11 individuals.
Model Development and Analysis
In the first step, the FFS Medicare data set was partitioned into 3 independent groups: a calibration (derivation) set and 2 testing sets, designed to all be comparable with respect to average calculated CMS risk score. The predictor variables in CMS risk score models are Hierarchical Condition Categories (HCCs), which are based on prior-year claims and reflect diagnosis as well as severity, and several demographic factors. The CMS model for the risk score considered in this study was based on a set of 70 HCCs (see Table 1).
Variables Included in the Final Model.
Note. HCCs = Hierarchical Condition Categories; SSA = Social Security Administration.
Race designations were derived from the SSA. As the SSA only classified race/ethnicity as white, black, other, or unknown prior to 1980 and as most current Medicare members were born prior to 1980, this study conformed to those categories.
Eligibility for Medicaid in addition to Medicare.
Next, a logistic regression model was designed, using SAS Enterprise Guide 5.1, to define the probability of death in a given year for the calibration group in the FFS Medicare data set. Demographics, Medicare eligibility characteristics, morbidity data (chronic conditions, CMS risk score, HCC count), and prior-year claims dollars (as a proxy for health risk) were considered for inclusion in the model as covariates (predictor variables).
Each possible predictor variable was considered for its potential to create bias in the model. As risk-adjusted payments create an incentive for MA plans to more completely document beneficiary diagnoses than is typical in FFS Medicare, inclusion of diagnosis-based variables as model predictors might elevate predicted mortality in the MA data set and thus artificially reduce the actual-to-predicted mortality ratio, creating a bias in favor of MA. To reduce this potential bias, CMS risk score was omitted from the list of variables eligible for the model. However, a variable for an individual’s total number of HCCs (HCC count) remained eligible for the model, as did several variables reflecting the presence of chronic or acute conditions (see Table 1). Variables for diabetes and renal disease were omitted because of pending changes with respect to these disease areas in the CMS HCC model. 9 Next, a forward stepwise selection process was used to determine which of the eligible variables would be included in the final model. Each variable was assessed for contribution to the overall model fit using the residual chi-square test with a significance requirement of P = .05. Of the variables considered, only heart arrhythmia and panel year were omitted from the final model. In the final model, each variable had a P value <.01 according to the chi-square test.
A final sensitivity analysis was then conducted by assessing the accuracy of models with different combinations of variables. The final model with the retained condition variables plus HCC count had greater accuracy at the individual level (80.2% concordance) than that of a model with the chronic conditions only (78.7% concordance) and that of a model with demographic variables only (71.2% concordance). Furthermore, the actual-to-predicted ratio in the model with condition variable plus HCC count was less favorable to MA (ratio, 0.8042) than the model with demographics only (ratio, 0.7875). Thus, inclusion of the selected morbidity measures actually yielded a more conservative estimate of MA-FFS differences, despite potential coding bias. The tau-c statistic for the final model was 0.809, which signified a relatively strong association between predicted and actual mortality rates.
Although prior-year claims dollars was considered as a predictive variable, it was ruled out due to its potential as a source of bias. While prior-year spending has been shown to be a good predictor of morbidity, differences in spending between MA and FFS Medicare would be confounded by differences in reimbursement models, types of benefits, and out-of-pocket spending caps.
After selection of the predictor variables, the model was fit to the calibration group, which represented 1 437 937 person-years for 2010-2012. The final set of predictor variables is presented in Table 1. The resulting odds ratios for the calibration group are presented in Table S1 in the Supplementary Material.
When applied to the first FFS Medicare testing group, the model produced an actual-to-predicted mortality ratio of 1.0008. Out of the 1 437 880 individuals in the group, predictions were off only by 49 deaths, producing a Z score of 0.21. The actual-to-predicted ratios by subgroup (eg, subgroups defined by HCC count or gender) were checked to assure that the model maintained group-level accuracy across the subgroups as well as for the overall testing group. Ratios remained close to 1.00 (range, 0.9368-1.2017, with the extremes being tied to relatively small subgroups), and 37 of 40 Z scores were within 2 standard deviations. Given these indications of group-level accuracy in the first testing group, the model was not tested in the second testing group. A plot showing model accuracy at various levels of predicted mortality is included in Figure S1 in the Supplementary Material. This revealed some bias for individuals at the highest level of predicted mortality, but with a good model fit overall.
Finally, the resulting model was applied to the MA data set to yield the expected mortality rate had the MA enrollees been in FFS Medicare. Additional analysis was conducted according to subgroups defined by gender, age, race, dual eligibility in the prior year, HCC count, plan type, duration of Medicare eligibility, and the basis of eligibility for Medicare (age or disability). The statistical significance of the difference between actual and predicted mortality overall and within each subgroup was assessed by Z scores.
Results
Study Population
The FFS Medicare calibration group and MA data set were similar in mean age (71.5 and 71.1 years, respectively), gender distribution (56% and 55% female), and clinical risk according to mean HCC count (1.55 and 1.58). Both groups were predominately white (84.8% and 83.0%), but the FFS Medicare group included fewer black participants (9.3% vs 12.2%) and slightly more other nonwhite participants (5.5% vs 4.4%). FFS Medicare participants were more likely to have dual eligibility in the prior year (20.2% vs 14.7%). The MA population was more heavily concentrated in the South (64% vs 40% of FFS Medicare participants). More than 66% of MA participants were in newer, non-HMO plan types. Population characteristics are presented in Table S2 of the Supplementary Material.
Predicted Versus Actual Mortality in MA
When the mortality regression model was applied to the MA data set, predicted mortality was 3.7%, whereas actual mortality was 2.9% (P < .001 for the difference). These results compare with both a predicted and an actual rate of 4.3% (2010-2012) in the FFS Medicare calibration group. The difference in predicted mortality between MA and FFS Medicare (3.7% vs 4.3%) suggests that the MA members were healthier, which could explain the lower actual mortality in the MA population versus FFS Medicare (2.9% vs 4.3%). Nevertheless, the actual-to-predicted ratio for the MA data set was <1.00 (0.804; 95% confidence interval [CI], 0.801-0.808). In other words, MA participants were less likely to die in a 1-year time frame than would have been expected for individuals with a similar risk profile enrolled in FFS Medicare. These results suggest that factors other than demographics and morbidity are responsible for mortality differences.
Demographic Subgroups
Actual-to-predicted mortality ratios were lower (more favorable to MA) in individuals above the age of 50 compared with younger members and lower in women compared with men, but ratios favored MA in both women and men. All subgroups with known race had actual mortality rates that were lower than predicted. Compared with the white subgroup, nonwhite subgroups (black, other) exhibited considerably lower actual-to-predicted ratios even though predicted mortality was comparable between the black and white subgroups. The actual-to-predicted ratio was also lower in the subgroup with dual eligibility in the prior year, compared with other participants (see Table 2).
Mortality by Demographic Subgroup.
Note. CI = confidence interval; SSA = Social Security Administration.
P < .001 for all absolute differences between actual and predicted mortality, except in the subgroup of unknown race (P < .9765) and in the age bands of 20 (P < .9120), 30 (P < .9526), and 40 (P < .8155).
Race designations were derived from the SSA. As the SSA only classified race/ethnicity as white, black, other, or unknown prior to 1980 and as most current Medicare members were born prior to 1980, this study conformed to those categories.
Eligibility for Medicaid in addition to Medicare.
Morbidity Subgroups
The difference between actual and expected mortality was more pronounced among those individuals with the highest morbidity, as represented by number of HCCs. Members without any HCCs had low and similar predicted and actual mortality rates. As the number of HCCs increased, both actual and predicted mortality increased, but actual mortality increased more gradually so that the absolute difference between them grew. Relative differences, expressed as actual-to-predicted ratios, consistently favored MA in individuals with an HCC count ≥1 (see Figure 1).
Mortality by Hierarchical Condition Category count.
Subgroups Defined by Plan Type and Eligibility Characteristics
Individuals with Local Preferred Provider Organization (LPPO) and Regional Preferred Provider Organization (RPPO) plan types had the lowest actual and predicted mortality rates. Those with an HMO plan had the highest predicted mortality but the lowest actual-to-predicted ratio, suggesting that the greatest effect relative to FFS Medicare occurred in HMO plans. The Private Fee-For-Service (PFFS) subgroup showed the least difference between actual and predicted mortality. Actual-to-predicted mortality ratios were lower in members who had been eligible for Medicare for more than 1 year compared with members who had become eligible in the current year. No difference in actual-to-predicted mortality was shown between individuals who qualified for Medicare because of disability and those who qualified on the basis of age (see Table 3).
Mortality According to Plan Type and Eligibility Characteristics.
Note. CI = confidence interval; PPO = Preferred Provider Organization; HMO = Health Maintenance Organization.
P < .001 for all absolute differences between actual and predicted mortality.
Post Hoc Analysis
As the analysis consistently showed lower than expected mortality rates in the MA population, a post hoc analysis was conducted to investigate participation in a relatively new care management service as one possible driver of this improvement. This holistic service is designed for individuals who not only have serious medical needs but who also are expected to benefit from assistance with a range of other needs. For the year 2013, the first year of the program with substantial participation, members who were eligible and participating in the program experienced a mortality rate that was 3.1 percentage points lower than that for those who were eligible but not participating (6.4% vs 9.5%), while both groups had a similar predicted mortality rate (participating, 7.3%; not participating, 7.2%). The actual-to-predicted ratio for participants favored MA, but the ratio for eligible nonparticipants did not, suggesting that the benefit of MA for this subgroup was mediated through the care service. However, these findings may reflect selection bias as participation is voluntary, and these early findings may not be replicated as the program matures. Furthermore, the accuracy of the model in this subgroup with high morbidity could not be tested during model development because there was no way to identify a comparable subgroup in the FFS Medicare data set.
Discussion
Actual mortality in the MA data set was less than risk-adjusted, expected mortality, both overall and in most subgroups. Some of the largest differences between actual and predicted mortality were observed in certain vulnerable subgroups defined by race or income, and in HMO plans as opposed to PFFS or PPO plans. Although subject to the possibility of unmeasured confounders, these findings provide an important contribution to the sparse body of literature evaluating the relationship between MA participation and mortality.
Findings in the Context of Other Research
Only 1 other study published since 2000 has compared general mortality as an outcome measure between FFS Medicare and MA. 8 Dowd and colleagues found no difference in 2-year mortality between FFS Medicare and MA HMOs when considering observed confounders. In contrast, the present study suggested that MA is associated with lower adjusted mortality. Several aspects of the Dowd study and the present study may explain the discordant findings (Dowd vs present study): largely self-reported versus objective observed confounders, 1996-2000 versus 2010-2012 and 2011-2013 time frames, sample size 15 164 survey respondents versus 6 913 915 person-years, all MA providers versus a single provider, and exclusion versus inclusion of individuals in Medicare because of disability and individuals eligible for Medicaid.
Some comparative studies have evaluated mortality primarily as a baseline indicator of health status rather than as an outcome. Krumholz et al found that the difference in mortality rates between the fee-for service and MA populations remained stable from 2003 through 2013, with a difference (0.72-0.89 percentage point) favoring MA. 5 This difference compares with an unadjusted difference of 1.4%, also favoring MA, in the present study. Song has shown that as MA attracts a greater share of the market, the health status of MA populations improves relative to FFS Medicare populations. 7 The studies by Krumholz et al and Song suggest the possibility of a selection bias favoring MA, but in contrast to the present study, neither adjusted for morbidity. The authors of both studies noted that their analysis could not assess the extent to which MA plans contribute to the health of their members as opposed to attracting healthier members.
Unobserved Confounding and Other Factors Affecting Mortality Differences
Between 2004 and 2007, CMS policy efforts sought to minimize favorable selection in MA. To the extent that favorable selection persists, it may have created bias in the present study in favor of MA and thus reduced the actual-to-predicted mortality ratio. One group investigating the impact of these policy changes has concluded that although some selection bias favoring MA plans remains, it has been substantially reduced.10-13 In the present study, predicted mortality was indeed lower in the MA population than in the FFS Medicare population, which suggests that healthier individuals do tend to choose or remain in MA. However, the actual-to-predicted mortality ratios suggested that differences in baseline morbidity do not fully explain mortality differences; that is, either program factors or patient factors other than baseline chronic disease are likely responsible for the observed findings favorable to MA.
Various aspects of MA plans may explain lower mortality rates, such as the use of alternative payment models or emphasis on preventive care and disease management. In this study, the greatest reduction from predicted mortality was in HMO plans, which account for the majority of MA participants across all contractors, suggesting that HMO plans are particularly effective relative to the other plan types assessed. However, as the HMO subgroup also had the highest predicted mortality, further study is needed to assess whether a benefit was observed in this subgroup simply because MA is particularly effective in less healthy individuals.
Alternatively, it is possible that the observation of lower mortality rate in MA, even after correction for differences in morbidity, reflects unmeasured patient factors that favor MA. For example, individuals who choose or stay in MA may have better lifestyle habits or health-seeking behaviors that help them achieve better outcomes even when demographic and clinical factors put them at risk. Interestingly, an additional analysis in the aforementioned study by Dowd et al 8 used novel statistical techniques to adjust for potential unobserved as well as observed confounders and showed 2-year mortality to be lower in MA, whereas analysis based only on observable confounders had shown no difference between MA and FFS Medicare. Statistical significance in the analysis that included unobservable confounders depended on how the variable representing geographic fixed effects was defined. Dowd and colleagues described these findings as evidence of a selection bias against MA. 8 Other examples of assessing the effect of unmeasured confounders in comparisons of MA and FFS Medicare were not identified in the literature.
Some research has suggested that mortality differences between FFS Medicare and MA reflect the fact that switching from MA to FFS Medicare is more common than switching in the reverse direction among individuals in nursing homes or inpatient facilities, that is, among individuals most likely to die in the near term. 4 The present study’s findings neither contradicted nor supported those findings but rather showed that for a given MA panel, mortality was lower than morbidity and demographic risk would predict.
Subgroups
Important findings from this study included evidence that MA may be especially effective in addressing health care disparities in vulnerable subgroups: nonwhite groups and individuals with low income as reflected by dual eligibility. Previous research has also suggested relatively greater MA benefits for minorities.14,15 These findings are relevant in light of national policy. The Healthy People Initiative of the Centers for Disease Control and Prevention (CDC) has set a goal of eliminating health disparities by 2020 in groups that are disadvantaged due to factors such as race/ethnicity, socioeconomic status, and disability. 16 In keeping with these concerns, the CMS Office of Minority Health recently released for the first time MA quality data stratified by race/ethnicity. 17
Limitations
Certain factors may detract from the comparability of the MA and FFS Medicare data sets. The Northeast and the West geographic regions were underrepresented in the MA study population due to low membership in those areas, and the FFS Medicare data set was not selected to match the geographic distribution of the MA population. However, a visual inspection of state-specific actual mortality rates showed that in both the FFS Medicare and MA data sets, states in the Northeast and West regions were more likely to have relatively lower mortality rates. Thus, any bias that might have resulted from these geographic differences would likely have increased the observed MA actual-to-predicted mortality ratio, making results less favorable to MA. In addition, previous research has shown that when patients in FFS Medicare moved to another hospital referral region, the increase in the average number of diagnoses was considerably greater if they moved to a region characterized by high-intensity practice patterns (reflected in utilization as well as coding). 18 If the 2 populations differed with respect to the mix of regions with low-intensity and high-intensity practice patterns, irrespective of whether patients were in FFS Medicare or MA, results could be biased in one direction or the other. The 3-year time frame differed by 1 year between the 2 data sets, but the authors do not know of a reason this would have substantially altered results, given the stability of mortality rates in both FFS Medicare and MA in recent years. 5 Finally, dual eligibility status represents a limited means of adjusting for socioeconomic differences between the 2 populations.
Several limitations inherent to study design apply to this work. First, as noted previously, unmeasured confounding may have biased results. Second, some individuals in the MA data set may not have been enrolled long enough for their survival to be attributable to participation in the plan. However, as the average tenure within any 1-year panel of individuals in the insurer’s MA plans is approximately 4 years, findings can reasonably be assumed to reflect an effect of participation in MA. The analysis may in fact underestimate the eventual benefit as the total cumulative enrollment, over time, is substantially greater than 4 years for the MA population evaluated by this study.
Claims data (whether from FFS Medicare or an MA plan) are subject to missing values and incorrect coding, and the FFS Medicare sample may have been more likely than the MA data set to have missing morbidity data. The extent of this discrepancy is unknown, but selection of predictor variables was designed to minimize the difference and sensitivity testing, using models with and without morbidity variables, suggested that study results could not be fully attributed to coding bias.
The authors acknowledge that the results may not be generalizable to MA populations served by other insurers. However, one advantage of the model derived from the CMS sample is that it can be applied to other MA populations to generate actual-to-predicted ratios specific to those populations.
Conclusion
Overall, this study showed an association between enrollment in the insurer’s MA plan and reduced mortality after adjustment for demographics and morbidity. The differences between actual and predicted mortality were particularly pronounced in low income, nonwhite race, and HMO plan type subgroups. These findings suggest that differences in morbidity do not fully explain differences in mortality between the 2 programs. Findings should be interpreted with some caution because of the previously acknowledged study limitations.
Additional research is needed to more directly assess the effect of MA on mortality. Longitudinal cohort studies tracking concurrent MA and FFS Medicare groups for several years are needed. The apparent greater effect of MA in subpopulations with more chronic disease merits further exploration to identify the operant factors. The impact of duration of enrollment on mortality differences might also be explored to better assess the plausibility of a causal relationship between MA participation and reduced mortality. Methods for assessing the effect of selection bias are needed. As the present study applied to a single MA provider’s population, similar analyses in other MA populations are needed to help establish the effect of MA enrollment.
Footnotes
Acknowledgements
The authors wish to acknowledge the very helpful editorial contributions of Laura Happe, PharmD, MPH.
Authors’ Note
At the time the manuscript was completed, Dr Meredith C. Williams was in the Office of the Chief Medical Officer, Humana Inc. However, she is currently in Contracting Negotiations at Humana Inc. In addition, Mr Sean M. Mendes now works in the Actuarial—Corporate Division. The results of the research described in this article were previously presented in the Poster Sessions of the 2016 Annual Meeting of the American College of Preventive Medicine, held February 24-27, 2016, in Washington, DC.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The authors declare that there are no conflicts of interest other than employment of some of the authors (RB, SM, AC, TR, VO, MW) by the Medicare Advantage provider.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
