Abstract
This study randomized a sample of households covered by one large health plan to two different surveys on health insurance coverage and matched person-level survey reports to enrollment records. The goal was to compare accuracy of coverage type and uninsured estimates produced by the health insurance modules from two major federal surveys – the redesigned Current Population Survey Annual Social and Economic Supplement (CPS) and the American Community Survey (ACS) – after implementation of the Affordable Care Act. The sample was stratified by coverage type, including two types of public coverage (Medicaid and a state-sponsored program) and three types of private coverage (employer-sponsored, non-group, and marketplace plans). Consistent with previous studies, accurate reporting of private coverage is higher than public coverage. Generally, misreporting the wrong type of coverage is more likely than incorrectly reporting no coverage; the CPS module overestimated the uninsured by 1.9 and the ACS module by 3.5 percentage points. Other differences in accuracy metrics between the CPS and ACS are relatively small, suggesting that reporting accuracy should not be a factor in decisions about which source of survey data to use. Results consistently indicate that the Medicaid undercount has been substantially reduced with the redesigned CPS.
References
1.
American Association of Public Opinion Research. 2016. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 9th edition. Available at: https://www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions20169theditionfinal.pdf (accessed March 2019).
2.
Bennefield, R.L. 1996. Dynamics of Economic Well-Being: Health Insurance, 1992 to 1993, Who Loses Coverage and for How Long? Available at: https://www.census.gov/prod/1/pop/p70-54.pdf (accessed March 2019).
3.
Blewett, L.A. and M. Davern. 2006. “Meeting the Need for State-Level Estimates of Health Insurance Coverage: Use of State and Federal Survey Data.” Health Services Research 41(3p1): 946–975. Doi: https://doi.org/10.1111/j.1475-6773.2006.00543.x.
4.
Blumberg, S.J. and M.L. Cynamon. 1999. Misreporting Medicaid Enrollment: Results of three studies linking telephone surveys to state administrative records. Available at: https://www.cdc.gov/nchs/data/hsrmc/hsrmc_7th_proceedings_1999.pdf (accessed March 2019).
5.
Blumberg, S.J., L. Osborn, J.V. Luke, L. Olson, and M.R. Frankel. 2004. “Estimating the prevalence of uninsured children: an evaluation of data from the National Survey of Children with Special Health Care Needs, 2001.” Vital and Health Statistics. Series 2, (136): i–38. Available at: https://www.cdc.gov/nchs/data/series/sr_02/sr02_136.pdf (accessed March 2019).
6.
Call, K.T., M.E. Davern, J.A. Klerman, and V. Lynch. 2013. “Comparing errors in Medicaid reporting across surveys: Evidence to date.” Health Services Research 48(2 PART1): 652–664. Doi: https://doi.org/10.1111/j.1475-6773.2012.01446.x.
7.
Call, K.T., A.R. Fertig, J. Pascale, and D. Oellerich. 2018. Who gets it right? Characteristics associated with accurate reporting of health insurance coverage. In Paper presented at the Academy Health Annual Research Meeting, Seattle, WA. U.S.A. June 25, 2018. Available at: https://www.academyhealth.org/events/2018-06/2018-annual-research-meeting (accessed March 2019).
8.
Cantor, J.C., A.C. Monheit, S. Brownlee, and C. Schneider. 2007. “The adequacy of household survey data for evaluating the nongroup health insurance market.” Health Services Research 42(4): 1739–1757. Doi: https://doi.org/10.1111/j.1475-6773.2006.00662.x.
9.
Czajka, J.L. and K. Lewis. 1999. Using National Survey Data to Analyze Children’s Health Insurance Coverage: An Assessment of Issues. Available at: https://www.mathematica-mpr.com/our-publications-and-findings/publications/using-national-surveydata-to-analyze-childrens-health-insurance-coverage-an-assessment-of-issues (accessed March 2019).
10.
Davern, M., K.T. Call, J. Ziegenfuss, G. Davidson, T.J. Beebe, and L. Blewett. 2008. “Validating health insurance coverage survey estimates: A comparison of self-reported coverage and administrative data records.” Public Opinion Quarterly 72(2): 241–259. Doi: https://doi.org/10.1093/poq/nfn013.
11.
Eberly, T., M.B. Pohl, and S. Davis. 2009. “Undercounting Medicaid enrollment in Maryland: Testing the accuracy of the current population survey.” Population Research and Policy Review 28(2): 221–236. Doi: https://doi.org/10.1007/s11113-008-9078-5.
12.
Farley-Short, P. 2001. Counting and Characterizing the Uninsured. Available at: http://rwjf-eriu.org/pdf/farleyshort-final.pdf (accessed March 2019).
13.
Fertig, A.R., J. Pascale, K.T. Call, and D. Oellerich. 2018. Design and Sampling Strategy for a Validation Study Linking Enrollment Records to Survey Reports: the CHIME Study. Available at: https://www.census.gov/content/dam/Census/library/workingpapers/2018/adrm/rsm2018-10.pdf (accessed December 2018).
14.
Hill, S.C. 2007. “The Accuracy of Reported Insurance Status in the MEPS.” Inquiry 44(4): 443–468. Doi: https://doi.org/10.5034/inquiryjrnl_44.4.443.
15.
Klerman, J.A., M. Davern, K.T. Call, V. Lynch, and J.D. Ringel. 2009. “Understanding The Current Population Survey’s Insurance Estimates And The Medicaid ‘Undercount’.” Health Affairs 28(6): w991–w1001. Doi: https://doi.org/10.1377/hlthaff.28.6.w991.
16.
Lewis, K., M.R. Ellwood, and J.L. Czajka. 1998. Counting the Uninsured: A Review of the Literature. Available at: https://www.urban.org/research/publication/countinguninsured (accessed March 2019).
17.
Lurie, I.Z. and J. Pearce. 2018. Health Insurance Coverage from Administrative Tax Data. Available at: https://www.treasury.gov/resource-center/tax-policy/Pages/tax_analysis_paper.aspx (accessed May 2019).
18.
Mach, A. and B. O’Hara. 2011. Do people really have multiple health insurance plans? Estimates of Nongroup Health Insurance in the American Community Survey. Available at: https://www.census.gov/content/dam/Census/library/workingpapers/2011/demo/SEHSD-WP2011-28.pdf (accessed March 2019).
19.
Marquis, M.S. 1983. “Consumers’ Knowledge about their Health Insurance Coverage.” Health Care Financ Rev 5(1): 65–80. Available at: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4191335/ (accessed May 2019).
20.
Minnesota Department of Health. 2018. Section 4: Individual and Small Group Health Insurance Markets – Chart Summaries. Available at: https://www.health.state.mn.us/data/economics/chartbook/summaries/section4summaries.html (accessed March 2019).
21.
Nelson, D.E., B.L. Thompson, N.J. Davenport, and L.J. Penaloza. 2000. “What people really know about their health insurance: A comparison of information obtained from individuals and their insurers.” American Journal of Public Health 90(6): 924–928. Doi: https://doi.org/10.2105/AJPH.90.6.924.
22.
Noon, J.M., L.E. Fernandez, and S.R. Porter. 2019. “Response error and the Medicaid undercount in the current population survey.” Health Services Research , 54(1): 34–43. Doi: https://doi.org/10.1111/1475-6773.13058.
23.
Pascale, J. 2008. “Measurement Error in Health Insurance Reporting.” Inquiry 45(4): 422–437. Doi: https://doi.org/10.5034/inquiryjrnl_45.04.422.
24.
Pascale, J. 2009. Health Insurance Measurement A Synthesis of Cognitive Testing Findings. In Questionnaire Evaluation Standards (QUEST) Meeting, May 18–20 . Bergen, Norway. Available at: https://wwwn.cdc.gov/qbank/QUest/2009/pres10.pdf (accessed March 2019).
25.
Pascale, J. 2016. “Modernizing a major federal government survey: A Review of the redesign of the current population survey health insurance questions.” Journal of Official Statistics 32(2): 461–486. Doi: https://doi.org/10.1515/JOS-2016-0024.
26.
Pascale, J., M. Boudreaux, and R. King. 2016. “Understanding the New Current Population Survey Health Insurance Questions.” Health Services Research 51(1): 240–261. Doi: https://doi.org/10.1111/1475-6773.12312.
27.
Pascale, J., K.T. Call, and A.R. Fertig. 2018a. Using a Machine Learning Approach to Classify Health Insurance Type from Survey Responses Using Enrollment Data. In Paper presented at the AcademyHealth Annual Research Meeting, Seattle, WA.U.S.A. June 25, 2018. Available at: https://www.academyhealth.org/events/2018-06/2018-annual-research-meeting (accessed March 2019).
28.
Pascale, J., K.T. Call, and A.R. Fertig. 2018b. Using Enrollment Records to Guide Categorization of Health Insurance Coverage Type Post-ACA. In Paper presented at the AcademyHealth Annual Research Meeting, Seattle, WA.U.S.A. June 25, 2018. Available at: https://www.academyhealth.org/events/2018-06/2018-annual-researchmeeting (accessed May 2019).
29.
Pascale, J., J. Rodean, L. Leeman, C. Cosenza, and A. Schoua-Glusberg. 2013. “Preparing to Measure Health Coverage in Federal Surveys Post-Reform.” Inquiry 50(2): 106–123. Doi: https://doi.org/10.1177/0046958013513679.
30.
Pascale, J., M.I. Roemer, and D.M. Resnick. 2009. Medicaid underreporting in the CPS: Results from a Record Check Study. Public Opinion Quarterly 73(3): 497–520. Doi. https://doi.org/10.1093/poq/nfp028.
31.
Rosenbach, M. and K. Lewis. 1998. Estimates of Health Insurance Coverage in the Community Tracking Study and the Current Population Survey. Available at: https://www.researchgate.net/publication/267196753_Estimates_of_Health_Insurance_Coverage_in_the_Community_Tracking_Study_and_the_Current_Population_Survey (accessed March 2019).
32.
Swartz, K. 1986. “Interpreting the Estimates from Four National Surveys of the Number of People Without Health Insurance.” Journal of Economic and Social Measurement 14(3): 233–242. Doi: https://doi.org/10.3233/JEM-1986-14306.
33.
U.S. Census Bureau. 2016a. Table DP05 ACS Demographic and Housing Estimates. Available at: http://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml?src=CF (accessed June 2016).
34.
U.S. Census Bureau. 2016b. Table S1201 Educational Attainment. Available at: http://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml?src=CF (accessed June 2016).