ANNUAL RWJF BRIEF • OCTOBER 2015 Comparing Federal Government Surveys That Count the Uninsured: 2015 Introduction Timely and accurate estimates of the number of people who do not have health insurance coverage are important for understanding trends in health insurance coverage and the impacts of policy changes that affect health insurance. This brief provides an annual update to comparisons of uninsurance estimates from five federal surveysi: • The American Community Survey (ACS) • The Current Population Survey (CPS) • The Medical Expenditure Panel Survey – Household Component (MEPS-HC) • The National Health Interview Survey (NHIS) • The Behavioral Risk Factor Surveillance System (BRFSS) In this brief, we present current and historical national estimates of uninsurance along with the most recent available state-level estimates from these surveys. We also discuss the main reasons for variation in the estimates across the different surveys. National Estimates Table 1 shows the most recent available estimates of uninsurance from each of the five surveys. Some of the surveys produce estimates of the number of adults who were uninsured for an entire year, while others estimate uninsurance at a specific point in time (i.e., at the time of the survey), and some collect multiple measures of uninsurance. TABLE 1. COMPARISON OF NATIONAL UNINSURANCE ESTIMATES: FIVE FEDERAL SURVEYS Survey Time Period Uninsured for the Entire Year Uninsured at a Specific Point in Time Number (millions) Percent of Population Number (millions) Percent of Population Total Population ACS 2014 N/A N/A 36.7 11.7% CPS 2014 33.0 10.4% N/A N/A MEPS 2013 40.5 12.8% N/A N/A NHIS 2014 26.3 08.4% 36.0 11.5% Nonelderly Adults (Ages 18 to 64) ACS 2014 N/A N/A 31.9 16.3% CPS 2014 27.9 14.2% N/A N/A MEPS 2013 36.8 18.8% N/A N/A NHIS 2014 23.9 12.3% 31.7 16.3% BRFSS 2014 N/A N/A 33.4 17.2% Sources: CPS estimates from U.S. Census Bureau, 2015, “Health Insurance Coverage in the United States: 2014”; ACS estimates for civilian noninstitutionalized population from U.S. Census Bureau, 2015, “Health Insurance Coverage in the United States: 2014” and American Fact Finder, accessed September 18, 2015; NHIS estimates from Cohen and Martinez, 2015, “Health Insurance Coverage: Early Release of Estimates from the National Health Interview Survey, 2014”; MEPS estimates from http://meps.ahrq.gov/mepsweb/ data_stats/summ_tables/hc/hlth_insr/2013/alltablesfy.pdf. BRFSS estimates analyzed by SHADAC using 2014 public use file. i See Appendix A for key information from each of these surveys, such as who is included in the survey, when and how the survey is conducted, response rates, and the availability of state-level insurance estimates. | STATE HEALTH ACCESS DATA ASSISTANCE CENTER 1 Comparing Federal Government Surveys that Count the Uninsured: 2015 National Trends State-Level Estimates The uninsurance estimates from the five surveys have The ACS, CPS, NHIS, and BRFSS are designed to demonstrated similar national trends over time, as produce state-level estimates for all 50 states and shown in Figure 1. See Appendix B for information on the District of Columbia. No state-level estimates of historical changes to the BRFSS and CPS that affect insurance coverage are published from the MEPS-HC. trend analyses. FIGURE 1. TREND IN NATIONAL NUMBER OF UNINSURED, 2000 to 2014 ACS, NHIS, and BRFSS point-in-time estimates of the uninsured; CPS and MEPS estimates of the full-year uninsured All Ages 60 50 Number Uninsured (Millions) * Dashed line 40 “---” indicates a ACS 30 break in series. CPS* MEPS 20 NHIS 10 0 Nonelderly Adults (Ages 18 to 64) 50 Number Uninsured (Millions) 45 40 35 Sources: ACS 30 CPS estimates from U.S. BRFSS* Census Bureau, 2015, 25 20 CPS* “Health Insurance Cover- age in the United States: 15 MEPS 2014”; ACS estimates for 10 NHIS civilian noninstitutionalized 5 population from U.S. Census 0 Bureau, 2015, “Health Insurance Coverage in the United States: 2014” and American Fact Finder, Children (Ages 0 to 17) accessed September 18, 10 2015; NHIS estimates from Cohen and Martinez, 2015, Number Uninsured (Millions) 9 8 “Health Insurance Coverage: Early Release of Estimates 7 from the National Health 6 ACS Interview Survey, 2014”; 5 CPS* MEPS estimates from http:// 4 meps.ahrq.gov/mepsweb/ MEPS 3 data_stats/summ_tables/hc/ NHIS hlth_insr/2013/alltablesfy. 2 1 pdf. BRFSS estimates 0 analyzed by SHADAC using 2014 public use file. 2 | STATE HEALTH ACCESS DATA ASSISTANCE CENTER Comparing Federal Government Surveys that Count the Uninsured: 2015 Table 2 presents the most recent state-level estimates uninsurance for states varies across surveys; however, of uninsurance from the ACS, CPS, NHIS, and BRFSS. general patterns are consistent, insofar as states with As with the national estimates, the estimated level of low uninsurance rates are low in all the surveys. TABLE 2. 2014 STATE-LEVEL UNINSURED RATES FROM FOUR FEDERAL SURVEYS Total Population Non-Elderly Adults (Ages 18 to 64) ACS CPS NHIS ACS CPS BRFSS NHIS (Point-in-Time) (Full Year) (Point-in-Time) (Point-in-Time) (Full Year) (Point-in-Time) (Point-in-Time) United States 11.7 10.4 11.5 16.3 14.2 17.2 16.3 Alabama 12.1 11.0 9.8 18.2 15.5 17.5 14.8 Alaska 17.2 14.5 19.4 21.8 17.6 17.1 24.6 Arizona 13.6 12.4 14.6 18.5 16.0 17.9 19.5 Arkansas 11.8 9.9 10.9 17.5 14.8 20.2 15.6 California 12.4 10.1 12.0 17.3 13.7 17.8 16.7 Colorado 10.3 11.2 9.5 13.9 14.5 15.3 13.3 Connecticut 6.9 7.0 7.0 9.5 9.8 10.6 10.0 Delaware 7.8 6.8 4.4 10.5 8.9 10.8 6.0** District of Columbia 5.3 6.4 3.0 6.7 8.1 9.6 * Florida 16.6 14.6 15.3 23.8 20.3 22.8 23.0 Georgia 15.8 15.5 14.2 22.1 21.7 25.0 20.2 Hawaii 5.3 5.3 2.0** 7.2 7.3 9.8 * Idaho 13.6 10.5 13.3 19.3 14.4 20.2 21.9 Illinois 9.7 8.9 10.3 14.0 12.5 14.3 15.0 Indiana 11.9 10.6 12.0 16.4 14.6 18.0 18.3 Iowa 6.2 6.2 5.6 8.8 8.1 9.6 8.4 Kansas 10.2 10.8 9.0 14.5 14.5 17.6 13.9 Kentucky 8.5 6.7 10.9 11.9 9.1 12.2 15.6 Louisiana 14.8 12.9 12.9 21.6 19.1 22.7 18.9 Maine 10.1 9.4 11.3 14.2 13.2 13.9 16.9 Maryland 7.9 5.8 7.9 11.0 8.0 11.0 12.3 Massachusetts 3.3 4.4 2.6 4.5 5.3 5.4 3.8** Michigan 8.5 7.0 8.0 12.3 9.9 12.7 11.6 Minnesota 5.9 6.7 5.7 7.9 8.5 8.8 8.0 Mississippi 14.5 12.3 14.9 21.5 17.7 23.0 22.4 Missouri 11.7 8.8 12.4 16.1 12.5 16.1 16.9 Montana 14.2 12.6 11.2 20.1 17.4 16.1 18.0 Nebraska 9.7 9.6 11.2 13.6 13.6 15.3 16.9 Nevada 15.2 12.6 15.0 20.4 15.9 20.8 20.4 New Hampshire 9.2 7.2 8.0 12.8 10.0 14.2 11.6 New Jersey 10.9 10.7 9.4 15.4 14.5 15.0 12.9 New Mexico 14.5 11.6 11.3 20.7 16.4 18.8 18.7 New York 8.7 7.7 9.4 12.3 10.4 14.6 12.9 North Carolina 13.1 11.8 14.8 19.0 15.6 20.0 22.5 North Dakota 7.9 8.7 6.0 9.8 11.3 10.5 9.3 Ohio 8.4 7.4 7.6 11.6 10.4 12.5 10.9 Oklahoma 15.4 15.6 18.1 21.6 20.2 17.2 26.6 Oregon 9.7 7.8 8.8 13.9 11.6 13.5 13.3 Pennsylvania 8.5 8.1 7.9 11.7 11.0 12.7 11.9 Rhode Island 7.4 5.4 6.4 10.4 7.1 9.8 9.0 OCTOBER 2015 | www.shadac.org 3 Comparing Federal Government Surveys that Count the Uninsured: 2015 TABLE 2. 2014 STATE-LEVEL UNINSURED RATES FROM FOUR FEDERAL SURVEYS Total Population Non-Elderly Adults (Ages 18 to 64) ACS CPS NHIS ACS CPS BRFSS NHIS (Point-in-Time) (Full Year) (Point-in-Time) (Point-in-Time) (Full Year) (Point-in-Time) (Point-in-Time) South Carolina 13.6 12.9 14.5 19.9 18.1 21.3 21.0 South Dakota 9.8 9.2 8.5 13.8 12.3 12.2 13.4 Tennessee 12.0 9.6 10.8 17.4 13.0 17.7 14.8 Texas 19.1 16.9 19.4 25.7 23.1 29.2 25.7 Utah 12.5 11.6 12.9 16.2 14.6 16.2 16.2 Vermont 5.0 5.8 8.6 7.1 7.7 8.4 9.1 Virginia 10.9 9.9 10.8 14.9 13.2 15.8 15.2 Washington 9.2 9.1 9.8 12.8 12.2 12.9 13.3 West Virginia 8.6 6.5 7.6 12.9 9.4 13.0 12.2 Wisconsin 7.3 7.3 6.3 10.1 9.9 10.4 8.7 Wyoming 12.0 10.2 10.9 16.7 13.4 18.5 15.2 * The NHIS does not release estimates with a relative standard error greater than 50%. **The NHIS recommends using estimates from these states with caution because of their relative standard error between 30% and 50%. Sources: ACS estimates for civilian noninstitutionalized population from U.S. Census Bureau, 2015, “Health Insurance Coverage in the United States: 2014” and Amer- ican Fact Finder, accessed September 18, 2015; CPS estimates from U.S. Census Bureau, 2015, CPS Table Creator, Accessed October 12, 2015; NHIS estimates from Cohen and Martinez, 2015, “Health Insurance Coverage: Early Release of Estimates from the National Health Interview Survey, 2014”; BRFSS estimates analyzed by SHADAC using 2014 public use file. Factors Contributing to Differences in The CPS Annual Social and Economic Supplement, conducted in February through April each year, has his- Survey Estimates torically asked respondents about their health insurance There are many reasons why health insurance estimates coverage during the entire previous calendar year, with vary across surveys. The surveys are designed to fulfill respondents being asked to report their coverage for a different goals and use different questions, statistical time period as long as 16 months prior to the interview. designs, and data collection and processing methods. Beginning with data collected in 2013, the CPS asks Each of these factors likely contributes to differences in respondents about their current coverage status (point uninsurance estimates. The following section articulates in time), as well as the current year-to-date and previous specific differences between the surveys included in this calendar year (up to 16 months). For their measures of brief. coverage during the prior year, NHIS and MEPS have shorter recall periods than the CPS. The ACS and Conceptual differences in measures of uninsurance BRFSS collect information about current coverage only. As noted earlier, some surveys collect information about whether a person lacked health insurance coverage for a Differences in survey questions full year, while others collect information on point in time Differences in the ways that health insurance ques- insurance status, and some collect multiple measures. tions are asked can lead to differences in uninsurance estimates. For example, when the Census Bureau added Reference period a “verification question” to the CPS in 2000 that asked Differences in the time period for which coverage is people who did not report any coverage if they were in being reported contribute to differences in the survey fact uninsured for all of 1999, the estimated number of estimates; in addition, differences in the length of time people without health insurance declined by 8 percent, that respondents are being asked to recall their insur- from 42.6 million to 39.3 million.6 The CPS, NHIS, and ance coverage status can also result in differences in MEPS all verify insurance status for people who do not measurement error across the surveys.1,2,3,4,5 report any of the specific types of coverage that the 4 | STATE HEALTH ACCESS DATA ASSISTANCE CENTER Comparing Federal Government Surveys that Count the Uninsured: 2015 survey asks about, but the ACS and BRFSS do not. the number of uninsured. Each survey provides a unique Another difference in survey questions that can lead to view of the problem of uninsurance, and together the different estimates across surveys is the fact that some surveys provide a wealth of information about how of the surveys (CPS, NHIS, and MEPS) use state-spe- uninsurance varies by population characteristics and cific names for Medicaid and Children’s Health Insur- how it is associated with differences in access to and ance Program (CHIP) programs, while the ACS and use of health care services and with health status. BRFSS do not. Missing data and imputation Suggested Citation Some of these surveys have processes in place to Planalp, C., Au-Yeung, C., & Turner, J. “Comparing Fed- manage missing data and impute missing values. In the eral Government Surveys That Count the Uninsured: CPS supplement that includes the health insurance 2015.” Annual Robert Wood Johnson Foundation Brief. questions, about 10 percent of the respondents do Minneapolis, MN: State Health Access Data Assistance not answer any questions, and the missing values are Center. imputed by the Census Bureau. Similarly, in the 2008 ACS about 11 percent of responses had one or more About SHADAC of the health insurance items missing; these missing SHADAC is a multidisciplinary health policy research data were imputed by the Census Bureau. In contrast, center located at the University of Minnesota School of the NHIS and MEPS impute little or no health insurance Public Health. SHADAC is a resource for helping states coverage, because the data are much more complete collect and use data for health policy, with a particular than the CPS or ACS. The BRFSS does not impute focus on monitoring rates of health insurance coverage health insurance coverage. and understanding factors associated with uninsurance. Conclusion For more information, please visit us at www.shadac. org or contact us at shadac@umn.edu. Federal surveys are essential resources for estimating REFERENCES 1 Klerman J.A., Ringel, J.S., & Roth, B. 2005. Under-reporting of Medicaid and welfare in the Current Population Survey. Working Paper. Santa Monica CA: RAND, March 2005. 2 Short, P.F. 2001. Counting and characterizing the uninsured. Working Paper Series. Ann Arbor MI: Economic Research Initiative on the Uninsured, December 2001. 3 Sudman, S., Bradburn, N. & Schwarz, S. 1996. Thinking about Answers. San Francisco: Jossey-Bass. 4 Bhandari, S. 2004. People with health insurance: A comparison of estimates from two surveys. Working Paper No. 243. Washing- ton DC: U.S. Census Bureau, June 2004. Available at: http://www.census.gov/dusd/MAB/wp243.pdf 5 Lewis, K, MR Elwood, and J Czajka. 1998. Counting the uninsured: A review of the literature. Washington DC: The Urban Institute, July 1998. 6 Nelson, C.T. and R.J. Mills. 2001. “The March CPS Health Insurance Verification Question and Its Effect on Estimates of the Unin- sured.” 2001 Proceedings of the Section on Survey Research Methods, Alexandria, VA: the American Statistical Association. OCTOBER 2015 | www.shadac.org 5 Comparing Federal Government Surveys that Count the Uninsured: 2015 Appendix A COMPARISON OF FEDERAL SURVEYS USED TO ESTIMATE UNINSURANCE ACS CPS MEPS-HC NHIS BRFSS Sponsor(s) Census Bureau Bureau of Labor Agency for Healthcare National Center for Centers for Disease Statistics, U.S. Dept. of Research & Quality Health Statistics, Control and Prevention Labor (conducted by the (conducted by Census Centers for Disease (conducted by states) Census Bureau) Bureau) Control and Prevention) Primary Focus General household Labor force participation Health care access, Population health Population health, risk survey; replaced and unemployment utilization, and cost factors, and health decennial census long behaviors form Target Entire population Civilian non- Civilian non- Civilian non- Adult civilian non- Population institutionalized institutionalized institutionalized institutionalized population population population population Sample Frame Address-based (National Address-based (Census NHIS respondents Address-based (Census Telephone-based Master Address File) 2010 sampling frame 2000 sampling frame (households with updated with new updated with new landline telephones, plus construction) construction) cell phones added in the 2011 survey) Data Collection Mail; in-person; phone; In-person; phone In-person In-person Phone Mode internet Type of Point-in-time All of prior calendar year: Point-in-time; all of prior Point-in-time; all of prior Point-in-time; uninsured Uninsurance point-in-time (added in year; if uninsured, length year; if uninsured, length at some point in the Measures 2014) of time uninsured; of time uninsured; past year (an optional uninsured at some point uninsured at some point question adopted by 38 in the past year in the past year states and D.C. in 2013) Health Insurance No Yes Yes Yes No Coverage: Verification Question for Uninsured State-Specific No Yes Yes Yes No Names Included for Medicaid/ CHIP Response Rate 96.7% (2014) 74.3% (2014) 58.0% (2013) 73.8% (2014) 47.0% (2014 combined landline/cell phone median reponse rate for states) Survey Period Monthly February through April Panel over two calendar February, May, August, Monthly years November State Health 50 states and D.C. 50 states and D.C. Not published 50 states and DC 50 states and DC Insurance Estimates Years Available 2008 to 2014 1987 to 2014 (plus 1996 to 2013 1998 to 2014 (plus first 1991 to 2014 limited point-in-time quarter of 2015) estimates for 2015) 6 | STATE HEALTH ACCESS DATA ASSISTANCE CENTER Comparing Federal Government Surveys that Count the Uninsured: 2015 Appendix B Within-Survey Changes Over time: Questions & Methodology In the same way that estimates across different surveys may not be comparable, estimates within the same survey may not always be comparable over time. This incomparability can be due to changes in survey questions and/or changes in survey methodology. Changes in the BRFSS In 2011, the BRFSS began using a new sampling frame, adding cell phones to the landlines it had historically sampled. The purpose of this revision was to capture the growing segment of the U.S. population that uses cell phones exclusively so that the survey estimates would more closely reflect the overall population.i Because of this methodological change, the CDC advises against comparing BRFSS estimates from 2011 onward against those from 2010 and earlier.ii Changes in the CPS In 2014, the CPS incorporated a revised set of survey questions designed to improve the accuracy of its uninsurance estimate, which researchers have suggested more closely resembled a point-in-time measure than a measure of insurance coverage during the previous year (as was intended).iii,iv,v Data from 2013 and onward are not comparable to data from 2012 and earlier. The CPS has made other revisions that have created a break in its time series. In 2000, the survey added an insurance verification question, asking people who did not report coverage whether they were, in fact, uninsured. vi This change improved the accuracy of the CPS’s uninsurance estimate by allowing respondents to confirm their coverage status, but it also caused a break in the comparability of CPS estimates from 1998 and earlier versus 1999 and later.vii The CPS has also made methodological changes that could affect the comparability of its estimates over time, such as changes in the weighting of data.viii i U.S. Centers for Disease Control and Prevention. 2011. “Comparability of Data: BRFSS 2011.” Available at: http://www.cdc.gov/ brfss/annual_data/2011/compare_11_20121212.pdf U.S. Centers for Disease Control and Prevention. 2013. “Methodologic Changes in the Behavioral Risk Factor Surveillance System in ii 2011 and Potential Effects on Prevalence Estimates.” Available at: http://www.cdc.gov/surveillancepractice/reports/brfss/brfss.html Turner, J, Boudreaux, M. 2014. “An Introduction to Redesigned Health Insurance Coverage Questions in the 2014 CPS.” SHADAC iii Brief #39. Minneapolis, MN: State Health Access Data Assistance Center. Available at: http://www.shadac.org/publications/cpsbrief Planalp, C., Sonier, J., Turner, J. 2014. “Using Recent Revisions to Federal Surveys for Measuring the Effects of the Affordable Care iv Act.” Issue Brief #41. Minneapolis, MN: State Health Access Data Assistance Center, University of Minnesota. Available at: http:// www.shadac.org/publications/using-recent-revisions-federal-surveys-measuring-effects-affordable-care-act v Davern M., G. Davidson, J. Ziegenfuss, et al. 2007. “A Comparison of the Health Insurance Coverage Estimates from Four National Surveys and Six State Surveys: A Discussion of Measurement Issues and Policy Implications.” Final report for U.S. DHHS Assistant Secretary for Planning and Evaluation, Task 7.2. Minneapolis, MN: University of Minnesota. Available at: http://www.shadac.org/files/ sha-dac/publications/ASPE_FinalRpt_Dec2007_Task7_2_rev.pdf vi Turner, J., Boudreaux, M. 2012. “Implementation of Improvements to the Allocation Routine for Health Insurance Coverage in the OCTOBER 2015 | www.shadac.org 7 Comparing Federal Government Surveys that Count the Uninsured: 2015 CPS ASEC. Minneapolis, MN: State Health Access Data Assistance Center, University of Minnesota. Available at: http://fcsm.sites. usa.gov/files/2014/05/Turner_2012FCSM_I-C.pdf To improve the comparability of CPS estimates of uninsurance over time, the State Health Access Data Assistance Center (SHA- vii DAC) has created an enhanced time series of CPS data, available at: http://datacenter.shadac.org. The enhanced time series is available from 1987 to 2012. It does not bridge the re-design introduced in 2013. State Health Access Data Assistance Center. 2010. “Comparing State Uninsurance Estimates: SHADAC-Enhanced CPS and viii CPS.” Issue Brief #21. Minneapolis, MN: University of Minnesota. Available at: http://www.shadac.org/files/shadac/publications/ IssueBrief21.pdf Bridging the gap between research and policy @ www.shadac.org Funded by a grant from The Robert Wood Johnson Foundation ©2015 Regents of the University of Minnesota. All rights reserved. The University of Minnesota is an Equal Opportunity Employer.