OFFICE OF RESEARCH REPORT HEALTH POLICY October 5, 2021 ee ee Associations Between County-level Vaccination Rates and COVID-19 Outcomes Among Medicare Beneficiaries ASSISTANT SECRETARY FOR PLANNING AND EVALUATION _ ASPE Analysis of Medicare data and county vaccination rates indicates that COVID-19 vaccinations from January until May 2021 were associated with estimated reductions of approximately 265,000 COVID-19 infections and 39,000 deaths among Medicare beneficiaries. Lok Wong Samson, Wafa Tarazi, E. John Orav, Steven Sheingold, Nancy De Lew, and Benjamin D. Sommers KEY POINTS e COVID-19 vaccines are a key component in controlling the COVID-19 pandemic. Clinical data show vaccines are highly effective in preventing COVID-19 infections and severe outcomes including hospitalization and death. e = In this analysis of individual-level health data and county-level vaccination rates, we find that higher county vaccination rates were associated with significant reductions in the odds of COVID-19 infection, hospitalization, and death among Medicare fee-for-service (FFS) beneficiaries between January and May 2021. e Comparing the rates of these outcomes to what our model predicts would have happened without any vaccinations, we estimate COVID-19 vaccinations were linked to estimated reductions of approximately 107,000 infections, 43,000 hospitalizations, and 16,000 deaths in our study sample of 25.3 million beneficiaries. These estimates correspond to estimated reductions of approximately 265,000 infections, 107,000 hospitalizations, and 39,000 deaths for the full Medicare population of 62.7 million people. e After accounting for the potential underreporting of COVID-19 deaths in Medicare FFS claims data, and considering alternative models, the number of deaths prevented among the full Medicare population could plausibly range from 12,000 to 49,000 deaths. e Reductions in cumulative weekly deaths were found nationally, for all racial and ethnic groups, and across all 48 states included in our sample. e The difference in vaccination rates for those age 65 and older between the lowest (34%) and highest (85%) counties and states by the end of May highlights the continued opportunity to leverage COVID-19 vaccinations to prevent COVID-19 hospitalizations and deaths. OCTOBER 2021 RESEARCH REPORT 1 BACKGROUND The United States introduced COVID-19 vaccinations starting in December 2020 after 20 million people had been infected and 352,000 lives were lost over the first 9 months of the COVID-19 pandemic.* Nearly 80 percent of these deaths were estimated to be among persons 65 and older who are also Medicare eligible based on age." Medicare beneficiaries therefore are a high-risk group who are likely to see significant benefit from COVID-19 vaccinations. Understanding how the initial U.S. COVID-19 vaccination effort potentially reduced COVID-19 infections, hospitalizations, and deaths in the Medicare population can help inform continued efforts to improve vaccination rates and mitigate the harms from COVID-19. COVID-19 vaccines were first made available to health care workers and nursing home residents; states then rolled out vaccines with different timelines and priority groups, but most prioritized seniors ages 65 and older. COVID-19 vaccines were made available without charge to all U.S. residents, including Medicare beneficiaries. Moderna and Pfizer first gained emergency use authorization in December, and both required two doses several weeks apart. Johnson and Johnson's single dose vaccine also received emergency use authorization and became available in February 2021. Initial clinical studies showed high efficacy of vaccines in preventing severe outcomes from COVID-19 and continued protection against the emerging Delta variants, although effectiveness may be somewhat lower.** The purpose of this study is to identify associations between COVID-19 infections, hospitalizations, and deaths among Medicare fee- for-service (FFS) beneficiaries and the proportion of the population fully vaccinated at the county-level between January and May 2021. We do so by using a combination of person-level Medicare FFS claims and Centers for Disease Control and Prevention (CDC) data on county-level vaccination rates. Based on these results, we then estimated the net reduction in infections, hospitalizations, and deaths among all 62.7 million Medicare beneficiaries associated with the first 5 months of the U.S.'s COVID-19 vaccine roll-out. METHODS Data and Study Period The study period for developing the estimation model was based on Medicare FFS claims from September 6, 2020, to May 29, 2021 to capture the risk of COVID-19 outcomes both before vaccinations were available and with COVID-19 vaccinations from January-May 2021. The latter period was then used to calculate the difference in expected number of COVID-19 outcomes compared to if no vaccines were available. Data through the end of May 2021 reflect the most complete Medicare FFS claims data currently available from CMS that capture all relevant COVID-19 outcomes. Because Medicare beneficiaries received COVID-19 vaccinations from a variety of sites, many of which may not have generated a claim, Medicare FFS data do not fully capture vaccination status. County-level vaccination data were therefore used to illustrate the association of vaccinations with the estimated reduction of COVID-19 infections and deaths. The study cohort for the regression model includes 25.3 million Medicare FFS who were continuously enrolled in Part A and B for 12 months prior to the study period until they reached an endpoint in the study period (death or disenrollment). The look-back period allows capture of relevant comorbidities. OCTOBER 2021 RESEARCH REPORT 2 Of note, Texas and Hawaii did not provide county-level estimates of vaccinations in the CDC data, so those two states were excluded from the estimation model. This reduced the study cohort from 27.3 million to 25.3 million beneficiaries. Study Design We employed a panel study design and combined individual-level Medicare claims data with weekly cumulative county-level vaccination rates from CDC to estimate the association between county-level COVID-19 vaccinations and weekly changes in COVID-19 outcomes among Medicare beneficiaries over time.? Our three outcomes were COVID-19 infections captured by ICD-10 diagnosis code for COVID-19 (U07.1), hospitalizations within three weeks, and deaths within nine weeks of the initial COVID-19 diagnosis. Deaths in Medicare were identified from updates to the Medicare enrollment data from the Social Security Administration/Railroad Retirement Board, or from discharge codes in facility claims or physician claims indicating death after an ambulance was called. The outcomes are measured weekly in association with the county vaccination rates in the same week. Model Variables The key study predictors were CDC county-level COVID-19 vaccination rates for adults ages 18-64 and those 65 years and older who were fully vaccinated (e.g. received two doses for Pfizer/Moderna and one dose for Johnson and Johnson vaccines). Our model included those measures as separate predictor variables, expressed in terms of cumulative share of the population to date fully vaccinated by the end of each week in the study period. These vaccination rates reflect both the likelihood that Medicare beneficiaries aged 65 and older were vaccinated, and the extent to which people around beneficiaries may be vaccinated, including whether beneficiaries younger than 65 eligible for Medicare due to a disability or end-stage renal disease were vaccinated. The model estimated weekly COVID-19 outcomes during the study period as a function of these variables, with the county vaccination rates as the key independent variables of interest. For each week, we then compared the estimated COVID-19 outcomes to a counterfactual scenario where vaccination rates were zero for all age groups from January to May 2021. The weekly difference between outcomes with and without vaccines was summed for each week from January to May 2021 to estimate the cumulative change nationally in COVID-19 outcomes associated with the introduction of COVID-19 vaccines at the end of December 2020. State- specific estimates were generated by summing the weekly differences for each of the 50 states including Puerto Rico and the District of Columbia. For the two states excluded from the estimation model, Texas and Hawaii, the estimated reduction in COVID-19 outcomes were imputed based on another state with a similar vaccination rate as part of reporting state-specific estimates. The model, adapted from our previously published approach,° adjusted for beneficiary demographic and comorbid characteristics, as well as local area characteristics such as population density and the CDC's Social Vulnerability Index, both at the Census tract level. Beneficiary characteristics included age, sex, race/ethnicity, dual enrollment in Medicaid, long-term nursing home residency (>90 days in a skilled nursing facility), and disability or end-stage renal disease as the original reason for Medicare enrollment. The definition of long-term nursing home resident did not include beneficiaries living in assisted living facilities or other congregate housing. Beneficiary clinical characteristics were based on flags for hierarchical condition categories (HCC) for comorbidities considered relevant to COVID-19 and count of these comorbidities (0,1,2,3,4+) identified in the 12 months prior to the study period (see Appendix for details). The model also included state fixed effects to control for differences in states' COVID-19 OCTOBER 2021 RESEARCH REPORT 3 mitigation efforts and other unmeasured demographic and health care differences across states that were stable over time. Our main model used month fixed effects to adjust for any national temporal trends and fluctuations in COVID-19 outcomes over the course of the study period. In sensitivity analyses, we ran the model with data limited to January - May 2021 and replaced monthly fixed effects with weekly fixed effects to better control for changes in temporal trends. This sensitivity analysis also used categories of community vaccination rates in order to model both temporal trends and vaccination rates over time without a linear assumption (See Appendix for details). In addition to nationwide and state-specific estimates, COVID-19 outcomes were examined by race/ethnicity, and nursing home status given the substantial disparities in COVID-19 outcomes that have been observed during the pandemic.' Projections to the Full Medicare Population We extrapolated the Medicare FFS study sample estimates to the entire Medicare population which includes both Traditional FFS Medicare and beneficiaries enrolled in Medicare Advantage plans. As our sample includes 25.3 million Medicare FFS beneficiaries and the total Medicare population is approximately 62.7 million beneficiaries, we used a factor of 2.48 (the ratio of total Medicare population to study cohort) to inflate the study cohort estimates to the total Medicare population.® We used a factor for each subgroup (based on the ratio of the total Medicare population to each subgroup population) to project our study cohort estimates to the racial/ethnic and nursing home subgroups. We also compared the demographic characteristics of the study cohort with the full Medicare population to check our assumptions and representativeness of the study sample (see Appendix). Detailed methods on the regression models are available in the Appendix. OCTOBER 2021 RESEARCH REPORT 4 FINDINGS Association between cumulative county-level COVID-19 vaccination rates and the risk of COVID-19 outcomes At the start of the COVID-19 vaccine roll-out, weekly US vaccination rates for COVID-19 (e.g., proportion of population fully vaccinated) in January 2021 started from less than 1% in both 18-64 and 65+ age groups, then increased to a cumulative total of 47% and 80% respectively by the last week of May. Table 1 shows the regression results for our key predictor We find an 11-12% variables. These results reflect the independent association . oer decrease in COVID-19 between county-level COVID-19 vaccinations and observed oo, changes in COVID-19 outcomes among Medicare beneficiaries hospitalizations and over time, after controlling for beneficiary and local area deaths for every 10% characteristics, as well as state and month fixed effects to increase in county account for differences between states and temporal trends." vaccination rates, and a For the risk of COVID-19 infection, a 10% increase in COVID-19 similar decrease in vaccination rate among those 65 and older was associated with infections among an 11% decrease in the odds of COVID-19 infection (based on odds ratio [OR]=0.989, 95% confidence interval [Cl] 0.988-0.991, P<0.001), with an estimated reduction between 9-12%. Medicare beneficiaries. For COVID-related hospitalizations and deaths, a 10% increase in COVID-19 vaccinations in those ages 18-64 was associated with approximately an 11% (OR=0.989, 95% Cl 0.982-0.995) and 12% (OR=0.988, 95% Cl 0.978-0.999) decrease in the odds of COVID-19 hospitalizations and deaths, respectively, among Medicare beneficiaries infected with COVID-19 (P<0.05). Table 1. Association between COVID-19 Vaccination Rates and COVID-19 Outcomes, Sept 2020 - May 2021: Regression Odds Ratios [OR] Key Predictors of Risk of COVID-19 Risk of COVID-19 Risk of COVID-19 COVID-19 Outcomes Tavira d(ela pe); 3 hospitalization, OR death, OR (95% Confidence (95% Confidence (95% Confidence Interval) Interval) Interval) Vaccination Rate 18-64 0.997 0.989** 0.988* (0.994, 1.000) (0.982-0.995) (0.978-0.999) Vaccination Rate 65+ 0.989** 1.003 1.002 (0.988-0.991) (1.000-1.006) (0.996-1.007) Notes: *P<0.05. ** P< 0.001. Note: N = 25.3 million Medicare FFS beneficiaries. Models adjusted for beneficiary demographic characteristics, comorbidities, local county characteristics, and state and month fixed effects. Results are reported to 3 decimal places which translates to percentages to 1 decimal place. * Since the absolute risk of each outcome are all small, for simplicity the changes in odds are interpreted as changes in the rate of outcomes. The decrease in odds is calculated as the odds ratio for the vaccination rate minus 1 times 100%, e.g. 0.989-1 *100% = - 0.011 *100% = - 1.1%. A 10% increase in vaccination rate would therefore decrease odds of infection by 11%). OCTOBER 2021 RESEARCH REPORT 5 Estimated Vaccine-Associated Reduction in COVID-19 outcomes from January-May 2021 among Medicare Beneficiaries Figure 1 shows the difference in COVID-19 infections and hospitalizations in our study sample with and without vaccines, based on the association with county-level vaccination rates in the regression models described above. The solid blue, brown and red lines in Figure 1 plot the cumulative weekly number of COVID-19 infections, hospitalizations and deaths estimated by the model based on the observed association between COVID-19 vaccinations and outcomes. The dotted lines plot the COVID-19 outcomes as if no vaccines were available, termed the counterfactual scenario, by modeling zero vaccination rates. As can be seen in Figure 1, without the introduction of vaccines, there would have many more COVID-19 infections among Medicare beneficiaries. The initial roll-out of vaccinations from January-May flattened the curve of COVID-19 infections, hospitalizations and deaths. Figure 1. Comparison of the cumulative weekly number of COVID-19 infections, hospitalizations, and deaths estimated with vaccines vs. counterfactual without vaccines in study cohort, January - May 2021 700,000 z About 107,000 fewer COVID-19 ® 600,000 infections (~18%) compared to if ~- ee x c -- £&a no vaccines available, study cohort _ex- -% t 2 & 500,000 Jan-May 2021 =x £2 x ae & § 400,000 3 Y About 43,000 fewer COVID-19 9§ hospitalizations (~21%) compared g £ 300,000 : : : = 8 to if no vaccines available, study z g 500,000 cohort Jan-May 2021 ~~ o , x= os = ye Ht $100,000 = 1/3/2021 2/3/2021 3/3/2021 4/3/2021 5/3/2021 -#?- Estimated COVID-19 Infections with Vaccines -+- Estimated COVID-19 Hospitalizations with Vaccines = -© = Counterfactual COVID-19 Infections if no Vaccines == @ ~@ Counterfactual COVID-19 Hospitalizations if no Vaccines a 80,000 8 g 70,000 =~ =a xX & 60,000 ~~ -* = 50,000 x - --- = 50 ee om = 40,000 About 16,000 fewer COVID- 3 19 deaths (~22%) compared 3 30,000 to if no vaccines available, 5 study cohort Jan-May 2021 ®& 20,000 5 2 10,000 $ - 1/3/2021 2/3/2021 3/3/2021 4/3/2021 5/3/2021 = Xe Counterfactual COVID-19 Deaths if no Vaccines e=m¢eee Estimated COVID-19 Deaths with Vaccines Notes: Cumulative weekly number of COVID-19 infections, hospitalizations and deaths without vaccines "counterfactual" OCTOBER 2021 RESEARCH REPORT 6 (dotted line) and number with vaccines estimated by the model (solid line). The scale for deaths is smaller than the scale for infections and hospitalizations. The weekly cumulative reductions in all three COVID-19 outcomes associated with COVID-19 vaccinations (the difference between these two lines) are shown in Figure 2. To get an estimate of the total number of COVID-19 outcomes prevented by the introduction of vaccinations, we summed the weekly differences between the estimated and counterfactual number of outcomes represented by the dashed versus solid lines in Figure 1. Figure 2 summarizes the total number of COVID-19 prevented among the study cohort over time. It shows that as the vaccinations are rolled out, more COVID-19 outcomes are prevented. The figure plots the estimated weekly cumulative reduction in COVID-19 outcomes in our sample associated with county-level vaccination rates. The three lines in the figure show COVID-19 related outcomes: infections in blue, hospitalizations in orange, and deaths in grey. The bars in the figure show county-level vaccination rates increasing over time: ages 18- 64 in the grey bar, and ages 65+ in the yellow bar. The figure shows the cumulative estimated reduction in the number of COVID-19 infections, hospitalizations, and deaths compared to what they would have been among our study cohort in the absence of vaccination, starting in early February and continuing throughout the initial vaccination roll- out period. Figure 2. Estimated Reduction in COVID-19 Outcomes in Association with COVID-19 Vaccinations Among the Medicare Study Sample (N=25.3 million) 120,000 100 o ~ S 90 & ® 100,000 = = 100, s0 & o 00 E a 8 80,000 8 S 5 a 60,000 50 8 > > g © 8 ® 40,000 30 $ - & SD = 20,000 20 E g 10 3 6 x 2 0 0 6 1/3/2021 2/3/2021 3/3/2021 4/3/2021 5/3/2021 = % Week = Average County Vaccination % (18-64) (mmm Average County Vaccination % (65+) <=@=-= Cumulative reduction in COVID-19 deaths -<=@=-= Cumulative reduction in COVID-19 cases ==@== Cumulative reduction in COVID-19 hospitalizations Notes: Reductions in cumulative COVID-19 cases, hospitalizations, and deaths are weekly differences between estimated and predicted counts for each outcome with and without vaccines between January and May 2021 in the Medicare study cohort (N=25.3 million). This included 48 states and Puerto Rico and District of Columbia in the estimation model and excluded Texas and Hawaii. Average vaccine rates for the proportion of the population ages 18-64 and 65+ fully vaccinated are at the county level and come from the CDC vaccine data hitps://www.cdc.gov/coronavirus/2019- ncov/vaccines/distributing/about-vaccine-data.html OCTOBER 2021 RESEARCH REPORT 7 Table 2 shows the estimated reduction in COVID-19 outcomes among Medicare beneficiaries in our sample related to county-level COVID-19 vaccinations from January 2021-May 2021. By the end of May when 68% of people 65 and over on average had been vaccinated in the U.S., our model estimated potentially 107,000 fewer infections, 43,000 fewer hospitalizations, and 16,000 fewer deaths in the study sample (Table 2). This represents an estimated 18% reduction in COVID-19 infections, 21% reduction in COVID-19 hospitalizations, and 22% reduction in COVID-19 deaths based on the number of associated with roughly a potential COVID-19 outcomes (the counterfactual scenario) 1/5 reduction that we project would have occurred in the absence of in COVID-19 infections vaccines from January to May 2021. Vaccinations were among Medicare For example, without vaccines, we would have expected beneficiaries, as well as 598,000 COVID-19 infections in the study sample; instead COVID-19 related with vaccinations, there were about 491,000 infections, resulting in 107,000 fewer infections. The percent reduction is then calculated as the number of COVID-19 infections prevented divided by the number of infections expected without vaccines (107,000/598,000=18%). hospitalizations and deaths. Table 2. Estimated Reductions in COVID-19 Outcomes in Association with COVID-19 Vaccinations Among Medicare Study Cohort and Projected to Full Medicare Population (Jan-May 2021) COVID-19 COVID-19 COVID-19 Infections Hospitalizations Y= dak STUDY COHORT N=25.3 million Medicare FFS beneficiaries If no el available, number 598,000 203,000 71,000 (predicted with zero vaccination rate) Md estimated outcomes with 491,000 160,000 56,000 vaccines, number Estimated reduction in number of 107,000 43,000 16,000 COVID Outcomes' (percent reduction) (17.9%) (21.2%) (22.0%) FULL MEDICARE POPULATION N=62.7 million Medicare beneficiaries If no vaccines available, number , , a 1,481,000 504,000 177,000 (predicted with zero vaccination rate) Md estimated outcomes with 1,216,000 397,000 138,000 N-lxoll aye aleloal elcis EOVIb Outeomest(peteent reduction) SEDI 107,000 39,000 P (17.9%) (21.2%) (22.0%) Notes: Estimates are predicted based on probabilities for the outcome from regression models with cumulative county-level weekly vaccination rates for adults ages 18-64 and 65+, controlling for beneficiary demographic characteristics, comorbidities, local county characteristics, and state and month fixed effects. 48 states and Puerto Rico and District of Columbia were included in the estimation model. Texas and Hawaii were excluded from the model due to lack of county- level vaccination rate data. COVID-19 infections are identified using ICD-19 diagnosis U07.1 for COVID-19. Hospitalizations are within 3 weeks and deaths are within 9 weeks of the initial COVID-19 diagnosis. Numbers may not sum precisely due to OCTOBER 2021 RESEARCH REPORT 8 rounding. All regression estimates are rounded to the nearest hundred for estimates below 10,000 and to the nearest thousand for numbers above 10,000. "Reductions in Outcome = weekly number without vaccinations - weekly number with vaccinations, summed for each week from January-May 2021. The percent reduced is calculated as the reduction in outcomes divided by the number of outcomes if no vaccinations were available (counterfactual). Table 2 includes projected estimates for the full Medicare population, including 26 million Medicare Advantage enrollees and other FFS beneficiaries who did not meet our primary sample inclusion criteria. To estimate the number of COVID-19 outcomes reduced in association with vaccinations among all Medicare beneficiaries, we projected the reduction in COVID-19 outcomes among the study cohort to the entire Medicare population of 62.7 million beneficiaries.® For simplicity, assuming that all Medicare beneficiaries had the same COVID-19 risks and vaccine uptake as those in our FFS study cohort, we estimate that vaccinations were associated with approximately 265,000 fewer COVID-19 infections, 107,000 fewer COVID-19 hospitalizations, and 39,000 fewer COVID-related deaths among all Medicare beneficiaries by the end of May. An alternative model with week fixed effects and modeled vaccination rates as a categorical variable estimated a smaller reduction in COVID-19 outcomes than the main model. This provides a lower bound on the estimated reductions in COVID-19 outcomes associated with COVID-19 vaccinations. Among the study cohort in this model, COVID-19 vaccinations were associated with estimated reductions of 35,000 infections, 14,000 hospitalizations and 4,900 deaths. When projected to the full Medicare population, these lower bound estimates from the alternate model suggest COVID-19 vaccination prevented at least 87,000 infections, 34,000 hospitalizations and 12,000 deaths among all Medicare beneficiaries. These sensitivity analyses confirm the negative association between COVID-19 vaccination rates and reduction in COVID-19 outcomes, especially hospitalizations and deaths, but show the magnitude of the estimated reductions depends on the model assumptions and the comparison group selected. The alternate model may have smaller estimated reductions in COVID-19 outcomes because including weekly fixed effects may have absorbed much of the variation in vaccination rates compared to using month fixed effects. Another potential reason for the smaller estimated reductions may be that the 20 indicators for weekly temporal changes are collinear with the 32 indicators for vaccination rates, and this may have led to less stable estimates of the concurrent impact of changes in vaccination rates and time (see Appendix for detailed methods on sensitivity analyses). Impacts of COVID-19 Vaccines on Demographic Subgroups Table 3 shows these results for demographic subgroups in the study cohort. COVID-19 vaccination rates were associated with an estimated 17-21% reduction in COVID-19 infections and an estimated 21-25% reduction in COVID-19 deaths across racial and ethnic groups among Medicare beneficiaries. Note that the estimates for smaller subgroups are more imprecise due to small sample size. The largest percent reduction appeared to be for American Indians and Alaska Natives (AI/AN), with an estimated reduction in infections of 21%, compared to 18% among White beneficiaries, and reduction in deaths of 25% compared to 22% among White beneficiaries. The largest estimated number of reductions in COVID-19 outcomes were among White beneficiaries, who comprised about 81% of the study cohort (see Appendix table 3 for breakdown of study cohort by race/ethnicity compared to Medicare population). OCTOBER 2021 RESEARCH REPORT 9 Table 3. Reductions in COVID-19 Outcomes in Association with Vaccination Among Medicare Study Cohort (January - May 2021), by Race/Ethnicity Study Cohort mem EL ay = Based on actual Estimated reduction' in vaccination rates COVID-19 Outcomes COVID-19 Infections Projected to Total Medicare Projected reduction in COVID-19 Outcomes tyee 1119 reexelU(aa{o) ha EEe-Y a aaa oli ayy available Study Cohort 598,000 491,000 107,000 265,000 17.9% (N=25.3 million) White 479,000 393,000 86,000 193,000 17.9% (N=20.6 million) Black 51,000 42,000 8,800 29,000 17.3% (N=2.0 million) Hispanic 36,000 30,000 6,400 29,000 17.7% (N=1.2 million) Asian 12,000 9,900 2,300 7,600 18.7% (N=676,441) Al/AN 5,500 4,400 1,100 2,000 20.5% (N=145,488) Other 13,000 11,000 2,500 6,300 18.5% (N=737,449) COVID-19 Hospitalizations Study Cohort 203,000 160,000 43,000 107,000 21.2% (N=25.3 million) White 155,000 122,000 33,000 74,000 21.4% (N=20.6 million) Black 22,000 18,000 4,000 15,000 19.9% (N=2.0 million) Hispanic 14,000 11,000 3,000 13,700 20.9% (N=1.2 million) Asian 5,500 4,300 1,200 4,000 21.8% (N=676,441) AI/AN 2,200 1,700 500 1,000 23.8% (N=145,488) Other 4,000 3,100 900 2,300 22.4% (N=737,449) Study Cohort 71,000 56,000 16,000 39,000 22.0% (N=25.3 million) White 56,000 44,000 12,000 28,000 22.0% (N=20.6 million) Black 6,600 5,200 1,400 4,600 20.9% (N=2.0 million) Hispanic 4,900 3,800 1,100 5,000 21.9% (N=1.2 million) Asian 1,800 1,400 400 1,400 23.3% (N=676,441) Al/AN 900 700 200 400 24.8% (N=145,488) Other 1,000 800 200 600 23.5% (N=737,449) OCTOBER 2021 RESEARCH REPORT 10 Notes: Estimates are predicted based on probabilities for the outcome from primary regression models with cumulative county-level weekly vaccination rates for ages 18-64 and 65+, controlling for beneficiary demographic characteristics, comorbidities, local county characteristics, and state and month fixed effects. These estimates assume that the overall model applies to racial/ethnic groups. The total Medicare estimates are projected from study cohort estimates by multiplying them by a factor for each subgroup (the ratio of total Medicare population to Medicare beneficiary counts in our sample). Numbers may not sum precisely due to rounding. All regression estimates are rounded to the nearest hundred for estimates below 10,000 and to the nearest thousand for numbers above 10,000. AI/AN = American Indians and Alaska Natives. "Reductions in Outcome = weekly number without vaccinations - weekly number with vaccinations, summed for each week from January-May 2021. "*Percent reduction is calculated as the proportion of estimated reduction in outcomes divided by the number of outcomes if no vaccines were available (counterfactual). We estimated reductions of 29,000 infections and nearly 4,600 deaths among Black beneficiaries in the total Medicare population, reductions of 29,000 infections and nearly 5,000 deaths among Hispanic beneficiaries, reductions of nearly 7,600 infections and 1,400 deaths among Asian beneficiaries, and reductions of 2,000 infections and 400 deaths among AI/AN beneficiaries in the total Medicare population. Impacts of COVID-19 Vaccines on Nursing Home Residents Another group at high-risk of COVID-19 infections and death are long-term nursing home residents, who were disproportionately affected by COVID-19.° For long-term nursing home residents identified in this study sample, Table 4 shows COVID-19 vaccinations were associated with estimated reductions of about 8,400 infections, 1,900 hospitalizations and 2,200 deaths in the study cohort, which translates to an estimated reduction of nearly 21,000 infections, 4,900 hospitalizations and 5,600 deaths when projected to the total Medicare population. Compared with beneficiaries living in the community, long-term nursing home residents were less likely to be hospitalized than beneficiaries living in the community, presumably because they are already in a health care setting. Table 4. Reductions in COVID-19 Outcomes in Association with Vaccination for Nursing Home Residents vs. Community-dwelling Medicare Beneficiaries January - May 2021) Study Cohort Projected to Total Medicare Nursing Home If no vaccines Based on actual Estimated reductions" Projected reductions" 1-101 8 ek available vaccination rates in COVID-19 outcome in COVID-19 outcome _-reduction™" COVID-19 Infections Study Cohort 598,000 491,000 107,000 265,000 17.9% Community 543,000 444,000 99,000 244,000 18.2% N=24.9 million Nursing Home 55,000 47,000 8,400 21,000 15.3% N= 330,524 COVID-19 Hospitalizations Study Cohort 203,000 160,000 43,000 107,000 21.2% Community 193,000 152,000 41,000 102,000 21.4% Nursing Home 10,000 8,300 1,900 4,900 18.8% OCTOBER 2021 RESEARCH REPORT 11 Study Cohort Projected to Total Medicare Nursing Home Ifno vaccines -_ Based on actual Estimated reductions" Projected reductions" Teen) 4 Ar 1H0K available vaccination rates in COVID-19 outcome in COVID-19 outcome reduction™" COVID-19 Deaths Study Cohort 71,000 56,000 16,000 39,000 22.0% Community 60,000 46,000 14,000 33,000 22.7% Nursing Home 12,000 9,800 2,200 5,600 18.4% Notes: Estimates are predicted based on probabilities for the outcome from the primary regression models with cumulative county-level weekly vaccination rates for ages 18-64 and 65+, controlling for beneficiary demographic characteristics, comorbidities, local county characteristics, and state and month fixed effects. These estimates assume that the overall model applies to nursing home and community-dwelling residents. The total Medicare estimates are projected from study cohort estimates by multiplying them by a factor for each subgroup (the ratio of total Medicare population to Medicare beneficiary counts in our sample). Numbers may not sum precisely due to rounding. All regression estimates are rounded to the nearest hundred for estimates below 10,000 and to the nearest thousand for numbers above 10,000. *"Reductions in Outcome = weekly number without vaccinations - weekly number with vaccinations, summed for each week from January-May 2021. "Percent reduction is calculated as the proportion of estimated reduction in outcomes divided by the number of outcomes if no vaccines were available (counterfactual). State Estimates of Vaccine-Related Reductions in COVID-19 Outcomes State-specific estimates were generated for all 50 states based on the state indicator in the estimation model. Table 5 shows state-specific estimates of reductions in COVID-19 outcomes for the 20 most populous states in our sample (excluding Texas) and projected to the total Medicare population. Nationally, on average this study found COVID-19 vaccinations were associated with a reduction of 420 COVID-19 infections per 100,000 beneficiaries, a reduction of 170 COVID-related hospitalizations per 100,000 beneficiaries and a decrease of 60 COVID-related deaths per 100,000 beneficiaries, with a range of results reflecting the range of initial vaccination rates. Results for all 50 states, Puerto Rico and D.C. per 100,000 Medicare beneficiaries are shown in the Appendix, including imputed estimates for Texas and Hawaii. However, estimates for smaller states should be interpreted cautiously. Table 5: State-specific Estimates of COVID-19 Reductions Associated with COVID-19 Vaccinations for Top 20 Most lous States, Cohort and to Total Medicare lation Total US 25,295,000 107,000 | 43,000 16,000 265,000 | 107,000 | 39,000 California 2,439,000 8,900 3,900 1,500 22,000 9,700 3,800 Florida 1,810,000 6,900 2,700 1,000 17,000 6,700 2,400 New York 1,450,000 6,600 2,700 1,100 16,000 6,700 2,600 Illinois 1,167,000 5,400 2,500 800 13,000 6,200 2,100 Pennsylvania 1,115,000 4,400 2,000 700 11,000 4,900 1,800 OCTOBER 2021 RESEARCH REPORT 12 North Carolina 913,000 36.1 71.5 4,500 1,700 600 11,000 4,200 Virginia 860,000 22.1 34.9 1,600 700 300 3,900 1,600 Michigan 803,000 40.7 73.0 3,400 1,600 600 8,500 3,900 New Jersey 782,000 48.5 70.5 3,600 1,600 600 8,900 3,900 Massachusetts 746,000 46.7 70.0 2,400 1,000 300 6,000 2,500 Georgia 712,000 14.9 34.4 2,100 800 300 5,100 2,100 Maryland 672,000 49.6 79.3 2,700 1,200 400 6,700 2,900 Washington 662,000 43.6 77.7 1,400 500 2090 3,400 1,300 Indiana 611,000 36.7 76.4 3,800 1,500 500 9,500 3,600 Tennessee 593,000 31.1 67.1 3,500 1,200 500 8,600 3,100 Arizona 570,000 37.6 67.4 2,800 1,100 400 6,800 2,800 South Carolina 567,000 30.3 67.8 2,600 900 300 6,300 2,300 Missouri 552,000 29.9 63.6 2,700 1,100 300 6,700 2,700 Wisconsin 487,000 44.3 83.1 2,200 900 300 5,300 2,200 Notes: Estimates are predicted based on probabilities for the outcome from primary regression models with cumulative county-level weekly vaccination rates for ages 18-64 and 65+, controlling for beneficiary demographic characteristics, comorbidities, local county characteristics, and state and month fixed effects. The total Medicare estimates are projected from study cohort estimates by multiplying them by a factor of 2.45 (the ratio of total Medicare population to Medicare beneficiary counts in our sample). "Reductions in Outcome = weekly number without vaccinations - weekly number with vaccinations, summed for each week from January-May 2021. Numbers may not sum precisely due to rounding. All estimates are rounded to the nearest hundred for estimates below 10,000 and to the nearest thousand for numbers above 10,000. Estimates below 100 are rounded to the nearest ten and masked if below 50 for protection of privacy. Texas and Hawaii are not included in the estimation model or this table, as they lacked county-level vaccination rate. Estimates for those two states are imputed and included in the Appendix. Figure 3 compares the estimated cumulative reduction in COVID-19 outcomes per 100,000 Medicare beneficiaries in each state to average vaccination rates from January to May 2021. As expected, there was generally a greater reduction in cumulative COVID-19 infections, COVID-related hospitalizations, and deaths per 100,000 beneficiaries related to vaccination in states with higher average vaccination rates. The figure also shows a steeper gradient at low levels of vaccination, as the projected reductions in COVID-19 outcomes rise more dramatically on the right side of the figure, in states with lower vaccination rates compared to those with higher rates. OCTOBER 2021 RESEARCH REPORT 13 Figure 3. Reductions in COVID-19 Outcomes per 100,000 Beneficiaries Associated with COVID-19 Vaccinations, By State Vaccination Rates Among Adults 65 and older 800 700 © e@ 600 ee . 500 400 300 © @ e@ e@ =| e @ ° © Beneficiaries 100 Reductions in COVID-19 Outcomes per 100,000 Medicare FFS States i xD x2 & & & ° > Q .. o> 2 LD 2 & Oo 2 2 re SS & Wd RS es ¥ < Ne PF PS OP 6? & ws "oe Ee Se ¥ See PS FL we ee oe® Fe Set $ . o 3 e VSS OF ww PP LMM GL MATE WS s SS SF oe s (from highest to lowest vaccination rates) ™@ Cases reduced per 100,000 beneficiaries ™ Deaths reduced per 100,000 beneficiaries @ Avg Vaccinations 65+ B® Hospitalizations reduced per 100,000 beneficiaries © Avg Vaccinations 18-64 Notes: This figure shows the estimated reductions in COVID-19 infections, hospitalizations, and deaths for each state calculated per 100,000 Medicare FFS beneficiaries. The estimated reductions are the cumulative weekly differences in counts for each outcome estimated by the model with and without vaccines from January to May 2021. Average vaccine rates for ages 18-64 and 65+ are at the county level from the CDC data. *This figure does not show Hawaii and Texas as the they were excluded from the model sample due to lack of data on county vaccination rates; however, these were imputed using estimated vaccine effectiveness from a state with a similar vaccination rate and shown in the Appendix. In addition to comparing states, we examined counties by their vaccination levels. After dividing counties into thirds based on their vaccination rates at the end of May among those 65 and older, counties in the top and middle thirds had an average vaccination rate of 70-80% compared with 51% for counties in the lowest third of vaccination rates; for vaccination rates among adults 18-64, these were 39-47% in middle and top thirds vs. 27% in the lowest third. Comparing the top and middle third vaccination rate counties with the lowest third counties, the average reduction in COVID-19 outcomes was 446-450 vs. 361 infections, 179-184 vs. 143 hospitalizations, and 65-67 vs. 52 deaths per 100,000 Medicare beneficiaries. This suggests that if low vaccination counties increased their vaccination rates among adults 65 and older by about 19%-30% (e.g., from 51% to >70%) they could potentially prevent another 84-89 infections, 36-41 hospitalizations and 12-14 deaths per 100,000 beneficiaries (see Appendix Table 2). OCTOBER 2021 RESEARCH REPORT 14 Average Vaccination Rates (18-64, 65+), % Discussion In this population-level regression analysis, we found that county-level COVID-19 vaccination rates were significantly associated with reductions in the odds of COVID-19 infections and severe outcomes among Medicare beneficiaries. Of note, we found a strong negative linear relationship between county-level vaccination rates and COVID-19 outcomes among Medicare beneficiaries, which appeared to taper off at higher vaccination rates, as supported by sensitivity analyses. This suggests that initial increases in vaccination rates are likely very effective and even a modest increase in vaccination rates has a large payoff; accordingly, increasing vaccination rates in low vaccination rate counties may have a larger impact on mitigating COVID-19 harms than further increasing rates in counties that already have high Initial increases in vaccination rates are likely very effective vaccination rates. and even a modest increase in vaccination rates has a large Furthermore, our study shows that both vaccination rates for payoff; accordingly, increasing those ages 65 and older and those 18-64 had a significant vaccination rates in low relationship with improved COVID-19 outcomes among the . . . Medicare FFS study cohort, especially reductions in severe vaccination rate counties may outcomes. This suggests that to protect Medicare beneficiaries, have a larger impact on a high vaccination rate among those 65 and older on its own is mitigating COVID-19 harms not as effective as high vaccination rates among all adults. . . . than further increasing rates in Based on the negative association between county vaccination counties that already have rates and corresponding COVID-19 outcomes in 2021, we then high vaccination rates. estimated that without COVID-19 vaccinations more than half a million Medicare FFS beneficiaries in the study cohort might have had a COVID-19 infection between January-May 2021, nearly 203,000 of those might have been hospitalized with COVID-19, and about a third of those hospitalized (about 71,000) might have died. Projected to the full Medicare population, our study estimated vaccinations were associated with approximately a quarter of a million (265,000) fewer Medicare beneficiaries with COVID-19 infection, 107,000 fewer COVID-19 hospitalizations, and 39,000 fewer COVID-related deaths among all Medicare beneficiaries by the end of May 2021. COVID-19 vaccinations Other studies have also found significant reductions in deaths associated with COVID-19 vaccinations in 2021.1°7? However, . those studies did not examine the relationship between a quarter of a million fewer vaccinations and COVID-19 outcomes in the Medicare COVID-19 infections, population, and our study used rich individual-level claims 107,000 fewer COVID-19 data to control for potential confounders and other factors not measured in prior studies. As discussed earlier, our estimates likely reflect both higher vaccination rates among 39,000 fewer COVID-19 the Medicare population and lower risks of infection in related deaths among all seniors, who may have taken more measures to limit social contacts and restrict mobility in the fall of 2020 than the younger adult population. Other reasons for differences in were associated with about hospitalizations, and Medicare beneficiaries by the end of May 2021. OCTOBER 2021 RESEARCH REPORT 15 our estimates relative to other studies may be due to different analytical methods and data. In summary, we found that the initial roll-out of COVID-19 vaccinations in 2021 was associated with an 18% reduction in infections among Medicare beneficiaries who might have had a COVID-19 infection and was associated with 21-22% reductions in hospitalizations and deaths compared to if no vaccines were available. Implications of Study Findings for the Future These study findings reflect the relationship between the initial vaccine roll-out on and reductions in COVID-19 infections and severe outcomes. Vaccination rates have continued to rise over the summer, and with the surge in cases from the Delta variant, the importance of vaccination has likely grown substantially given the Delta variant is more than twice as contagious as previous variants and has a higher transmission rate among unvaccinated people." As of September 7, 2021, CDC data show about 64% of adults 18 to 64 have been vaccinated, nearly twice as high as the vaccination rates at the end of May, and nearly 82% of people aged 65 and older are fully vaccinated, about 20% higher than rates at the end of the study. Further study of population-level impacts can help assess the effectiveness of COVID-19 vaccines against severe illness with the Delta variant, as a complement to recent clinical studies. Those who remained unvaccinated (or only partially vaccinated) are at far higher risk of COVID- 19 infection and severe outcomes.?? Health Equity Implications of COVID-19 Vaccinations Vaccinations were associated with reduced risks of COVID-19 among Black, Hispanic, Asian, and AIl/AN beneficiaries. We estimated COVID-19 vaccinations were associated with reductions of approximately 19,000 infections, 9,000 hospitalizations and 3,000 deaths among Medicare beneficiaries in the study cohort from these communities. Projected to the full Medicare population, we estimated COVID-19 vaccinations were associated with reductions of 68,000 infections, 33,000 hospitalizations, and 11,000 deaths among Black, Hispanic, Asian, and Al/AN Medicare beneficiaries. Study Limitations Our model has important limitations. These estimates are likely a conservative estimate of the initial vaccine rollout period as they reflect the lower rate of COVID-19 vaccination during the early vaccination ramp-up period, and before Delta variant started in late June. While vaccinations among adults 65 and older have increased to more than 80% by the end of August," these results may not be as applicable for the summer months. For these reasons, we did not project our findings beyond the study period. Another important limitation of our study is the apparent undercount of COVID-19 related deaths in Medicare FFS claims. This may result from delays in death data from the Social Security Administration/Railroad Retirement Board used to update the CMS beneficiary enrollment file, as well as technical differences in how we are classifying COVID-19 related deaths compared to the CDC. Updates to data on death in Medicare claims are lagged by 2-3 months." We used any death within 9 weeks of COVID-19 diagnosis to identify COVID-19 related deaths. However, claims data do not include information on cause of death, and some deaths that occurred may not have had an associated COVID- 19 diagnosis. These factors may have caused baseline COVID-19 death counts in our study to be underestimated, reducing our model's estimated change in deaths. CDC's provisional count for COVID- OCTOBER 2021 RESEARCH REPORT 16 19 deaths reports nearly 162,000 deaths among adults 65 and older from January to May 2021. Our estimates projected to the full 62.7 million Medicare population would yield about 139,000 COVID-19 deaths in seniors, roughly 15% below the CDC reported counts; in addition, 15% of Medicare beneficiaries are under age 65, suggesting our dataset may have undercounted COVID-19 related deaths by as much as 26%.*©" Applying the proportional reduction in deaths we found associated with vaccinations to the CDC numbers would imply as many as 49,000 Medicare deaths prevented.' After accounting for the potential underreporting of COVID deaths in Medicare FFS claims data, and considering alternative models, the number of deaths prevented among the full Medicare population could plausibly range from 12,000 to 49,000 deaths. There are important confounders that were not accounted for this study, worthy of consideration in future refinements to the model and sensitivity analyses. First, we did not have complete individual- level data on vaccination status, even though we had individual-level outcomes and covariates. Second, we lacked information on transmission risks based on household composition such as whether the beneficiary lives alone or lives with a family member who may be working in the community and therefore at higher risk of COVID-19 infection. Third, we included state fixed effects but did not specifically include information about the timing of state mitigation efforts that could have influenced the observed association with COVID-19 outcomes. Not including these confounders could have inflated our estimates of the potential reduction in COVID-19 associated with vaccinations, if county and state mitigation efforts were positively correlated with differential county vaccination rates. Finally, to project the estimated reduction in COVID-19 outcomes from the FFS study cohort to the full Medicare population and subgroups, we assumed the risks of COVID-19 in the study cohort are the same in the full Medicare population. However, beneficiaries enrolled in Medicare Advantage - who make up 40% of the Medicare population - are more likely than FFS beneficiaries to be older, Black or Hispanic, lower-income, and medically complex."" Our study also sought to identify frail long-term nursing home residents but may not have fully captured them using Medicare data. This could underestimate the potential reductions in COVID-19 infections and severe outcomes among higher-risk beneficiaries in Medicare Advantage, long-term nursing home residents or FFS beneficiaries who did not meet the study inclusion criteria, who would have a higher baseline risk of COVID-19. Our estimates should be considered conservative for this group. Our sensitivity analyses confirm the association between COVID-19 vaccinations and reduction in COVID- 19 infections, hospitalizations, and deaths. In future research, additional sensitivity analyses and model refinements can assess how different study assumptions may revise these modeled estimates. tT If we assume beneficiaries under 65 had similar risks as those over 65, this would imply that the 162,000 deaths reported by the CDC among those 65 and older would correspond to 186,000 deaths when adding in the 15% of Medicare beneficiaries under 65 (115%*162,000). Compared to 186,000 deaths, our implied total of 138,250 Medicare deaths would be a 26% underestimate. While beneficiaries under 65 are younger, which is associated with lower risk of severe COVID-19 outcomes, those under 65 qualify for Medicare via disabilities, which would indicate higher-than-average risk. * To account for the 26% underestimate, 126% * 38,945 = 49,070 estimated reduction in deaths in the full Medicare population. OCTOBER 2021 RESEARCH REPORT 17 Conclusion Consistent with evidence on the clinical effectiveness of COVID-19 vaccinations, our population-level regression-based estimates indicate that COVID-19 vaccinations were associated with thousands of fewer deaths among the Medicare population during the pandemic in early 2021. As cumulative vaccination rates continue to increase over time, more beneficiaries are expected to be saved from COVID-related hospitalizations and deaths. The difference in vaccination rates for those age 65 and older between the lowest (34%) and highest (85%) states by the end of May highlights the continued opportunity to leverage COVID-19 vaccinations to prevent unnecessary hospitalizations and premature deaths. OCTOBER 2021 RESEARCH REPORT 18 APPENDIX: SUPPLEMENTAL ANALYSES Appendix Table 1. State Estimates of Reductions in COVID-19 Outcomes per 100,000 Medicare beneficiaries Associated with Vaccination, Study Cohort and Projected to Total Medicare Population Total US 25,295,000 265,000 107,000 39,000 Alabama 410,000 5,200 2,000 700 Alaska 82,000 800 200 100 Arizona 570,000 6,800 2,800 1,000 Arkansas 347,000 4,200 1,400 600 California 2,439,000 22,000 9,700 3,800 Colorado 375,000 1,700 700 250 Connecticut 266,000 3,100 1,300 500 Delaware 137,000 1,400 600 200 District of 48,000 400 200 100 Columbia Florida 1,810,000 17,000 6,700 2,400 Georgia 712,000 5,100 2,100 700 Hawaii* 100,000 110 50 <50 Idaho 167,000 1,900 200 Illinois 1,167,000 13,000 2,100 Indiana 611,000 9,500 1,300 lowa 377,000 5,100 700 Kansas 332,000 4,300 700 Kentucky 427,000 5,500 800 Louisiana 361,000 5,300 600 Maine 149,000 800 100 Maryland 672,000 6,700 1,100 Massachusetts 746,000 6,000 900 Michigan 803,000 8,500 1,400 Minnesota 384,000 4,500 600 Mississippi 354,000 5,600 600 Missouri 552,000 6,700 900 Montana 150,000 2,300 200 Nebraska 218,000 2,500 300 Nevada 226,000 2,100 400 OCTOBER 2021 RESEARCH REPORT 19 New Hampshire 182,000 32.1 69.4 400 200 New Jersey 782,000 48.5 70.5 8,900 3,900 New Mexico 188,000 41.6 60.3 1,500 600 New York 1,450,000 50.6 73.1 16,000 6,700 North Carolina 913,000 36.1 71.5 11,000 4,200 North Dakota 84,000 37.0 70.4 1,600 500 Ohio 935,000 39.1 74.3 13,000 5,300 Oklahoma 415,000 32.4 67.3 7,500 2,800 Oregon 345,000 40.3 73.9 1,500 600 Pennsylvania 1,115,000 40.7 71,7 11,000 4,900 Puerto Rico 64,000 35.9 63.1 200 100 Rhode Island 78,000 51.4 85.2 900 300 South Carolina 567,000 30.3 67.8 6,300 2,300 South Dakota 108,000 38.1 64.2 1,600 600 Tennessee 593,000 31.1 67.1 8,600 3,100 Texas* 1,873,000 50.1 50.1 24,000 9,700 Utah 178,000 36.6 57.6 1,500 600 Vermont 102,000 37.7 66.7 90 30 200 100 Virginia 860,000 22.1 34.9 180 80 3,900 1,600 Washington 662,000 43.6 777 210 80 3,400 1,300 West Virginia 207,000 18.2 37.4 280 110 1,400 600 Wisconsin 487,000 44.3 83.1 460 180 60 5,500 2,200 Wyoming 89,000 30.2 67.4 560 210 70 1,200 500 Notes: Estimated reductions in COVID-19 infections, hospitalizations, and deaths for each state are cumulative weekly differences between estimated and predicted counts for each outcome between January and May 2021 and calculated per 100,000 Medicare FFS beneficiaries. Estimates are predicted based on probabilities for the outcome from primary regression models with cumulative county-level weekly vaccination rates for ages 18-64 and 65+, controlling for beneficiary demographic characteristics, comorbidities, local county characteristics, and state and month fixed effects. The total Medicare estimates are projected from study cohort estimates by multiplying them by a factor of 2.45 (the ratio of total Medicare population to Medicare beneficiary counts in our sample). Numbers may not sum precisely due to rounding. All estimates are rounded to the nearest hundred for estimates below 10,000 and to the nearest thousand for numbers above 10,000. Estimates below 100 are rounded to the nearest ten and masked if below 50 for protection of privacy. "Estimates for Texas and Hawaii are imputed based on a state with a similar vaccination rate (California). The imputed number of reduction in outcomes is the difference in the imputed counterfactual and actual number of outcomes for these 2 states. These two states were not included in the estimation model study cohort. The national totals presented in this report do not include these two states' imputed values. In addition, the vaccination rates for these two states shown are based on all adults 18+. "* Note: estimates for states with smaller populations are less precise, in particular for deaths. See predicted/actual ratios for outcomes by state in the Appendix Figure 2. OCTOBER 2021 RESEARCH REPORT 20 Appendix Table 2. Estimated COVID-19 Outcomes Reduced per 100,000 Beneficiaries by County Vaccination Levels County Vaccination Rates Reductions per 100,000 Beneficiaries at end of May County Vaccination Levels LY :<-1) LY:<24) COVID-19 COVID-19 COVID-19 Ray) 65+ Tiered toyans Hospitalizations Deaths All US Counties 39% 68% 423 171 62 N=25.3 million beneficiaries ° ° High Vaccination Rate Counties 471% 80% 450 179 65 N=8.9 million Medium Vaccination Rate Counties 30% 70% AAG 184 67 N=9.2 million Low Vaccination Rate Counties 27% 51% 361 143 52 N=7.1 million Appendix Table 3. Medicare beneficiary characteristics in the study cohort and full Medicare population Study cohort Total Medicare Population N=25.3 million N=62.7 million Percent of Percent of Number in subgroup relative Number in subgroup relative Subgroup a ay mi aera Subgroup to total Medicare u population Race White 20,561,000 74% 46,199,000 81% Black 1,992,000 11% 6,636,000 8% Hispanic 1,184,000 9% 5,471,000 5% Asian 676,000 4% 2,247,000 3% AI/AN 145,000 0.4% 264,000 0.6% Other 737,000 3% 1,883,000 3% Nursing home status Resides in community 24,965,000 99% 61,860,000 99% Nursing home 331,000 1% 840,000 1% Notes: Numbers may not sum precisely due to rounding. All estimates are rounded to the nearest hundred for estimates below 10,000 and to the nearest thousand for numbers above 10,000. AI/AN = American Indians and Alaska Natives. Long-stay nursing home residents are those who have been in a skilled nursing facility for greater than 90 days and not discharged to the community for more than 14 days, identified using nursing home assessment data and Medicare claims. OCTOBER 2021 RESEARCH REPORT 21 Appendix Figure 1. Comparison of the cumulative weekly number of COVID-19 outcomes: actual observed vs. estimated by model with vaccines in study cohort, January - May 2021 (main model) 600,000 500,000 400,000 300,000 200,000 100,000 I Ue UR ee ok Ww? we wey we we ay Avs we "eo o ~ we ow Pe wv w we w Ww xX we ww -®- Actual COVID-19 Infections -@=- Estimated COVID-19 Infections with Vaccines -?- Actual COVID-19 Hospitalizations -@=- Estimated COVID-19 Hospitalizations with Vaccines -- Actual COVID-19 Deaths This figure shows our estimation model closely predicts the number of COVID-19 outcomes compared to the number that actually occurred. In addition, the curves start to become less steep after the end of January suggesting vaccinations are slowing down the transmission of COVID-19 infections and therefore also slowing down COVID-19 related hospitalizations and deaths. OCTOBER 2021 RESEARCH REPORT 22 €2 LY¥Od3u¥ HIYVISIY TZ0Z YAGOLIO syieed 6T-GIAOD Peleus, @ SUONI2I4UI GT-CGIAOD Peieuwins;s mw syieeq OT-CIAOD IeNev SUONYVEJUI GT-GIAOD) |e NPvV 9G Yd IH WY GN AM 1A GS 3G AW IW Gl IN HN WN AM AN AN 195 SH YO YV SW WI OD VI NW TW HO AX IM OW OS ZV NL NI VM OW VO YWWOIN IW VA SON HO Wd TW AN 14° WO 000'S 000'0T 000'ST 000'0% 000'SZ 000'0€ 000'SsE 000'0r 000'Sv 0000S (j@pow ule) a}e}s Aq 'WOYOs Apnys Ul] SAUIDDEA YUM JaPOW Aq paj}eWijsa "SA paAsasqo [ene - S9W0I}NO GT-GIAOD JO Jequunu aaljejnwind ayy Jo uosedWwoO? *7 aun8l4 xipuaddy APPENDIX: DETAILED METHODOLOGY AND ASSUMPTIONS Model Covariates The key predictors were CDC's weekly county-level cumulative vaccination rates (18-64 and 65+) in the model; included as a continuous variable in the main model and as a categorical variable in the alternate model. Beneficiary characteristics included age, sex, race/ethnicity, dual enrollment in Medicaid, and disability or end-stage renal disease as the original reason for Medicare enrollment. Medicare beneficiaries residing long-term in a nursing home are identified as they are at higher risk of poor outcomes. This captures beneficiaries who have been in a nursing home for at least 90 days until they return to the community for at least 14 days. The study included a 12-month look-back period to capture comorbidities using hierarchical condition categories (HCC) from August 2019 to September 2021. Comorbidities HCC flags include: chronic kidney disease, ESRD, chronic obstructive pulmonary disorders (COPD), other respiratory disease, cardiac disorder, diabetes, immune deficiency, severe neurologic condition, cancer, hypertension, dementia (including Alzheimer's), breast/prostate cancer, HIV/AIDS, obesity. In addition to individual condition flags, the models included a count of these comorbidities (0,1,2,3,4+). Local area characteristics includes log tract density - the log of the population density at the census tract level based on American Community Survey 2017 data), with extreme values trimmed. Density is calculated as the number of persons living in a census tract divided by the square mileage of that tract. For tracts without density data, values were imputed. To capture differences in population demographics and resources, the study included CDC's Social Vulnerability Index (SVI) at the census tract level, which include four SVI themes: socioeconomic status, household composition & disability, minority status & language, and housing type and transportation. The model included state fixed effects (to control for fixed differences across states such as state-level mitigation policies that are in place throughout the study period) and month fixed effects to control for temporal trends unrelated to cumulative county vaccination rates. We considered county fixed effects but we opted for state-level instead due to concerns with model convergence and over- parameterization. As discussed earlier, Texas and Hawaii were excluded from the model due to the lack of county-level vaccination data. To estimate potential reduction in COVID-19 outcomes for these two states, we took the state-wide reported vaccination rate, and used a state with a similar vaccination rate (California) to estimate COVID-19 outcomes based on the size of the Medicare population in each of the two states. Sensitivity Analyses In sensitivity analyses, we ran an alternate model with data from January - May 2021 since data prior to the availability of vaccines do not contribute to the estimated effect of vaccinations on COVID-19. Both temporal trends and vaccination rates over time are modeled non-parametrically without a linear assumption. We modeled the relationship between vaccination rates and COVID-19 outcomes over time using 17 categories of community vaccination rates (starting at 0%, then increments of 2% up until 10%; OCTOBER 2021 RESEARCH REPORT 24 followed by increments of 5% until 60%). We replaced monthly fixed effects with weekly fixed effects to better control for changes in temporal trends. Otherwise, all other variables in the model and the forms of the regression models were the same as the main analysis. The estimates from this alternate model were lower than the main model and provide a lower bound on the estimated reductions in COVID-19 outcomes associated with COVID-19 vaccinations. Among the study cohort in this model, COVID-19 vaccinations were associated with an estimated reduction of 35,000 infections (vs. 107,000 estimated by the main model), a reduction of 14,000 hospitalizations (vs. 43,000) and 4,900 deaths (vs. 15,000). When projected to the full Medicare population, these alternate model estimates suggest COVID-19 vaccination prevented at least 87,000 infections, 34,000 hospitalizations and 12,0000 deaths among all Medicare beneficiaries. All estimates are rounded. The data was prepared, and analyses conducted by Acumen LLC under the supervision of ASPE. Detailed Methodology Main analysis: The specific regression models and approaches to estimating the number of COVID-19 outcomes based on vaccination rates and the counterfactual without vaccines are described next. L Estimation We estimated three models: (i) the probability of a COVID-19 diagnosis, (ii) the probability of death within 3 weeks of a COVID-19 diagnosis, and (iii) the probability of a hospitalization within 9 weeks of a COVID-19 diagnosis. From these models we obtain the predicted number of deaths and hospitalizations for each state based on actual vaccination rates and the corresponding numbers based on the counterfactual of zero vaccinations. The COVID-19 diagnosis model was estimated by a discrete time hazard model of the probability of a diagnosis at the beneficiary-week level and the hospitalization and mortality models by logits. Study period: 9/6/2020 to 5/29/2021, yielding 38 (week) periods. 1. Probability of a COVID-19 diagnosis Data: Beneficiary-period level dataset for entire Medicare FFS population, restricted to those who were continuously enrolled during the 38-week study period and the lookback period from Aug 2019 to Sep 2020 used for measuring comorbidities. Beneficiaries will contribute observations from the first period until the period in which they first receive a COVID-19 diagnosis (first time during the study period). Beneficiaries are censored in subsequent periods. Beneficiaries who do not receive a COVID-19 diagnosis during the study period will contribute observations for all 38 periods. Outcome: COVID-19 diagnosis in the period (binary) OCTOBER 2021 RESEARCH REPORT 25 Model: Discrete-time hazard model, estimated by a logit 2. Probability of First Hospitalization Conditional on COVID-19 Diagnosis Data: Beneficiary-period level datasets for those beneficiary-periods in which a COVID-19 diagnosis first Occurs. Outcome: Hospitalization within 3-weeks of the COVID-19 diagnosis Model: Logit 3. Probability of Mortality Conditional on COVID-19 Diagnosis Data: Beneficiary-period level datasets for those beneficiary-periods in which a COVID-19 diagnosis first occurs. Outcome: Mortality within 9-weeks of the COVID-19 diagnosis Model: Logit i. Predicting Outcomes and Counterfactual Outcomes Prediction: From the results of these models, we predicted the probability of contracting COVID-19 in each period: P(COVID in period t), the probability of hospitalization conditional on contracting COVID: P(hospitalization | COVID in period t), and the probability of death conditional on contracting COVID: P(death | COVID in period t) based on observed covariates, including observed vaccination rates. We also predicted counterfactual hazards and probabilities based on observed covariates except for vaccination rates, which were set to zero. The probability of contracting COVID in each period was constructed from the estimates of the hazard model: P(COVID in period t) = S(t-1)*h(t), where S(t) is the probability of not contracting COVID prior to period t (the survival probability) and h(t) is the hazard of contracting COVID in period t. From these values we predicted the probability of hospitalization for each beneficiary in each period as P(hospitalization with COVID-19 in period t) = P(COVID in period t) * P(hospitalization | COVID in period t). We similarly predicted the counterfactual probability of hospitalization. Summing these predicted probabilities over beneficiaries and periods by state yielded an estimated number of hospitalizations over the study period by state and the counterfactual number of hospitalizations over the study period by state (setting vaccination rates equal to zero). We predicted the probability of death for each beneficiary in each period as P(death with COVID-19 in period t) = P(COVID in period t) * P(death | COVID in period t) and similarly predicted the counterfactual probability of death. Summing these predicted probabilities over beneficiaries and periods by state OCTOBER 2021 RESEARCH REPORT 26 yielded an estimated number of deaths over the study period by state and the counterfactual number of deaths over the study period by state (setting vaccination rates equal to zero). The probabilities predicted above were also aggregated along other beneficiary categories of interest including state, race/ethnicity and long-term nursing home residents. REFERENCES 10. 11. Johns Hopkins University. Confirmed COVID-19 cases and deaths as of 12.31.2020. 2021; https://coronavirus.jhu.edu/us-map. Accessed September 14, 2021. Centers for Disease Control and Prevention. Race, Ethnicity, and Age Trends in Persons Who Died from COVID-19 - United States, May-August 2020. 2020; https://www.cdc.gov/mmwr/volumes/69/wr/mm6942e1.htm. Accessed September 15, 2021. Centers for Disease Control and Prevention. Interim Estimates of COVID-19 Vaccine Effectiveness Against COVID-19--Associated Emergency Department or Urgent Care Clinic Encounters and Hospitalizations Among Adults During SARS-CoV-2 B.1.617.2 (Delta) Variant Predominance - Nine States, June-August 2021. 2021; https://www.cdc.gov/mmwr/volumes/70/wr/mm7037e2.htm?s_ cid=mm7037e2_e&ACSTrackingID=USCD C_921-DM65565&ACSTrackingLabel>MMWR%20Early%20Release%20- %20Vol.%2070%2C%20September%2010%2C%202021&deliveryName=USCDC_921-DM65565. Accessed September 16, 2021. Centers for Disease Control and Prevention. Effectiveness of COVID-19 mRNA Vaccines Against COVID-19- Associated Hospitalization - Five Veterans Affairs Medical Centers, United States, February 1-August 6, 2021. 2021; https://www.cdc.gov/mmwr/volumes/70/wr/mm7037e3.htm?s cid=mm7037e3 e&ACSTrackingID=USCD C_921-DM65565&ACSTrackingLabel>MMWR%20Early%20Release%20- %20VOI.%2070%2C%20September%2010%2C%202021&deliveryName=USCDC 921-DM65565. Accessed September 16, 2021. Centers for Disease Control and Prevention. About COVID-19 Vaccine Delivered and Administration Data. 2021; https://www.cdc.gov/coronavirus/2019-ncov/vaccines/distributing/about-vaccine-data.html. Accessed September 1, 2021. Bosworth A, Finegold, K., Samson, L., Sheingold, S., Tarazi, W., Zuckerman, R. Risk of Covid-19 Infection, Hospitalization, and Death in Fee-For-Service Medicare (Issue Brief No. HP-2021-05). 2021; https://aspe.hhs.gov/system/files/pdf/265271/risk-score-issue-brief. pdf. Accessed April, 7, 2021. Simmons A, Chappel, A., Kolbe, A., Bush,L., Sommers, B. Health disparities by race and ethnicity during the covid-19 pandemic: current evidence and policy approaches. https://aspe.hhs.gov/reports/health- disparities-race-ethnicity-during-covid-19-pandemic-current-evidence-policy-approaches. Accessed September 15, 2021. Freed M, Fuglesten Biniek, J, Damico, A, Neuman, T. Medicare advantage in 2021: enrollment update and key trends. 2021; https://www.kff.org/medicare/issue-brief/medicare-advantage-in-2021-enrollment- update-and-key-trends/. Accessed September 8, 2021. Tarazi WW, Finegold K, Sheingold SH, Wong Samson L, Zuckerman R, Bosworth A. COVID-19-Related Deaths And Excess Deaths Among Medicare Fee-For-Service Beneficiaries. Health affairs (Project Hope). 2021;40(6):879-885. Gupta S, Cantor J, Simon KI, Bento Al, Wing C, Whaley CM. Vaccinations Against COVID-19 May Have Averted Up To 140,000 Deaths In The United States. Health affairs (Project Hope). 2021:101377hIthaff202100619. Galvani A, Moghadas,S., Schneider, E. Deaths and hospitalizations averted by rapid U.S. vaccination rollout. 2021; https://www.commonwealthfund.org/publications/issue-briefs/2021/jul/deaths-and- hospitalizations-averted-rapid-us-vaccination-rollout. Accessed September 9, 2021. OCTOBER 2021 RESEARCH REPORT 27 12. 13. 14, 15. 16. 17. Centers for Disease Control and Prevention. Delta variant: what we know about the science. 2021; https://www.cdc.gov/coronavirus/2019-ncov/variants/delta-variant.html. Accessed September 8, 2021. Centers for Disease Control and Prevention. Effectiveness of COVID-19 Vaccines in Preventing SARS-CoV-2 Infection Among Frontline Workers Before and During B.1.617.2 (Delta) Variant Predominance - Eight U.S. Locations, December 2020-August 2021. 2021; https://www.cdc.gov/mmwr/volumes/70/wr/mm7034e4.htm?s_ cid=mm7034e4_w. Accessed September 14, 2021. Centers for Disease Control and Prevention. Decreases in COVID-19 Cases, Emergency Department Visits, Hospital Admissions, and Deaths Among Older Adults Following the Introduction of COVID-19 Vaccine - United States, September 6, 2020-May 1, 2021. 2021; https://www.cdc.gov/mmwr/volumes/70/wr/mm7023e2.htm?s_ cid=mm7023e2_e&ACSTrackingID=USCD C_921-DM59041&ACSTrackingLabelEMMWR%20Early%20Release%20- %20Vol.%2070,%20June%208,%202021&deliveryName=USCDC_921-DM59041. Accessed September 15, 2021. Centers for Medicare and Medicaid Services. Death Information in the Research Identifiable Medicare Data. 2021; https://resdac.org/articles/death-information-research-identifiable-medicare-data. Accessed September 10, 2021. Kaiser Family Foundation. An Overview of Medicare. 2019; https://www.kff.org/medicare/issue-brief/an- overview-of-medicare/. Accessed September 13, 2021. Koma W CJ, Neuman T,. A Snapshot of Sources of Coverage Among Medicare Beneficiaries in 2018. 2021; https://www.kff.org/medicare/issue-brief/a-snapshot-of-sources-of-coverage-among-medicare- beneficiaries-in-2018/. Accessed September 16, 2021. OCTOBER 2021 RESEARCH REPORT 28 U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES Office of the Assistant Secretary for Planning and Evaluation 200 Independence Avenue SW, Mailstop 447D Washington, D.C. 20201 For more ASPE briefs and other publications, visit: aspe.hhs.gov/reports ORC =] ABOUT THE AUTHORS Lok Wong Samson is a Social Science Analyst and Wafa Tarazi is an Economist in the Office of Health Policy in the Office of the Assistant Secretary for Planning and Evaluation. E, John Orav is Associate Professor in Biostatistics at the Harvard T.H Chan School of Public Health, and Brigham & Women's Hospital in Massachusetts. Steven Sheingold is Director of Healthcare Financing Policy in the Office of Health Policy Nancy De Lew is the Associate Deputy Assistant Secretary of the Office of Health Policy in the Office of Assistant Secretary for Planning and Evaluation. Benjamin D. Sommers is the Deputy Assistant Secretary of the Office of Health Policy in the Office of Assistant Secretary for Planning and Evaluation. The authors are grateful to the contributions of Acumen LLC to the analysis of this report SUGGESTED CITATION Samson, LW, Tarazi, W, Orav, EJ, Sheingold, S, De Lew, N and Sommers, BD. Associations Between County-level Vaccination Rates and COVID-19 Outcomes Among Medicare Beneficiaries. (Research Report No. HP-2021-23). Washington, DC: Office of the Assistant Secretary for Planning and Evaluation, U.S. Department of Health and Human Services. October, 2021. COPYRIGHT INFORMATION All material appearing in this report is in the public domain and may be reproduced or copied without permission; citation as to source, however, is appreciated. DISCLOSURE This communication was printed, published, or produced and disseminated at U.S. taxpayer expense. Subscribe to ASPE mailing list to receive email updates on new publications: https://list.nih.gov/cgi-bin/wa.exe?SUBED1=ASPE-HEALTH-POLICY&A=1 For general questions or general information about ASPE: aspe.hhs.gov/about OCTOBER 2021 RESEARCH REPORT 29