The Impact of Population Health Analytics on Health Care Quality and Efficacy Among CPC+ Participants 7 5 NOT ABSTRACT Analysis of the Comprehensive Primary Care (CPC) program found that the lack of infrastructure for health information exchange was a barrier for primary care practices participating in this federal multipayer advanced primary care program. This study examines whether population health management information technologies offered through a health information exchange (HIE) platform-which collects and stores data across providers-helped practices in the follow-up program, Comprehensive Primary Care Plus(CPC+), achieve better outcomes. We conducted a retrospective study to examine several outcome measures among a sample of 37 participating primary care practices in western New York. These practices were grouped in four categories based on their participation in CPC+ and population health information technology services. For the period of January 2020 to December 2020, practices with membership in both CPC+ and HIE saw 24.1% lower risk-adjusted hospital admission rates and 21.0% lower risk-ad- justed outpatient surgery rates compared with practices that didn't participate in either one. Average lengths of stay in hospitals were 32.7% lower and readmission rates were 30.4% lower. Given these observations, which are based on a robust and mature HIE system, we encourage the medical practices that participated in the CPC+ program or are engaged in other multipayer advanced primary care opportunities to also subscribe to population health management information technologies. INTRODUCTION Through a partnership of Medicare, state Medicaid agencies, and private payers, the Comprehensive Primary Care Plus(CPC+) program provided additional resources and incentives for primary care clinicians to improve quality, access, and efficacy of their services. Such improvements were achieved through five key functions: (1) Access and Continuity; (2) Care Management; (3) Comprehensiveness and Coordination; (4) Patient and Caregiver Engagement; and (5) Planned Care and Population Health. The structure of the CPC+ initiative is based on the lessons learned from the earlier Comprehensive Primary Care Initiative (CPC), a multipayer project spearheaded by the Centers for Medicare and Medicaid Services (CMS) between 2012 and 2016 in seven US regions. The goal of the previous program, which included nearly 500 primary care practices, was to study wheth- er payment reforms and incentives to use electronic health records would help primary care providers to improve care and lower their cost. The CPC program had modest effects on hospitalizations and emergency department (ED) visits but did not significantly reduce the costs of care. According to an analysis by CMS, the partici- pating practices faced many challenges, including "burden associated with quality monitoring and reporting, existing [fee-for-service] incentives that encourage volume, and lack of infra- structure for comprehensive and efficient health information exchange between health care providers." Medical providers who chose to be a part of the CPC+ program were "larger, more sophisticated electronic health record users, more likely to be owned by a hospital or health system, more likely to have experience with transformation efforts, and more likely to be in urban areas" than their Milbank Memorial Fund e www.milbank.org 2 counterparts.? These practices were also more likely to serve patient populations that are wealthier, more educated, and healthier.' Since the practices that volunteered to be a part of the CPC+ program tended to be more resourceful and also serve populations with better health status, they were expected to easily achieve CPC+ targets in quality and efficacy of health care services. Surprisingly, studies on the impact of CPC+ on the first two years of its implementation show that although the program led to incremental improvements in quality of care, patient experience, and service utilization, the savings from the program were not enough to cover its costs in that initial time period.*® Despite the alignment of economic incentives between payors and providers, one potential reason that CPC+ did not meet its high expectations could have been limited digital capabili- ties. Access to comprehensive medical information, seamless information flow between pro- viders and patients, and reliable analytics and information-processing tools would potentially help CPC+ participants to achieve the key functionalities of the program more easily. A health information exchange (HIE) platform collects, organizes, and stores medical data from various providers within a geographical region ona centralized or decentralized da- tabase. HIE participants with appropriate authorizations can then access the medical data of patients with their consent.® Digitally mature and well-functioning HIE platforms are well equipped to provide their members with the information services necessary to excel in all key areas of the CPC+ program. HEALTHeLINK, the HIE in the setting studied in this report, is a very well established, widely adopted HIE platform that has been operating in the region since 2006. The long history of HEALTHeLINK combined with its high adoption rate in the region give it a unique opportunity to consolidate comprehensive medical data of the patients in western New York. Such wealth of data, coupled with state-of-the-art analytics, allows HEALTHeLINK to develop digital capabilities for population health management monitoring and control. HEALTHeLINK provides a population health management tool called HEALTHeQUTCOMES. This tool provides near real-time practice and provider performance data for a large set of common clinical quality measures that is payer agnostic. In addition, the system provides this same payer-agnostic data for a set of cost and utilization measures. This allows practices to better identify trends in their practice instead of looking separately at disparate reporting from multiple data sources. Practices gain workflow efficiencies and save time by having their quality performance all in one place. In addition, HEALTHeOQUTCOMES delivers predictive analytics and risk scores using HEALTHeLINK's community data set of more than 400 data sources. This feature identifies patients who may generate potentially higher cost or be at higher risk for adverse events, so that practices can build out their care management programs. HEALTHeOQUTCOMES also generates robust chronic condition registries that include diagnoses from other providers and facilities across the community, which help practices stratify patients for more effective and accurate disease-specific interventions. Although prior research has conclusively demonstrated the effectiveness of using HIE services on reducing costs and increasing quality of medical services,'"® enticing providers to Milbank Memorial Fund e www.milbank.org 3 actively use these services to their fullest potential remains a challege.""" The CPC+ program provided an economic incentive structure for medical practices to actively use the services of available HIE platforms. As depicted in Figure 1, we considered four groups of practices based on their digital capa- bilities and the incentives. The first group includes those that have digital capabilities and also have the incentive to use such capabilities. The second group includes those that have the digital capabilities but not the economic incentive to put them to use. The third group consists of practices that need information capabilities, yet do not have them. The fourth group, which will be the baseline for our comparisons, consists of those practices that have neither digital capabilities nor the economic incentives to acquire them. The objective of this study was to examine the differences in outcomes across these four groups and analyze whether the medical practices that were enrolled in the CPC+ program and use HEALTHeQUTCOMES information services achieve better outcomes than their counterparts. Figure 1. Venn Diagram of the Four Groups of Practices Group 2: Group 3: HIE members CPC+ members Availability of digital capabilities Incentives to use digital capabilities Group 4: All other practices In this report we compare the performance of the practices among the four groups based on a set of widely used metrics and provide a set of recommendations for policymakers, medical providers, and HIE platforms. Milbank Memorial Fund e www.milbank.org 4 FINDINGS In our study, we first examined the impact of CPC+ membership on various indicators of qual- ity and efficiency by comparing the mean values of these indicators between groups 3(CPC+ only) and 4 (baseline). This design reveals associations between any difference in outcomes and CPC+ program participation, as it reduces the chance for the observed differences to be confounded by the potential effects of the HEALTHeQUTCOMES service. We then examined the potential impact of the HEALTHeOQUTCOMES service on the same indicators by compar- ing groups 2 (HIE only) and 4 (baseline). Finally, we examined whether a combination of both CPC+ and HEALTHeOUTCOMES is associated with significant improvements in outcomes by comparing groups 1(CPC+ and HIE) and 4 (baseline). (See Table 1.) Table 1. Pairwise Comparison Between Groups Comparison Outcome Group 3 vs Group 4 Association with CPC+ only Group 2 vs Group 4 Association with HEALTHeQUTCOMES only Group 1 vs Group 4 Association with both CPC+ and HEALTHeQUTCOMES To measure the quality and efficacy of health care services, we use the following metrics: rate of admissions, imaging, laboratory tests, outpatient surgeries, ED visits, and urgent care visit per 1,000 patients. We also measure inpatient length of stay in days and percent- age of readmissions to the hospital within 30 days from discharge. All of these metrics are risk-adjusted at the practice level, even though there was no statistical difference in the average Adjusted Clinical Group (ACG) risk score for the groups. Impact of membership in CPC+ program As reported in Table 2, in 2020, members of the CPC+ program outperformed their counter- parts in only one metric: inpatient length of stay. This metric has an average of 10.83 days for CPC+ members. Nonmembers have a significantly longer length of stay with an average of 15.72 days. Our findings about the impact of the CPC+ program on outcomes are fairly consistent with prior literature, which has also found that the impact of the program appears limited to small improvements.**® Milbank Memorial Fund e www.milbank.org 5 Table 2. Average Performance on Quality and Efficacy Metrics (January-December 2020) HIE Plus HIE Only | CPC+Only | Non-HIE | Group 3 vs | Group 2 vs | Group 1vs CPC+ (Group 2) | (Group 3) and (Group 1) Non-CPC+ (Group 4) Admis- . 42.73 47.20 46.21 56.30 sions per 0.11 0.40 0.05 (4.23) (4.10) (9.72) (19.88) 1,000 Inpatient 10.58 10.83 length of 9.01(5.15) 15.72 (7.16) 0.03 0.15 0.05 (2.39) (3.23) stay Imagin 99 | 2.00(0.34) | 1.49(0.47) | 1.67(0.64) | 1.65(0.65) | 0.94 0.89 0.18 per 1,000 Labs per 35.62 33.37 35.05 32.50 0.24 0.79 0.18 1,000 (3.45) (3.28) (4.07) (5.56) Outpa- tient 9.88 (0.60) | 12.24 (1.78) | 11.08(1.86) | 12.50 (3.18) 0.20 0.89 0.02 surgery per 1,000 ED visits 2.84 (0.61) | 3.99(1.30) | 3.55(0.80) | 2.89(1.78) 0.23 0.32 0.94 per 1,000 Urgent 29.85 care visits | 31.19(2.24) 2 54) 31.11(2.37) | 32.83 (2.11) 0.13 0.08 0.17 per 1,000 , Readmis- . 6.52(0.88) | 7.24(1.02) | 8.33(3.04) | 9.37(4.34) 0.53 0.37 0.06 sions % @ Standard deviations are reported in parentheses. P-values of pairwise comparison t-test are reported in the last three columns. Impact of subscription to HEALTHeOUTCOMES service We found that practices that have signed up for only HEALTHeOUTCOMES do not outper- form their counterparts in any of the metrics, except for rate of urgent care visits per 1,000 patients. As reported in Table 2, this metric has an average of 29.85 visits per 1,000 patients for HEALTHeQUTCOMES members. Nonmembers have a significantly higher rate of urgent care visits with an average of 32.83 visits per 1,000 patients. Note that we consider p-values that are equal to or less than 0.1 as indication of statistical significance. Milbank Memorial Fund e www.milbank.org 6 Impact of both CPC+ membership and HEALTHeQUTCOMES subscription Over the year 2020, the practices that were members of CPC+ and subscribers to HEALTHeOUTCOMES had much lower rates for admissions (42.73 per 1,000 people vs. 56.3 per 1,000 people) and outpatient surgery (9.88 per 1,000 people vs. 12.5 per 1,000 people). Moreover, these practices also outperformed their counterparts in length of stay and readmission percentages. Patients of practices that were members of both CPC+ and HEALTHeLINK spent much shorter time in hospitals (10.58 days vs. 15.72 days) and were much less likely to be readmitted within 30 days of discharge (6.52% vs. 9.37%). That is, over the year 2020, membership in both programs together has led to a 24.1% reduction in admission rate and a 21.0% reduction in outpatient surgery. It has also reduced the length of stay in hospitals by 32.7% and the readmission rate by 30.4%. HOW WE CONDUCTED THIS STUDY In partnership with HEALTHeLINK, the Regional Health Information Organization (RHIO) of western New York, we collected data on four different groups of medical practices in the region. To ensure that the differences in outcome metrics are not based on the inherent differences of the practices across the four groups, we selected our sample such that their observable features were statistically equal across all four groups. We compared practices based on three sets of features: practice level features, their experience with and use of HIE, and characteristics of their patients. Although these features encompass a wide variety of observable characteristics of practices in our sample, a more comprehensive data set would have allowed us to further refine such groupings. Practice features are focused on the practice itself and include the number of physicians, the location, ownership status, and experience with value-based payment models. HIE features measure how long and how much a practice uses the HIE. Patient features are focused on the population of patients served by a practice. They include number of Medicare beneficiaries served, the patients' various health conditions, and overall ACG score. Table 3 presents summary statistics for these groups. For each feature we conducted the appropriate statistical tests to examine whether there is a significant difference between the averages of metrics among groups. Specifically, for the comparisons between more than two groups, we conducted an analysis of variance (ANOVA) test, while for the comparisons between two groups, we conducted Student's t-test. The p-values of each test are reported in the last column of Table 3 and suggest that there is no statistical difference between the averages of features among these groups. There is no statistically significant difference between the four groups when it comes to the number of their primary care physicians, location, or ownership status. While equivalent proportion of practices among all groups are patient-centered medical home (PCMH) mem- bers, their experience with accountable care organization (ACO) membership is different. In particular, ACO participation may be associated with experience in data consumption, population health management, and accountability for cost and quality of delivered services. Milbank Memorial Fund e www.milbank.org 7 Specifically, all of the practices in group 4 are members of an ACO while none of those in group 2 are ACO members. Although the practices have some differences in the number of Medicare patients and the medical history of their patients, they are very similar overall when it comes to the ACG score, which is a much more accurate measure for an individual's con- sumption of health services. It should also be noted that most of the practices are located in suburban areas, which have significant demographic differences compared to urban and rural communities in western New York. We measure HIE use by the number of times a practice logs into the system per week. The HIE use patterns vary among different groups. Obviously, groups 3 and 4 have no HIE use since they are not members of the platform. Interestingly, practices in group 1{which are members of both HIE and CPC+ programs) use HIE more than their counterparts in group 2 (which are members of HIE only); however, the difference in HIE use is not statistically significant (p-value = 0.36). We measure the level of system experience by the number of days since the practice has adopted HIE use. This is indicated by variable "HIE experience" in Table 3. Note that Groups 1 and 2 have the same level of experience with the system. The practices are also similar in their ACG score. The Johns Hopkins ACG risk adjustment on practice panels, utilization, and cost metrics allows for a more realistic, "apples to apples" comparison of performance across practices, as it takes into account the health factors of the patients attributed to each practice. Risk adjustment is a necessary component in the evaluation of any health care organization's performance when comparing against a cohort of its peers. This is an industry-standard algorithm that provides a population case mix adjustment by grouping diagnoses into 1 of 32 clinical groups based on ICD 9 & 10 codes. Every diagnosis is evaluated on duration, severity, diagnostic certainty, etiology, and specialty care involvement needed. Based on the results presented in Table 3, we conclude that the practices selected to be included in the analysis are comparable and similar to each other in many of their fea- tures, except for the experience with ACOs, which is nonexistent among practices in group 2. We will elaborate on the potential impact of this difference on our results in the Conclusion. Since the groups are systematically designed to be different in their CPC+ and HEALTHeOQUTCOMES membership status, any differences in their quality and efficiency metrics could be attributed to either HIE use, membership in CPC+, or a combination of both. Milbank Memorial Fund e www.milbank.org 8 Table 3. Average Values of Observable Features of Practices Included in the Analysis* Feature a een, CPC+ Non-HIE lien LLU) en and Non-CPC+ Number of practices 10 4 17 6 - Avg. number of primary care 8.2 8.0 78 6.0 0.70 physicians (2.7) (4.5) (4.1) (4.0) 0% 25% B% 16% % in rural location 0.40 {0) (0.5) (0.24) (0.41) 0% 0% 6% 0% 8 % in urban location 0.77 5 (0) (0) (0.24) (0) if 3 100% 75% 88% 84% ' % in suburban location 0.54 © (0) (0.50) (0.33) (0.41) oo] © 100% 100% 76% 83% a. % of independently owned 0.32 (0} (0) (0.44) (0.41) 50% 0% 94% 100% % With ACO experience <0.001 (0.52) (0) (0.24) (0) °% with PCMH 100% 100% 100% 83% 01g membership (0) (0) (0) (0.41) o 87.4 50.0 @ HIE use (# of log-in times) - - 0.36 Z (73.9) (33.2) o LL 436.1 449.8 Lu : : = |HIE experience - - 0.82 <= (131.9) (136.5) Milbank Memorial Fund e www.milbank.org 9 1618.9 800.0 1301.2 723.5 Medicare patients 0.03 (424.9) (108.9) (779.1) (415.4) Patients with 0.005 0.196 0.213 . 0.103 (0.121) <0.001 hyperthyroidism (0.006) (0.063) (0.054) Patients with 0.545 0.589 0.585 a 0.592 (0.118) 0.89 w | hyperlipidemia (0.181) (0.139) (0.156) o 5 . . 0.116 0.139 0.138 = Patients with COPD 0.115 (0.013) 0.25 § (0.032) (0.034) (0.042) Li | Patients with 0.643 0.872 0.641 0.658 0.76 o = hypertension (0.050) (0.034) (0.0632) (0.074) a . . 0.079 0.080 0.084 0.087 Patients with CHF 0.88 (0.018) (0.024) (0.019) (0.031) . oo 0.224 0.247 0.241 0.235 Patients with diabetes 0.52 (0.023) (0.032) (0.031) (0.049) 3.59 3.54 3.56 3.56 ACG score 0.89 (0.07) (0.07) (0.12) (0.17) Abbreviations: ACG, Adjusted Clinical Group; ACO, accountable care organization; CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease; HIE, health information exchange; PCMH, patient-centered medical home. @Standard deviations are reported in parentheses. LIMITATIONS In this study, we have examined only a small number of primary care practices within a relatively small geographical area. Additionally, HEALTHeLINK, the HIE platform that provides the population health information technology solutions to the primary care practices in our study, is an exceptionally mature HIE platform that has been operating in the region for a very long time and has accumulated a significant wealth of medical data through its partnerships with all major medical providers in the region. It is therefore important to note that the findings from the current study may not be generalizable to other settings with different IT capabilities and patient populations. It is also important to note that given the design of this study, we have not differentiat- ed between HEALTHeEOQUTCOMES members based on their level of use of the system. A panel data set that identifies metrics of system use and outcomes at each practice level over different time periods would allow us to examine the extent to which an increase in HEALTHeOUTCOMES use leads to an improvement in outcomes. This could be examined in future studies. Milbank Memorial Fund « www.milbank.org 10 CONCLUSION The CPC+ program had significant potential to improve the quality and efficacy of health care services. Although the implementation of the program has led to some improvements, there are more benefits to be extracted. The program provided an economic incentive structure for medical providers to better manage the care of their patients and reduce costs while increasing the quality. To achieve their goals, members of the CPC+ program could benefit from access to specific information technologies that could be offered by mature HIE platforms. These technologies go beyond the simple exchange of medical records and include capabilities to analyze com- prehensive medical data for population health management. HEALTHeLINK, the regional HIE of western New York, is one of very few platforms to provide this type of population health management capability through a service called HEALTHeEOUTCOMES. Although the CPC+ program creates the incentives to reach higher goals, HEALTHeOUTCOMES provides the information that can help practices achieve them. We examined performance of four groups of practices that were comparable in a number of important aspects except for their CPC+ membership and HEALTHeQUTCOMES use to see if those practices with the incentives and the means for improvement perform significantly better than their counterparts. The current results confirm our contention and show that when both of the programs are in place at the same time, practices tend to have significantly better outcomes as compared to instances where only one of the programs is put in place. We observed that CPC+ members that are also HIE subscribers use HIE more than those subscribers that are not participating in the CPC+. However, this difference was not statisti- cally significant. We could have probably observed even stronger improvement in outcomes if the difference in HIE use was more salient. Furthermore, none of the practices in group 2 (members of HEALTHeQUTCOMES only) had prior experience with ACOs, unlike the rest of the groups. Assuming that ACO experience would have a positive impact on outcomes, this means that the impact of HEALTHeQUTCOMES (group 2) is being systematically underesti- mated in our analysis. With the national efforts to shift payment models toward value-based systems, CMS may be considering introducing other advanced primary care payment models. In this study we observe that the primary care practices that have subscribed and used population health information technologies such as HEALTHeOQUTCOMES tend to outperform their counterparts in various metrics. We therefore recommend that policymakers promote the development, adoption, and use of digital population health management tools that can offer capabilities similar to the ones studied in this research. Given the findings of our research on the ob- served improvements provided through HEALTHeEOQUTCOMES, we propose that other HIE platforms enhance their value proposition for their members by adding more sophisticated analytical tools on top of their current exchange services and offer their members population health management tools similar to the one studied here. Milbank Memorial Fund e www.milbank.org 11 NOTES ' Centers for Medicare and Medicaid Services. Synthesis of Evaluation Results Across 21 Medicare Models, 2012-2020. Baltimore, MD: Centers for Medicare and Medicaid Services; 2020. 2. Singh P, Orzol S, Peikes D, Oh EG, Dale S. Participation in the Comprehensive Primary Care Plus initiative. Ann Fam Med. 2020;18(4):309-317. 3. Fraze TK, Fisher ES, Tomaino MR, Peck KA, Meara E. Comparison of populations served in hospital service areas with and without Comprehensive Primary Care Plus medical homes. JAMA Netw Open. 2018;1(5):e182169-e182169. * Peikes D, Anglin G, Harrington M, et al. Independent Evaluation of Comprehensive Primary Care Plus (CPC+): First Annual Report. Princeton, NU: Mathematica Policy Research; 2019. 5. Peikes D, Swankoski K, Timmins L, et al. Independent Evaluation of the Comprehensive Primary Care Plus (CPC+): Third Annual Report. Princeton, NU: Mathematica Policy Research; 2021. 8. Yaraghi N, Du AY, Sharman R, Gopal RD, Ramesh R. Health information exchange as a multisided platform: adoption, usage, and practice involvement in service co-production. Inf Syst Res. 2015;26(1):1-18. 7. Yaraghi N. An empirical analysis of the financial benefits of health information exchange in emergency departments. J Am Med Inform Assoc. . Published 2015. Accessed September 6, 2015. 8. Fecher K, McCarthy L, Porreca DE, Yaraghi N. Assessing the benefits of integrating health information exchange services into the medical practices' workflow. Information Systems Frontiers. 2021;23(3):599-605. 3. Menachemi N, Rahurkar S, Harle CA, Vest JR. The benefits of health information exchange: an updated systematic review. U Am Med Inform Assoc. 2018;25(9):1259-1265. Lin SC, Everson J, Adler-Milstein J. Technology, incentives, or both? Factors related to level of hospital health information exchange. Health Serv Res. 2018;53(5):3285-3308. " Yaraghi N. A Sustainable Business Model for Health Information Exchange Platforms: The Solution to Interoperability in Healthcare IT. Washington, DC: Brookings Institution, 2015. Milbank Memorial Fund e www.milbank.org 12 ABOUT THE AUTHORS Daniel Porreca is executive director of HEALTHeLINK, the health information exchange, and HEALTHeNET, the administrative data exchange for Western New York(WNY). Mr. Porreca has been leading HEALTHeLINK since 2007. He serves on the governing board of the Buffalo Translational Consortium that oversees the Clinical and Translational Science Institutes (CTSI). Mr. Porreca also serves on the New York eHealth Collaborative (NYeC) Business and Operations Council that manages the Statewide Health Information Network for New York (SHIN-NY), serves as an advisor to the Stella Technology board of directors, and previously served on the board and as board chair for the Strategic Health Information Exchange Collaborative (SHIEC) and on the board of directors for the New York eHealth Collaborative. Niam Yaraghi is an assistant professor of business technology at University of Miami and nonresident senior fellow at the Brookings Institution's Center for Technology Innovation. Mr. Yaraghi has a BSc in Industrial Engineering from the Isfahan University of Technology in Iran, and a MSc from the Royal Institute of Technology in Sweden. He received his PhD in Management Science and Systems from the State University of New York at Buffalo. Milbank Memorial Fund e www.milbank.org 13 About the Milbank Memorial Fund The Milbank Memorial Fund is an operating foundation that aims to improve population health and health equity by collaborating with leaders and decision makers and connecting them with experience and sound evidence. We advance our mission by: e - Identifying, informing, and inspiring current and future state health policy leaders to enhance their effectiveness; e Working with state health policy decision makers to advance primary care transformation, sustainable health care costs, and healthy aging, and e Publishing evidence-based publications and The Milbank Quarterly, a peer-reviewed journal of population health and health policy. The Milbank Memorial Fund is an endowed operating foundation that engages in nonpartisan analysis, study, research, and communication on significant issues in health policy. In the Fund's own publications, in reports, films, or books it publishes with other organizations, and in articles it commissions for publication by other organizations, the Fund endeavors to maintain the highest standards for accuracy and fairness. Statements by individual authors, however, do not necessarily reflect opinions or factual determinations of the Fund. © 2022 Milbank Memorial Fund. All rights reserved. This publication may be redistributed digitally for noncommercial purposes only as long as it remains wholly intact, including this copyright notice and disclaimer. Milbank Memorial Fund 645 Madison Avenue New York, NY 10022 www.milbank.org Milbank Memorial Fund e www.milbank.org 14