= shiadac \ STATE HEALTH ACCESS DATA ASSISTANCE CENTER California Health Care Foundation LAN A MAY 2023 Opportunities to Learn More About Serving Justice-Involved Individuals Through 1115 Demonstration Evaluations Medicaid is a vital source of coverage for the almost four million justice-involved individuals living in the community - individuals who are disproportionately poor and people of color.! Upon release from prison or jail, many of these individuals face significant obstacles navigating and accessing medical and behavioral health care services. Therefore, many state Medicaid programs, as well as the federal government, are looking at ways to improve continuity of coverage, provide seamless transitions back to the community, and reduce disparities in health care access and outcomes for justice-involved individuals.? Justice-involved individuals: Those that are serving sentences in prisons or jails; awaiting trial or sentencing; and under community supervision, such as parole or probation. Prison: A facility (typically operated by a state Department of Corrections or the Federal Bureau of Prisons) that confines persons after they are convicted of a criminal offense. Persons confined in a prison are typically serving an incarceration sentence of more than one year. Jail: A facility (usually operated by a local law enforce- ment authority such as a sheriff, a police chief, or a county or city administrator) that confines persons awaiting trial or sentencing. Persons sentenced to serve a small amount of time (less than a year) may be housed in the local jail. Parole: The release of a prisoner to supervision in the community following the completion of a sentence in an institution. Probation: A period of supervision in the community imposed by the court as an alternative to imprisonment. *Under the Consoli riations A states now have an option to provide Medicaid and CHIP coverage to juvenile youth in public institutions. However, adults still cannot receive Medicaid coverage for most health care services while incar- cerated (except for inpatient care lasting 24 hours or more). State Health Access Data Assistance Center Figure 1. Composition of the Adult Correctional System 2020 Total Correctional System Population: 5,500,600 M Probation I Parole M Prison o Jail Communit supervise Probation 55.5% Source: Bureau of Justice Statistics (BJS), U.S. Department of Justice. 2022. Cor- rectional populations in the United States, 2020-Statistical tables. Washington, DC: BJS. A major obstacle states face in trying to improve access to care for justice-involved individuals is the Medicaid inmate exclusion policy - a provision in the Social Security Act Amendments of 1965 that prohibits use of federal Medicaid funds for most health care services during incarceration.* One way a growing number of states are trying to support reentry for justice-involved individuals is by providing Medicaid services prior to release from prison or jail through a Section 1115 demonstration waiver. Under Section 1115 of the Social Security Act, states can apply for awaiver to test new policies in their Medicaid program that federal rules typically do not allow. In return for these new flexibilities, states are required to monitor and evaluate the impact of these changes. In April 2023 the Centers for Medicare & Medicaid Services (CMS) issued new guidance outlining opportunities for states to design 1115 demonstration projects to improve care transitions for incarcerated individuals. Before the CMS guidance was released, in January 2023, California became the first state to receive 1115 demonstration authority approval to waive the inmate exclusion and provide some Medicaid services in the 90 days pre-release.* As of February 2023, an additional fourteen states have pending reentry demonstration proposals before CMS. Opportunities to Learn More About Serving Justice-Involved Individuals through 1115 Demonstration Evaluations Demonstrations like California's, and the others that have been proposed, could improve states' abilities to coordinate and pay for health care services for justice-involved populations as they transition back into the community. Furthermore, 1115 demonstrations' evaluation requirements present a valuable opportunity to learn about Medicaid's ability to improve outcomes and equity for justice-involved individuals - but only if those evaluations are done well. The objective of this issue brief is to identify the unique opportunities states should consider when designing evaluation plans specific to their justice- involved populations. The brief provides an overview of justice-involved 1115 demonstration initiatives, summarizes what is known from existing evaluations of these activities, and identifies a set of opportunities to design robust and equity-focused 1115 demon- stration evaluation plans specific to justice-involved populations. To complete this work, the State Health Access Data Assis- tance Center (SHADAC) reviewed states' 1115 waiver applications, related CMS guidance and public waiver doc- umentation, relevant grey and peer-reviewed literature, and conducted interviews with 11 key informants with expertise in state Medicaid, federal policy, justice-involved health care, evaluation, and lived experience. Medicaid and the Criminal Justice System The United States incarcerates more people than any other country in the world, and Black, Latino/x, and American Indian and Alaska Native people are substantially overrep- resented among those populations® The imprisonment rate for Black US residents (938 per 100,000), for example, is more than five times higher than that for White residents (183 per 100,000), a result of long-standing structural oppres- sion and racism. ¢ The disparities that these traditionally marginalized populations face in the criminal justice system exist in the health care system, and the wider community, as well. Historically, most of this justice-involved population lacked access to health insurance. They also experience an increased risk of death from drug overdose, cardiovascular disease, violence, and suicide, especially immediately after release from incarceration.' Following implementation of the Affordable Care Act (ACA), states that elected to expand Medicaid found that many jus- tice-involved individuals were newly eligible for Medicaid upon release. From 2015-2019, more than a quarter (28 percent) of adults under community supervision (probation or parole) were enrolled in Medicaid.? This large increase in coverage provided new opportunities and challenges for Medicaid agencies to serve the unique and complex needs of justice-involved populations, as well as reduce racial and ethnic disparities in health care and outcomes. States began using several mechanisms to address the specific needs of justice-involved populations through Medicaid, such as Health Homes, State Plan Amendments, Home and Com- munity-Based Service Waivers, Medicaid Managed Care contracts, and Section 1115 demonstration waivers.® Many states now are interested in pursuing Section 1115 demonstration waivers because they offer the greatest flex- ibility to customize the target population of the model, support data infrastructure improvements, and finance innovative programs through more flexible budget neutral- ity policies.'® Over the past several years, states have used Section 1115 waivers to expand eligibility, provide enroll- ment assistance, and provide housing, employment, and other social supports to justice-involved populations." California is the first state to receive approval to provide a limited set of services to incarcerated individuals 90 days prior to release. As of February 6, 2023, fourteen other states have made similar requests, although the eligible popula- tions, scope, and duration of covered services requested vary. (Table 1) Vermont, for example, is asking to provide full state benefits to all Medicaid eligible incarcerated indi- viduals 90 days prior to release; Montana is requesting per- mission to provide limited community-based consultation, care management, and medication 30 days prior to release for individuals with substance use disorder (SUD), serious mental illness (SMI), or serious emotional disturbance (SED). Opportunities to Learn More About Serving Justice-Involved Individuals through 1115 Demonstration Evaluations Table 1. Characteristics of Medicaid Reentry Section 1115 Demonstrations Hyperlinks to each individual state waiver proposal are included in the header row of this table. Sl A . Eligibility All Adults X X X X Adults with Risk Factors X X X X X X X X X X X Full State Benefits X X* X X X Limited State Benefits X X X X X X X X X* X X Care Coordination/ Case Management Peer Supports X X X X X X X Medications X X X X X X X X Medications for Addiction Treatment In-Reach Physical and/or Behavioral X X X X X X X X Health Services Durable Medical Equipment Housing/Tenancy Supports SUD Services X HIV/HCV Screening and Treatment 12-Month Continuous Eligibility Post-Release Duration of Pre-Release Services 30 Days X X X X X X X X X 45 Days X 60 Days X 90 Days X X X 36 Months Xt State Prisons X X X X X X X X X X X X X X X Jail/Local Correctional X X X X X X X X Facilities Estimated Year 1 e 200,000 | NS 5570 NS 300 588 2000 7500 23971 NS 166 3,200 5500 4,000 12,102 Population Size Notes: NS - Not Specified * Oregon proposes to provide full state benefits to individuals leaving jail and a set of limited benefits and services to individuals leaving prison. * Kentucky proposes to provide substance use disorder services up to 36 months prior to release, and care coordination services up to 30 days prior to release. Source: SHADAC analysis of pending reentry Section 1115 demonstrations, as of February 6, 2023 State Health Access Data Assistance Center w Opportunities to Learn More About Serving Justice-Involved Individuals through 1115 Demonstration Evaluations Current Efforts to Evaluate Justice-Involved 1115 Demonstrations Federal law requires states with Section 1115 demonstra- tions to conduct both monitoring and evaluation activities (see below). In 2017 and 2019, the Centers for Medicare and Medicaid Services (CMS) released guidance designed to strengthen these monitoring and evaluation requirements in order to produce more rigorous findings. For example, CMS instructed states to use an independent evaluator and to call for penalties when evaluation milestones are not met."? In its April 2023 Reentry Section 1115 demonstration guidance, CMS outlined monitoring and evaluation requirements specific to reentry initiatives, many of which are consistent with other approved demonstrations (see Appendix A for a summary of requirements)." Monitoring and Evaluation Monitoring protocols outline the key process metrics thatare used to track progress on implementation goals. This data, reported quarterly and annually throughout the demonstration period, is used to identify areas that might need course correction and to understand how implementation is affecting the program or popula- tion. Typically, monitoring metrics include information such as enrollment information, operational or policy updates, legislative updates, summaries of appeals and grievances, and some quality measures. Evaluation reports assess whether the demonstration has achieved its goals and is used to inform decisions about the future of the policy being tested. States must submit a series of evaluation deliverables: an evaluation design within 180 days of waiver approval; an interim evaluation report one year before the demonstration expires or with the demonstration renewal applica- tion; and a final summative report within 18 months of the end of the demonstration period. Final evalua- tion reports must include: results; conclusions about whether the demonstration met its goals; and a dis- cussion of policy implications, lessons learned, and the study's limitations. Each state funds and directs the scope of its 1115 evaluation, although CMS reviews and provides comments on these evaluation deliverables and must approve them before they become final and are made publicly available. This means that a state has considerable flexibility to decide how to carry out its evaluation, including how much to allocate toward the budget (and whether to pursue outside sources of funding, such as matching grants from foundations); how to make decisions about what hypotheses to test; and which target populations to study. While there are many challenges to designing and carrying out strong evaluations, stakehold- ers and CMS have produced several resources and technical assistance in recent years to guide state decision makers as they design and execute their evaluation plans.' To date, very little is known about how justice-involved pop- ulations have fared under existing 1115 demonstrations. SHADAC's review of publicly available evaluation design plans and reports for seven states with any approved jus- tice-involved related 1115 initiative (not just requests to waive the Medicaid inmate exclusion) found very few signifi- cant effects and very little qualitative data analysis specific to this population (Table 2). Table 2, States with Existing Justice-Involved 1115 Initiatives State Justice-Involved Waiver Provision Approval Date Arizona Targeted Investments 1.0 01/18/2017 California Whole Person Care (WPC) Pilots 12/30/2015 llinois SUD Case Management Pilot 07/01/2018 Rhode Island Peer Recovery Specialists 01/01/2019 Utah Targeted Adult Group 11/01/2017 Washington Accountable Communities for Health 01/09/2017 Virginia High Needs Supports 07/09/2020 For instance, California's recent final evaluation report exam- ining the Whole Person Care Program carried out under the state's previous Section 1115 demonstration which started Jan. 1, 2016, and ended Dec. 31, 2021 did not include any findings specific to justice-involved individuals beyond a summary of what services they received, even though they constituted 25 percent of enrollees in the pilots. Qualitative information was collected from service providers, but not enrollees.' Although Washington had indicated in their CMS approved evaluation design plan for their Medicaid Transformation Project (2017- 2021) that they planned to look at measures specific to justice-involved populations, they ultimately were unable to do so in their interim report spanning 2017 through 2019 due to small sample size, noting: "An additional population, people transitioning from jail, was identified as relevant to the evaluation; however, data were not available to support inclusion of this population for the interim report. This population may be included in subsequent reports."® Opportunities to Learn More About Serving Justice-Involved Individuals through 1115 Demonstration Evaluations Arizona included results for nine outcome measures specific to its justice-involved population in its interim evaluation report of its Arizona Health Care Cost Containment System (AHCCS) waiver, spanning 2017 through 2020. While they did not see any statistically significant differences between the target population and the control group on any measures, observed differences were in the correct direction for several measures including receiving a preventative or ambulatory visit, getting needed care right away, engaging in alcohol and other drug abuse or dependence treatment, and the rate of medication assisted treatment."" SHADAC also reviewed applications of the 14 states with pending demonstration requests to understand what hypotheses and measures states are contemplating evaluat- ing should their reentry demonstration request be approved (See Appendix B). Although the focus of these evaluation questions will most likely evolve now that CMS has released specific reentry demonstration evaluation guidance (see Appendix A), it is interesting to preview what states are con- sidering (See Spotlight on California). In these pending waiver applications, many states are proposing to track typical health utilization measures for their justice-involved initia- tives such as emergency department (ED) visits, behavioral health follow-up rates, and medication assisted treatment (MAT) prescriptions. But several states also discuss including new measures such as self-reported measures of stability and Spotlight on California As the first state to receive approval to provide health care services to justice-involved individuals prior to release from incarceration, California's 1115 demonstration (Cal- ifornia Advancing and Innovating Medi-Cal [CalAIM]) will be looked to as a model for other states. Although the state's reentry initiative evaluation design plan is not due until Aug. 2023, the state's Jan. 26, 2023 approval letter provides insight into CMS' expectations regarding evaluat- ing this specific type of initiative (paraphrased below). The reentry initiative evaluation hypotheses will focus on (but are not limited to): e Cross-system communication and coordination ¢ Connections between carceral and community services, access to, and quality of care in carceral and community settings ¢ Preventive and routine physical and behavioral health care utilization ¢ Non-emergent ED visits, inpatient hospitalizations, and all-cause deaths security and time spent in the community, and one state spe- cifically calls out examining potential reductions in racial and ethnic disparities for this population. It is important to note that justice-involved initiatives are often just one component of the complex, multifaceted inno- vations that states are testing and evaluating in their 1115 demonstrations. lllinois' 1115 demonstration, for example, authorizes the state to implement 10 different pilot projects ranging in enrollment from 320 to 18,000 individuals in each pilot, only one of which is justice-involved (and is capped at 2,040 people).'® Therefore, states often must balance the amount of time and resources devoted to studying one justice-involved initiative with the other programs they are testing. However, given the importance of the historic approval to provide Medicaid services pre-release for the first time, the potential cross-sector impact of justice-involved ini- tiatives, and the potential impact on populations that have been historically marginalized, there is a compelling case for prioritizing this topic in an evaluation. "The justice-involved population has suffered for too long for lack of attention, and we honestly don't know enough even about the impact of current policies. We've got catching up to do." - Stakeholder Evaluation metrics will include: o Utilization of applicable pre-release and post-release services (e.g., case management, MAT, clinical/behav- ioral health assessment pre-release, and primary and behavioral health services post-release) ¢ Administration of screenings to identify individuals who qualify for pre-release services ¢ Provision of health or social service referral pre-release and participants who received case management pre-release and enrolled in enhanced case manage- ment post-release ¢ Take-up of data system enhancements among partici- pating carceral settings ¢ Number of beneficiaries served by types of services rendered Opportunities to Learn More About Serving Justice-Involved Individuals through 1115 Demonstration Evaluations --- Spotlight on California cont. The state will also be expected to provide: ¢ An analysis of the distribution of services rendered over the 90-day coverage period before the beneficia- ry's expected date of release-any relationship iden- tified between the provision and timing of particular services with salient post-release outcomes, including utilization of acute care services for chronic and other serious conditions, decompensation, suicide-related death, overdose, and overdose-related deaths in the period soon after release ¢ An examination of carceral provider qualifications and standards as well as the experiences of carceral and community providers, including challenges encoun- tered as they develop relationships and coordinate transition of individuals into the community ¢ A comprehensive cost analysis, including covering associated services and budgetary effects ¢ A reinvestment plan that details the amount of reinvestment that will be required for all new federal dollars that cover services that are the responsibility of and were previously provided or paid for by the correctional facility. Additionally, the state must have an independent entity conduct a mid-point assessment of the reentry demonstra- tion initiative to be completed by the end of the third year of implementation. In Jan. 2023, the California Department of Health Care Services (DHCS) shared the state's initial thinking about the evaluation approach for its reentry initiative in response to SHADAC's interview questions. Many of the concepts the state is interested in align with the CMS approval letter. In addition to the hypotheses and measures outlined in the Opportunities for Developing Strong Evaluation Plans for Justice-Involved Initiatives The evaluation of Section 1115 demonstration projects provides an invaluable opportunity to learn not only how Medicaid policy can improve health outcomes and reduce costs but also how it can be used to improve equity, especially for the historically marginalized groups that are dispropor- tionately represented in the justice-involved population.' The following section captures themes identified in the litera- ture and heard throughout discussions with key stakeholders CMS approval letter, the state of California indicated it is also interested in tracking: ¢ Time from incarceration to onset of pre-release ser- vices ¢ Continuity of access to chronic disease medication ¢ Duration of Medi-Cal eligibility and enrollment for the eligible justice-involved population in the months following release e "Warm hand-offs,"i.e., services provided to individual during the pre-release period with services also pro- vided post-release by same provider (inclusive of care manager, behavioral health providers, and physical health providers) ¢ Continuity of providers and handoffs for patients who experience repeat encounters with the justice system e Other measures that may reflect critical non-health aspects of wellbeing that are intertwined with post- release challenges including recidivism, housing sta- tus, and employment status DHCS indicated they are also interested in collecting quali- tative program implementation data. Potential qualitative data sources the state is considering include interviews and focus groups with: e Correctional facility staff responsible for program administration ¢ Correctional facility staff responsible for screening individuals for pre-release services ¢ Correctional facility health care providers ¢ Pre-release care management providers ¢ Enhanced Care Management (ECM) providers that serve the individuals transitioning from incarceration regarding opportunities to learn more about justice-involved populations served through Section 1115 demonstrations. The value of self-reported and qualitative data to the success of this type of evaluation cannot be overemphasized. The lack of qualitative data representing people with lived experience is one of the largest existing gaps in 1115 demonstration evaluation approaches and findings. In California's 1115 demonstration approval letter, for example, CMS "strongly encourages" the state to evaluate the implementation of the reentry initiative, and it "underscores the importance" of undertaking a beneficiary survey and/or interviews but does not appear to require either. Opportunities to Learn More About Serving Justice-Involved Individuals through 1115 Demonstration Evaluations Given the complex dynamics, unique obstacles, and varied experiences justice-involved individuals face, a new, more equity-focused evaluation approach is needed. Researchers must be prepared to spend additional time and effort to evaluate these types of programs effective- ly.2 With those principles in mind, stakeholders identified five key recommendations for designing a robust and account- able evaluation of a justice-involved initiative: 1. Center the voices of people with lived experience in the evaluation approach 2, Collect robust monitoring data in order to build evi- dence for a topic where existing knowledge is limited 3. Consider additional meaningful outcome measures outside typical health outcomes 4. Seek to understand the unique experiences of differ- ent populations in the data analysis plan in order to address equity 5. Prioritize cross-sector data linkage activities and start planning early Center the voices of people with lived experience in the evaluation approach Recent CMS guidance pushes for more "rigorous" evaluation approaches, which generally means employing experimen- tal methods such as cost/benefit analyses or randomized controlled trials. But fundamental to rigor when conducting evaluations in complex settings is understanding the per- spective of people with lived experience.? This is especially significant in the context of trying to improve health equity. People with lived experience are those who have a "personal and unique perspective" that reflects every aspect of a person's life and identity.?? In this context, that could mean individuals who are or have been incarcerated themselves or possibly families of those who have been. Perhaps more than any other historically marginalized population in Medicaid, justice-involved individuals have a unique experience that most state staff and/or researchers designing evaluation plans cannot assume to know. Most people cannot extrapo- late from their own everyday experiences what it is like to be incarcerated. There are two related, yet distinct ways to involve people with lived experience in an evaluation. The first is to seek direct, qualitative feedback from program participants who were incarcerated through surveys, focus groups, interviews, or storytelling in order to better understand their experience in the program. This type of primary data is incredibly valuable as it can help reveal how and why an intervention may or may not work and how to improve it from the perspectives of those most affected.? In addition, this type of qualitative data can help elucidate structural inequi- ties that impact the policies and programs being studied.? A second and perhaps even more meaningful approach is to involve people with lived experience as part of the research team - not just as research subjects. This is often called a participatory, collaborative, or empowerment eval- uation, and can benefit both the research project itself and the participating justice-involved individuals by shifting the traditional power dynamic and making the research more accountable to the affected communities.? (See Evaluation Perspective from an Individual with Lived Experience.) "Direct engagement is important for any disad- vantaged population, but what''s different, though, about people in the justice system is none of us on the outside have any idea what they experience when they're inside. It's totally walled off for us. As an evaluator writing survey questions, | can extrapolate about the experience of going to a primary care doc- tor because I go to a primary care doctor. | have some basis of understanding. But | don't know anything about what it feels like on the inside." - Stakeholder There are several ways to include people with lived experi- ence in a participatory research approach, including: hiring co-researchers who have spent time in jail or prison; requir- ing evaluation contractors to have team members who have experience working in jail or prison settings; or by estab- lishing an advisory group of people with lived experience or their families who can provide bi-directional feedback on evaluation design and implementation tasks.? In any of those scenarios, people with lived experience should be empowered to provide input on the full range of the eval- uation cycle activities, such as: ¢ Determining the evaluation questions and methods. Individuals with lived experience can help the evaluation team identify the research questions that matter most to justice-involved populations, thereby producing evalua- tion findings that are more relevant and meaningful and provide deeper insights into topics. * Designing data collection (e.g., survey or interview) question focus and wording. People with lived expe- rience can help ensure that the topics and questions in data collection tools are rooted in racial equity and do not perpetuate harmful narratives or beliefs. Their input on framing and wording of interview questions can also help create more reliable and culturally responsive data. Opportunities to Learn More About Serving Justice-Involved Individuals through 1115 Demonstration Evaluations ¢ Making sense of any findings and identifying themes and gaps. The first-hand knowledge of people with lived experience can help inform and enhance the analysis of data by identifying themes that might have otherwise been missed, providing a validity check on conclusions that are drawn or bringing in lived experiences to correct misinterpretations. » Sharing the findings. People with lived experience can help disseminate data and findings to new audiences who differ from those who might typically read techni- cal evaluation reports. They can also serve as ambassa- dors to communities with whom academic researchers might not have established relationships or trust, thus increasing credibility of the results and the likelihood of acceptance. Both approaches (seeking direct feedback from justice-in- volved individuals and employing a participatory research approach) may take more time and resources than a con- ventional evaluation.?® Designing and implementing new primary data collection activities, such as interviews or surveys of justice-involved individuals, can be resource-in- tensive and require advance planning and skill. And participatory research approaches require a significant time investment to both train potential co-researchers who might never have taken part in research before and to build a mean- ingful foundation of trust among the members of the team.?® One common recommendation is to involve two or more people with lived experience to participate as co-research- ers. This can help address issues of power imbalance that could exist between conventional academic researchers and lay researchers with lived experience.® Also, to ensure these partnerships benefit everyone involved, researchers should provide equitable compensation to co-researchers or their families.>' In addition, research with justice-involved individuals presents its own unique ethical, privacy, and legal challenges that must be addressed.*? In light of the history of exploitive use of prisoners in scientific experiments, researchers must follow federal guidance for research on vulnerable subjects and address issues of consent, coercion, and fear of retri- bution that could arise for incarcerated individuals.3® As the National Institutes of Health explains, "Because prisoners may not be free to make a truly voluntary and uncoerced decision whether or not to participate as subjects in research, the reg- ulations require additional safeguards for the protection of prisoners in research.>* Evaluation Perspective from an Individual with Lived Experience Pedro, an individual who has been involved in the evaluation of justice-involved initiatives, and who has experience of being incarcerated, spoke to the impor- tance of understanding the culture of prison and the impact that culture can have on the intervention and outcomes being evaluated. Recognizing insights such as these would help an evaluator interpret the success of any reentry program or policy they are studying: e Thereality is that every prison has its own culture. There are hierarchies that will impact whether the inmates in that setting will even agree to take a medication or see a health care provider depending on which gang they are affiliated with, what racial group they are part of, or what part of the state they come from. e You're talking about a closed facility. Every time you walk onto a yard - everything that happens on that yard, everyone is aware of because of safety concerns. Everyone monitors everyone else's movements. If someone leaves the yard, it's "ok, where is he going? Is he going to the medical office? Is that a sign of weakness for that group or that population in the hierarchy?" e Assoon as you see a mental health provider and start taking medication, there are things that change in the prison yard. If you're taking psych meds - when it becomes 90 degrees or hotter - anyone that is on mental health medication has to leave the yard. An announcement comes over the speaker, and those people have to leave. So a lot of times people don't even want to take medication just so they don't have to be outed in the yard. e Anything coming from the criminal justice administration is often viewed suspiciously. If evaluators are coming into a prison setting, then you have to communicate that there is a clear separation between you, as the evaluator, and the administration, otherwise no one is going to answer your survey. Opportunities to Learn More About Serving Justice-Involved Individuals through 1115 Demonstration Evaluations Collect robust monitoring data in order to build evidence for a topic where existing knowledge is limited The process measures typically collected through monitoring reports are a particularly valuable data source when evalu- ating a novel policy approach that involves major systems change, such as using Medicaid funds to pay for health care services in a carceral setting. Because this kind of program is new, and relatively little is currently known by those outside of corrections about the existing provision of health care behind bars, Medicaid evaluators need to establish a funda- mental knowledge base about what is happening. In addition, the unique context in which each site across a state will carry out a justice-involved reentry initiative (e.g., jail vs. prison setting, rural vs. urban, diverse language needs of the population, capacity of community health providers, etc.) means that implementing the policy in a uniform way is nearly impossible. Therefore, the collection of key moni- toring measures will be vital in explaining what was actually implemented at each site, as well as understanding the context in which eventual evaluation outcomes are (or are not) achieved. State-specific evaluation deliverables such as rapid-cycle evaluation reports or policy-focused briefs using monitor- ing data can also be useful to identify and address any early warning signs that the program is not reaching the popula- tion of focus or if there are other barriers to implementation.® Per its new guidance, CMS will require states with reentry demonstrations to collect and report specific metrics (see Appendix A). However, states can also propose additional monitoring metrics to CMS depending on the type of program they are implementing. This presents an opportu- nity for states to align their monitoring data collection and reporting with the goals of their evaluation and to better understand the complex systems that are impacting their program outcomes. In addition to typical monitoring data such as how many individuals were served (and their charac- teristics) and services provided, additional justice-involved monitoring data that could be useful include: + How the money flows. In addition to how dollars/ resources are spent, how is funding distributed and how are systems organized to make decisions? Who is direct- ing funding decisions? Sheriffs? The state? Is it different at jails vs. prisons? » Who is providing services. In addition to what services were provided, who is providing those services? Were they provided by jail or prison health care workers? Private contractors? Community-based providers? Com- munity health workers? Is the service mandated (i.e., are incarcerated individuals required to meet with a care manager or do they have a choice)? What qualifications do the service providers have (e.g., are the providers cul- turally competent, have received training?) Who certifies or credentials the providers? How much staff time is ded- icated to providing services? Infrastructure capabilities (e.g., data exchange and billing capabilities). Are correctional facilities able to bill and claim for health care services? If so, how long did it take to get those systems up and running? Are correc- tional medical records digitized? Is there a data exchange system that facilitates transfer of medical records from jail to community providers, or enroliment information to Medicaid managed care plans? If so, what types of infor- mation was shared during handoffs with which partners and when? Community capacity/availability of health services. How long does it take justice-involved individuals to schedule an appointment with a community provider upon release from incarceration? Does the provider have evening or weekend hours? Is there transportation available to get to the provider? Does the provider have training and/or experience in serving people at reentry? "If we start providing more community-based care in jails and prisons, who is going to do that? We don't have thousands of health care providers sitting around saying I've got nothing to do. And in the community, we don't have some secret supply of mental health and substance use providers that are not taking patients. There's a finite amount of resources and that's going to impact success."- Stakeholder Structural and social context. What are the char- acteristics of the communities that justice-involved individuals return to? What is the residential address ZIP code of returning individuals? Can participants access housing? Employment? Are community-based provid- ers connected to parole/probation offices or are they free-standing/separate? Other contextual factors. What other interventions, in or outside Medicaid, are taking place for the same population? Are there other state-funded Health Home initiatives or managed care requirements providing services to the same population? Opportunities to Learn More About Serving Justice-Involved Individuals through 1115 Demonstration Evaluations Consider meaningful outcome measures beyond typical health outcomes Historically, 1115 demonstration evaluation plans have focused on individual-level health care utilization metrics such as ED use or hospitalizations (and these are reflected in new reentry demonstration evaluation guidance, as well). While those measures are useful, there is also an opportunity to define additional measures of success, both for individuals and the wider community. New kinds of meaningful outcome measures can help create broader arguments in support of the policy being tested. "It's really hard to move some of those individual 'flat' outcome metrics of success like recidivism or health care use. While important, those measures don't tell the full story about the implications of getting Medicaid behind bars for the first time in our history. There's so much more to be learned and gained from getting people access to trustworthy health care that isn't about how many times they get to the primary care office or whether they get picked up by a police officer." - Stakeholder States and evaluators should carefully consider including additional measures that are unique, specific, and meaning- ful to justice-involved populations.3 Quality over quantity is best so as not to overburden program staff or participants. As previously mentioned, people with lived experience can help identify measures that resonate with the population being studied. A data and measures scan can be useful to see what measures are already being collected and to identify priorities for where to invest in new data collection. Measure concepts (short- and longer-term) that are especially significant to understanding experiences of this population include: + Patient-centered engagement. Patient-centered en- gagement can help build trust with the health care system, an element that is largely missing for people who have experienced incarceration. Measuring the extent to which services provided while an individual was incar- cerated were patient-centered - and how engaged the patient was with that process - could ultimately be a critical indicator of the success of a program. « Continuity of care. Treatment continuity is a docu- mented goal of many of the justice-involved initiatives being proposed in 1115 demonstrations. It will be critical for evaluations to explicitly address whether and how care is continued once the individual is released (i.e.,, do individuals see the same provider while incarcerated as when they are released) and on what timeframe. « Self-reported overall well-being. Health is a state of complete physical, mental, and social well-being and not merely the absence of disease.?" Rather than only collect- ing direct measures of health (i.e., diabetes or depression outcomes), evaluations should consider including more holistic and positively framed measures of health such as self-reported well-being. This kind of measure is valuable as it allows individuals to reflect on the multidimensional factors (health, social relationships, living conditions, employment, etc.) that impact whether they perceive that their lives are going well. Research shows that measures of well-being can be effectively used to measure success following prison release.3® o Community-level outcomes. Efforts to strengthen coverage and continuity of care for justice-involved individuals can have benefits that extend beyond the individual to the greater community.*® Studies show that individuals returning from jails and prisons are not evenly distributed geographically and instead are concentrated in a relatively small number of communities.* Therefore, an examination of the impact on families, communities, and systems-level outcomes (e.g., improved public safety, social connectedness, social cohesion, or community engagement) could help tell a compelling broader story about the impact of the policy change being studied.*' In all cases, it will be important to make sure community- or systems-level measures align with the program's theory of change and evaluation questions. » Unintended consequences. Evaluations and logic models often only focus on intended (i.e, positive) outcomes, but reporting and sharing unintended results can be equally significant. For example, one poten- tial unintended consequence of providing Medicaid coverage 90 days prior to release could be that needed health care is delayed in correctional settings until Medicaid coverage begins. Understanding these types of unintended consequences is especially helpful when evaluating multidimensional, cross-sector policies in order to help identify challenges and possible solutions to complex policy problems* Involving stakeholders and being adaptive to learnings from ongoing monitor- ing data can be helpful strategies for identifying those unintended consequences. Open-ended questions in interviews can help ensure that data collection processes remain open to unintended results. » Self-reported measures of criminal activity. Most eval- uations use recidivism as the outcome measure to assess the impact of a policy change on public safety.? Although recidivism is a fundamental concept in criminal justice, and widely used as an outcome measure, there is growing § Recidivism refers to a person's relapse into criminal behavior, however there is no standard definition of the term and no national rates. Recidivism is defined and counted differently (e.g., re-contact, re-arrest, re-incarceration, or reconviction) in different locations and across different states. Opportunities to Learn More About Serving Justice-Involved Individuals through 1115 Demonstration Evaluations recognition that it can reflect racial bias underlying the justice system. Recidivism does not, for example, account for differences in social context such as the fact that Black individuals and Black neighborhoods are disproportion- ately targeted for police contact. In contrast, measuring self-reported criminal activity (e.g., whether the indi- vidual engaged in specific crime types over a specific follow-up period) can provide information on individu- als' actual behavior in a manner that is not influenced by criminal justice system surveillance.* Research has found this type of self-reported data to be of value as it can reveal details not otherwise available about post-release behaviors, including the context or risk setting.* Seek to understand the unique experiences of different populations in the data analysis plan in order to address equity The justice-involved population is not homogenous. The structural racism, discrimination, and levels of trauma that many racial, ethnic, and sexual minority groups (among others) face cause these groups to experience incarceration (and interventions like Medicaid behind bars) in vastly differ- ent ways."" Examining the outcomes of all justice-involved groups as one uniform category can mask the unique experi- ences within certain smaller subgroups, such as transgender people, women, or people with disabilities. Better disaggregated data is critical to understanding the disparate impacts of a policy on these different subpopula- tions, and for informing efforts to improve those inequities. One of the evaluation's primary goals should be to collect, analyze, and report demographic data by as many sub- populations as possible so disparities can be elucidated and paired with tangible efforts to improve them. * CMS' new reentry evaluation guidance also calls on states, to the extent feasible, to collect data to support analysis by sex, age, race/ethnicity, primary language, disability status, geography, and sexual orientation and gender identity. To accomplish this, an essential first task will be to proactively assess existing demographic data, identify secondary data gaps, and develop a plan to address those gaps." There are many strategies to improve the quality of data collection and analysis in relation to health inequities, including stan- dardizing and improving data collection tools, providing training to program staff as to the importance of collect- ing demographic data, and oversampling and aggregating data over multiple years in order to build larger samples (and overcoming the challenges of small sample size).*® Too many populations, such as American Indians and Alaska Natives, have effectively been erased from data due to concerns about small sample size. The Urban Indian Health Institute urges researchers to avoid using concerns over small sample size as a justification for excluding analysis by sub- populations.® Supplementing the analysis of demographic data with qualitative data from various subgroups of interest can be an effective way to foster understanding about quan- titative results that appear statistically insignificant due to small numbers.5® Reports and analyses should be transparent about any data limitations so those can be accounted for in evaluating the results, but also so that future endeavors can address and improve upon these limitations.®' California's 1115 Equity Approach for the Justice-Involved DHCS says it aims to incorporate an equity founda- tion throughout the process of initiative design and implementation in addition to later stages of outcome evaluations. "We will accomplish this by frequent assessments of potential risks of unintended bias due to policy or program elements, program monitoring, and soliciting continual feedback from community-based organizations, advisory groups, and individuals with lived experience." At minimum, DHCS plans to monitor the following with respect to the evaluation process: e Process and outcome measures stratified by race, ethnicity, preferred language, sexual orientation (if available), gender identity (if available), age, and geography. ¢ Network adequacy of care management and penetration rates of other services across geography and demographics (e.g., urban/rural, racial/ethnic make-up, structural socioeconomic disparities such as Area Deprivation Index). ¢ Challenges encountered by service providers in ensuring appropriate language access. ¢ Feedback on the program from individuals with lived experience California indicated that the state is interested in eval- uating the impacts of pre-release services for engaged enrollees (as compared to enrollees who did not engage in pre-release services) but is also interested in other potential comparison groups, including the possibility of case-matched controls or a pre/post analysis. Opportunities to Learn More About Serving Justice-Involved Individuals through 1115 Demonstration Evaluations Prioritize cross-sector data linkage activities and start planning early The ability to link data between the criminal justice system and other agencies such as health care and social services will be key to understanding community-level outcomes and the breadth of impacts (including cost savings to other areas) that result from providing Medicaid services pre-re- lease. Linking data and sharing information between criminal justice and health systems has historically been difficult due to a multitude of technological and privacy considerations.> Although efforts to link criminal justice data can be challeng- ing, they can be successful - especially if enough lead-up time is allowed, the goals of data sharing are clearly articulated, and there is leadership buy-in from both agencies.* For example, to understand how expanded access to SUD treatment provided under a 1115 demonstration affected Los Angeles county patients' use of services, UCLAs Inte- grated Substance Abuse Program undertook a three-year project to link data across multiple county agencies (the agencies already had an existing master data agreement in Spotlight on Arizona Arizona's justice-involved 1115 demonstration initiative, Targeted Investments (TI) Program 1.0, gives financial incentives to health care clinics that are co-located with, or adjacent to, probation and/or parole facilities. These co-located clinics must meet certain benchmarks for integrating and coordinating physical and behavioral health care for Medicaid beneficiaries transitioning from the criminal justice system. To evaluate Tl-participating partners' performance on quality measures, the Arizona Health Care Cost Containment System (AHCCCS) partnered with Arizona State University to create personalized data place). A successful data harmonization and linkage process matched SUD patients' data within five county departments: housing, social services, health services, behavioral health, and criminal justice and found that service utilization in other county departments increased (as did preliminary cost esti- mates) and that arrests/incarceration costs decreased.> The relatively short implementation timelines of 1115 demonstrations means that states interested in pursuing data linkages need to start planning for this work as soon as possible. States initiating this work for the first time will need to dedicate sufficient time to secure buy-in for new data sharing arrangements at the leadership level in both Medicaid and criminal justice agencies (and any other depart- ments). It is equally important to engage the technical staff who will operationalize the data sharing once it is in place, as well as legal staff at both agencies, early in the process. Several stakeholders also noted the benefit of collaborating with academic researchers who are skilled at analyzing large amounts of claims and/or encounter data to facilitate the linkage and analysis of data for use in a meaningful way (See Spotlight on Arizona). dashboards that are updated monthly and populated with claims and encounter data for justice-involved individuals served by that clinic (see figure below). These dashboards allow program participants to monitor progress toward their own goal and progress relative to other Tl justice organizations. Arizona stakeholders said this type of almost real-time (there is a 4-5 month data lag) data visu- alization was one of the most impactful tools they had to understand the impact of their work, to know which jus- tice-involved individuals attributed to them, and to know where and when to make adjustments. Targeted Investments Program Quality Improvement Collaborative (TIPQIC) Dashboard Performance on Measure (each month is a 12-month report period) Provider X vs. Providers in same Area of Concentration 90.00% 86.3% 85.00% 83.3% 82.9% 80.8% 81.3% 20.0% 80.00% 80.4% 80.8% 80.0% 75.00% 70.00% Year 5 Target: 68.0% Year 6 Target: 67.0% 79.7% 79.8% 87.0% 85.0% 85.3% 82.1% 78.3% e 78.7% gosl% 77.7% 76.8% 65.00% 07-2021 08-2021 09-2021 10-2021 Source: Image shared with SHADAC by AHCCCS, November 2022 11-2021 12-2021 01-2022 02-2022 03-2022 04-2022 05-2022 06-2022 Opportunities to Learn More About Serving Justice-Involved Individuals through 1115 Demonstration Evaluations In addition to being useful for understanding cross- sector outcomes, effective data linkages are also necessary to fully understand cost impacts. To date, very few evalu- ations have been able to capture cost savings of providing reentry services for justice-involved populations outside of Medicaid. An evaluation of Transitions Clinic Network (TCN), a model of integrated care designed to serve people return- ing to the community from incarceration, is one of a limited number of evaluations that was able to link Medicaid claims and criminal justice data to capture cost savings across systems. Researchers in this study estimated that every dollar invested in the TCN program yielded a 12-month return of $2.55 to the state of Connecticut through reduced probation costs.> Documenting these types of costimpacts will be espe- cially compelling in building an evidence base in support of policies like providing Medicaid services pre-release. As the TCN researchers noted, "States should invest in data linkage systems that facilitate cost analysis across even more systems (i.e. federal criminal justice system, social service agencies) to allow for quantifying benefits of intervention programs from a larger societal perspective. This would enable studying the collateral benefits (i.e. employment, food access) and changes to both individual and family well-being." The flexibility of 1115 demonstrations gives states an ideal opportunity to invest in the data infrastructure, data collec- tion, and communications systems needed to achieve the types of cross-sector data linkages that have been long- sought. California, for example, recently received approval under their 1115 demonstration project for a $1.85 billion ini- tiative to build the capacity and infrastructure of community partners, including data exchange capabilities.® Prioritizing these types of data linkages is essential to understanding any long-term cost and health outcomes and for building an evidence base for further adoption of these policies. Conclusions Very little is known about how justice-involved populations are currently served under existing Medicaid 1115 demon- strations. Evaluations of California's recently approved, and other states' upcoming, reentry waivers provide a unique opportunity to document and learn from the experiences and outcomes associated with justice-involved populations as states operationalize novel policies in Medicaid. As states like California begin providing Medicaid services while indi- viduals are still incarcerated, it will be important to capitalize on opportunities to demonstrate success that can be used to inform other state policy makers and stakeholders who may be interested in pursuing similar programs for justice-in- volved populations. Evaluations conducted under these demonstrations should be held accountable to the communities being served, not just the federal or state government. In order to address the equity concerns of these long-marginalized communities, evaluations should seek to: involve individuals with lived experience in all facets of the evaluation cycle; collect robust monitoring data that will explain what was implemented; report outcome measures that are meaningful to the jus- tice-involved population; document and explain the unique experiences of different subpopulations in the data analysis plan; and prioritize cross-sector data linkage activities in order to tell a broad story about the impacts of the policy. About the Author Emily Zylla, MPH, is a senior research fellow at the State Health Access Data Assistance Center (SHADAC), where she conducts qualitative research and evaluation on a range of projects aimed at helping states use data to inform policy. This work is supported by the California Health Care Foundation, based in Oakland, California. California Health Care Foundation HEALTH CARE THAT WORKS FOR ALL CALIFORNIANS &\ Opportunities to Learn More About Serving Justice-Involved Individuals through 1115 Demonstration Evaluations Endnotes ' Medicaid and the Criminal Justice System (PDF), Medicaid and CHIP Payment and Access Commission (MACPAC), July 2019, 2 Tyler N. A. Winkelman et al., "Health Insurance Trends and Access to Behavioral Healthcare Among Justice-Involved Individuals-United States, 2008-2014," Journal of General Internal Medicine 31, no. 12 (Sep. 2016): 1523-1529. 3 Erin Bagalman et al., HHS Roadmap for Behavioral Health Integration, Office of the Assistant Secretary for Planning and Evaluation (ASPE), U.S. Department of Health and Human Services (HHS), September 2022. 4 CMS Approval Letter: CalAIM Demonstration (PDF), Centers for Medicare & Medicaid Services (CMS), January 2023. 5 Ebony N. Russ et al., "Prison And Jail Reentry And Health," Health Affairs (Oct. 2021). 5 Elizabeth Ann Carson, Prisoners in 2020 - Statistical Tables (PDF), Bureau of Justice Statistics (BJS), U.S. Department of Justice (DOJ), December 2021; and Health Equity: Mass Incarceration and Structural Racism (PDF), National Association of County & City Health Officials (NACCHO), July 2019. 7 Jhamirah Howard et al., The Importance of Medicaid Coverage for Criminal Justice Involved Individuals Reentering Their Communities, ASPE, April 201; and Ingrid A. Binswanger et al., "Release from Prison - A High Risk of Death for Former Inmates," New England Journal of Medicine 356, no. 2 (Jan. 2007): 157-65. 8 Medicaid and the Criminal Justice System, MACPAC. ° Medicaid and the Criminal Justice System, MACPAC. Reentry Stakeholder Group, ASPE, January 2023; and Patricia Boozang, Anne Karl, and Adam Striar, Recent Updates to Section 1115 Waiver Budget Neutrality Policy: Overview and Implications for States, State Health & Value Strategies (SHVS), December 2022. " Lynn Blewett and Emily Zylla, Targeting Justice-Involved Populations through 1115 Medicaid Waiver Initiatives: Implementation experiences of three states, State Health Access Data Assistance Center (SHADAC), December 2019. 12 Section 1115 Demonstrations: Developing the Evaluation Design, CMS, January 2020. '3 Opportunities to Test Transition-Related Strategies to Support Community Reentry and Improve Care Transitions for individuals Who Are Incarcerated (PDF), CMS, April 2023. '4 Elizabeth Lukanen et al,, Understanding the effects of Medicaid innovation: Considerations for evaluating state Medicaid policies, National Governors Associa- tion Center for Best Practices (NGA Center), November 2020. '8 Nadereh Pourat et al,, Final Evaluation of California's Whole Person Care (WPC) Program, UCLA Center for Health Policy Research, February 2023, '¢ Shauna Petchel et al. Medicaid Transformation Project Evaluation Interim Report, Center for Health Systems Effectiveness, December 13, 2020. 17 Arizona Section 1115 Waiver Evaluation Interim Report, Health Services Advisory Group, August 2021. '8 CMS Approval Letter: lllinois Behavioral Health Transformation Demonstration (PDF), CMS, April 2021. ' Juliet Bui et al, "Understanding and Addressing Health Disparities and Health Needs of Justice-Involved Populations," Public Health Reports 134, no. 1 suppl {May 2019): 35-75. 2 Mark E Johnson et al., "Involving incarcerated individuals in the research process: Perspectives of key stakeholders," Action Research 16, no. 1 (Aug. 2016): 43-64. 2! Hallie Preskill and Jewlya Lynn, Rethinking Rigor: Increasing Credibility and Use, FSG, January 2016. 2 Given, Lisa M., ed. The SAGE Encyclopedia of Qualitative Research Methods. Thousand Oaks, CA: SAGE Publications, Inc., 2008. 2 Rachel Swaner and Monica Sheppard, Participatory Research: What Is It and How Can It Strengthen Your Reentry Program Evaluation?, DOJ, Bureau of Justice Assistance (BJA), April 2022, 2 Sacha Kendall et al., "Systematic review of qualitative evaluations of reentry programs addressing problematic drug use and mental health disorders amongst people transitioning from prison to communities," Health & Justice 6, no. 4 (Mar. 2018): 4. #Tiffany N. Ford and Annelies Goger, The value of qualitative data for advancing equity in policy, Brookings, October 2021. % Swaner and Sheppard, Participatory Research. 7 Lauren Farrell, Bethany Young, and Janeen Buck Willison, Participatory Research in Prisons, Urban Institute, April 2021. 28 Mark E Johnson et al., "Involving incarcerated individuals." Opportunities to Learn More About Serving Justice-Involved Individuals through 1115 Demonstration Evaluations 2 Cristina Sette, Participatory Evaluation, BetterEvaluation, November 2021; and Rachel Swaner et al., "How to Use Participatory Research in Your Reentry Program Evaluation (and Why You Might Want To)," DOJ, BJA, April 2022. 30 Lauren Farrell et al. Participatory Research in Prisons. 31 Amy B. Smoyer, Kim M. Blankenship and Bradis Belt, Compensation for Incarcerated Research Participants: Diverse State Policies Suggest a New Research Agenda. American Journal of Public Health. 2009 October; 99(10): 1746-1752. 32 Fthical Considerations for Research Involving Prisoners. Institute of Medicine, June 2006. 33 Lauren Farrell et al. Participatory Research in Prisons. 3#Vulnerable and Other Populations Requiring Additional Protections. National Institutes of Health 35 Lama Hassoun Ayoub and Christine Lindquist, Using Evaluation Results to Improve Service Delivery in Reentry Programs, DOJ, BJA, November 2021. 36 Redesigning Reentry: How Medicaid Can Improve Health and Safety by Smoothing Transitions from Incarceration to Community, Health and Reentry Project, July 2022, % Constitution of the World Health Organization, World Health Organization (WHO), July 1946. 38 The Limits of Recidivism: Measuring Success After Prison, (Washington, DC: The National Academies Press), 69-138. 3 Tasseli McKay et al., Health Coverage and Care for Reentering Men: What Difference Can It Make? (PDF), ASPE and the Office of Planning, Research & Evaluation of the Administration for Children & Families (OPRE/ACF), HHS, January 2016. 0 James P. Lynch and William J. Sabol. Prisoner Reentry in Perspective, Urban Institute, September 2001. # Suzanne Boswell. They Can't Quit Recidivism, Center for Court Innovation, September 2022, "2 Kathryn Oliver et al., "Evaluating unintended consequences: New insights into solving practical, ethical and political challenges of evaluation,' Evaluation 26, no. 1 (Jun. 2019): 61-75. 4 Christine Lindquist, Racial Equity Considerations When Using Recidivism as a Core Outcome in Reentry Program Evaluations, DOJ, BJA, April 2021. 4 The Limits of Recidivism, 69. *Valeria Bacak et al., ""Incarceration as a Health Determinant for Sexual Orientation and Gender Minority Persons," American Journal of Public Health 108, no. 8 (Aug. 2018): 994-999, *6 | everaging Section 1115 Demonstrations to Drive Equity in Medicaid, America's Essential Hospitals, April 2022, 7 Patricia Boozang et al., Centering Health Equity in Medicaid Section 1115 Demonstrations, SHVS, February 2022, 8 Leiha Edmonds, Clair Minson, and Ananya Hariharan, Centering Racial Equity in Measurement and Evaluation (PDF), Urban Institute, July 2021. * Joseph Friedman et al., "Deaths of despair and Indigenous data genocide," The Lancet S0140-6736, no. 22 (Jan. 2023): 02404-7. % Ford and Goger, The value of qualitative data. ' Data Genocide of American indians and Alaska Natives COVID-19 Report, Urban Indian Health Institute, February 2021. 52 Jane B. Wishner and Kamala Mallik-Kane, Information Sharing between Medicaid and Corrections Systems to Enroll the Justice-Involved Population: Arizona and Washington, Urban Institute, January 2017. 5 Kamala Mallik-Kane et al., Measuring Progress in Connecting Criminal Justice to Health: A How-to Guide to Performance Management for Practitioners (PDF), Urban Institute, March 2018. 5*Vandana Joshi and Tina Kim, "Substance Use Disorder Patients' Program Utilization from Other County Departments in Los Angeles County" (2022 American Public Health Association Annual Meeting, Boston Massachusetts). % Tyler D. Harvey et al. "Cost Savings of a primary care program for individuals recently released from prison: a propensity-matched study." BMC Health Ser- vices Research 22, 585 (2022). % CalAIM Providing Access and Transforming Health Initiative, California Department of Health Care Services (DHCS), January 31, 2023, State Health Access Data Assistance Center 15 Opportunities to Learn More About Serving Justice-Involved Individuals through 1115 Demonstration Evaluations Appendix A. Summary of Centers for Medicare & Medicaid Services Monitoring and Evaluation Requirements for Reentry Section 1115 Demonstrations Quarterly & Annual Monitoring Plans CMS expects quarterly and annual monitoring reports to include (but not be limited to) such metrics as: e Administration of screenings to identify individuals eligible for pre-release services ¢ Participating pre-release services providers ¢ Utilization of applicable pre-release and post-release services (e.g., primary, behavioral, medications for opioid use disorder, case man- agement) Provision of health or social service referral pre-release Participants with established care plans at release Take-up of data system enhancements among participating carceral settings Quality of care and health outcomes metrics known to be important for closing key quality and health equity gaps in Medicaid/CHIP (e.g., the National Quality Forum (NQF) "disparities-sensitive" measures) e Health equity-focused measures in alignment with CMS' forthcoming Health Equity Measure Slate ¢ Qualitative information including, but not limited to, the state's progress on data development and exchange Mid-point Assessment CMS expects an independent assessor will complete a mid-point assessment between years two and three of the demonstration implementation describing: o The state's progress in meeting milestones and performance measure targets e Any challenges the state is encountering e How the state is planning to overcome those challenges and apply lessons learned Evaluation Design, Interim, and Summative Reports The state is required to submit: an evaluation design within 180 days of the demonstration approval; an interim evaluation report on year before expiration of the demonstration (or when the state submits a proposal to extend the demonstration); and a summative evaluation report within 18 months after the demonstration period ends. The evaluation design and reports are required to include (but are not limited to): A mixed-methods approach How the state will test whether the demonstration improved care transitions for individuals who are released from incarceration How the demonstration improves coverage and quality of care Outcomes including: e Measurement of cross-system communication and collaboration ¢ Connections between carceral settings and community services e Provision of preventive and routine physical and behavioral health care e Avoidable ED visits ® * Inpatient hospitalizations All-cause deaths e A comprehensive cost analysis estimating the cost of implementing the demonstration and the cost of covering associated services Data to support analyses stratified by key subpopulations of interest (e.g., by sex, age, race/ethnicity, primary language, disability status, geography, and sexual orientation and gender identity) CMS also underscores the importance of the state undertaking well-designed provider, carceral facility, and/or beneficiary surveys and/ or interviews to assess key implementation challenges for case managers, providers and carceral facilities and their understanding of beneficiary experience, and beneficiary understanding of and experience with transitioning out of the carceral setting. Source: Opportunities to Test Transition-Related Strategies to Support Community Reentry and Improve Care Transitions for Individuals Who Are Incarcerated. CMS, April 2023. State Health Access Data Assistance Center 16 Opportunities to Learn More About Serving Justice-Involved Individuals through 1115 Demonstration Evaluations Appendix B. Proposed Hypotheses and Measures in Pending Reentry Section 1115 Demonstrations Proposed Hypotheses: Providing in-reach before release will create community linkages and ensure that individuals who are at high risk of experiencing homelessness receive needed coordination of care, physical and behavioral health services, medication and medication management, and critical social support upon release into the community. Proposed Measures: ¢ Not specified in application Kentucky Proposed Hypotheses: With the approval and implementation of this amendment recidivism will be reduced among Medicaid beneficiaries who receive Medicaid matched SUD treatment; SUD treatment will increase the likelihood of enroliment and continuity following release; and health outcomes will improve following release. Proposed Measures: Number and percent of members who receive Medicaid matched treatment with a re-incarceration Number and percent using SUD community-based treatment services before and after this amendment Number and percent of Medicaid beneficiaries with a re-incarceration before and after this amendment Number and percent using SUD community-based treatment services before and after this amendment Massachusetts Proposed Hypotheses: This expenditure authority would: increase engagement in primary and behavioral health care in the community; decrease avoidable hospitalizations and emergency department visits; improve behavioral health outcomes; reduce recidivism; and decrease disparities in health outcomes, as Black and Hispanic individuals are disproportionately represented in the Massachusetts justice- involved population. Proposed Measures: e Avoidable hospitalization and ED visits within first 30 days of release Drug overdoses within first 30 days of release Suicide attempts within first 30 days of release In-office visit with PCP and behavioral health clinician (if warranted) within first 30 days of release Completion of Hepatitis C treatment after release for individuals who initiated Hepatitis C treatment while incarcerated Individuals with substance use disorder maintaining medication-assisted treatment after incarceration Community tenure after incarceration Proposed Hypotheses: Increasing access to community-based treatment and recovery services, including tenancy supports, contingency management, and pre-release care management to be provided to inmates in the 30 days pre-release, will reduce ED utilization and preventable hospital admissions. Proposed Measures: e Number and percentage of Medicaid members with SUD and/or SMI/SED diagnoses with ED visits o Number and percentage of Medicaid members with SUD and/or SMI/SED diagnoses with hospital admissions o Number and percentage of Medicaid members with SUD and/or SMI/SED diagnoses New Hampshire Proposed Hypotheses: Providing a tailored Medicaid benefit 45 days prior to release will increase re-integration rates and reduce recidivism among Medicaid beneficiaries who receive the pre-release transitional services Proposed Measures: e Time in Community: length of time individual remains in the community or percentage of intervention population remaining in the community at 30, 60, 90, 120 days Relative Recidivism: comparison of similarly controlled individuals who opt out of pre-release transitional services vs. those who opt in Parole Violations: reduction in parole violations associated with SUD/relapse or mental health treatment non-adherence Reduce utilization of IMD services by formerly incarcerated members Reduce utilization of emergency department and inpatient hospital settings for SUD and SMI/SED treatment where the utilization is preventable or medically inappropriate, through improved access to other continuum of care services Opportunities to Learn More About Serving Justice-Involved Individuals through 1115 Demonstration Evaluations New Jersey Proposed Hypotheses: The introduction of pre-release services for incarcerated individuals may result in: a lower percentage of formerly incarcerated individuals having an ED visit for mental illness or alcohol or other drug treatment; a high percentage of individuals receiving behavioral health services within 30 days of release; a lower rate of re-engagement in the criminal justice system following release; improved stakeholder-reported assessments of post-incarceration transition to effective health services; and a reduction in racial/ethnic disparities in specific categories of ED visits or access to behavioral health services. Proposed Measures: Percentage of ED visits for mental illness or alcohol/other drug treatment after release Percentage of behavioral health services within 30 days after release Rate of re-engagement in criminal justice system after release Stakeholder-reported assessments of post-incarceration transition to effective health services Racial/ethnic disparities in certain categories of ED visits or access to BH services Proposed Hypotheses: Providing targeted benefits to high-need justice-involved members 30 days prior to release will increase access to and utilization of necessary behavioral and physical health services and medications (including MAT and DME). Proposed Measure: o Members exiting incarceration will have increased utilization of preventative services at 30 days, 90 days, and annually after release in comparison to a similar population in the four years preceding this benefit New York Proposed Hypotheses: The proposed 1115 waiver amendment initiatives will be associated with a decrease in health disparities across the demonstration. Increased utilization of Enhanced Transitional Housing Services throughout the period of the demonstration will advance health equity. Proposed Measures: (Note: these are example measures that have not yet been finalized) ¢ Population - HEDIS Quality Measures e Chronic disease: Hemoglobin A1c control for patients with diabetes - HbA1c poor control (>9.0%) e Behavioral health: follow-up after ED visit for substance use Oregon Proposed Hypotheses: Redistributing power and resources to individuals and communities most harmed by historical and contemporary racism will result in improvements in health inequities and self-reported measures of autonomy, health status and quality of life; Offering packages of SDOH support services to individuals experiencing transitions is more effective at improving integration and stabilization for successful transition than offering health care services alone. Proposed Measures: Self-reported measures of stability and security Reduced recidivism rates Reduced ED visits for Behavioral Health and nontraumatic dental needs Time to first appointment with patient centered primary care home/time to first appointment with behavioral health provider Rhode Island Proposed Hypotheses: Pre-release enroliment will improve health outcomes and access to healthcare for recently incarcerated members. Proposed Measures: Number of previously incarcerated individuals enrolling in Medicaid Number of previously incarcerated individuals accessing primary care services Primary care & preventative services Mental health and SUD/OUD services Inpatient hospitalization and re-hospitalization ED visits and potentially avoidable ED visits Proposed Hypotheses: This demonstration will promote continuity of Medication Assisted Treatment for individuals with an Opioid Use Disorder and Antipsychotic medication for individuals receiving that pharmaceutical treatment. Proposed Measures: e Number of Medication Assisted Treatment (MAT) prescriptions e Number of antipsychotic prescriptions Opportunities to Learn More About Serving Justice-Involved Individuals through 1115 Demonstration Evaluations Vermont Proposed Hypotheses: The demonstration will result in increased access to treatment services for Medicaid-eligible Vermonters who were previously incarcerated and released from DOC facilities. Proposed Measures: e Number or percentage of previously incarcerated Medicaid-eligible individuals utilizing treatment services before and after demonstration renewal Proposed Hypotheses: Re-entry coverage will increase medication adherence for justice-involved individuals enrolled in Apple Health and increase preventive care utilization and reduce ED visits, hospitalizations crisis services, and recidivism. Proposed Measures: o Changes in medication adherence over time based on administrative/claims data e Changes in preventive care, ED utilization, hospitalizations, crisis service utilization, and recidivism over time, based on administrative/ claims data West Virginia Proposed Hypotheses: Ensuring continuity of care for justice involved members will improve health outcomes and decrease recidivism rates upon release from incarceration. Proposed Measures: e Non-emergent ED utilization post-incarceration using Medicaid claims data e Number of individuals reinstated in Medicaid within 30 days of incarceration release using Medicaid claims data Source: SHADAC analysis of states 1115 Reentry Demonstration Applications on Medicaid.gov. State Health Access Data Assistance Center 19