U.S. Department of Health and Human Services Office of Inspector General Incidence of Adverse Events in Indian Health Service Hospitals Christi A. Grimm Principal Deputy Inspector General December 2020, OEI-06-17-00530 U.S. Department of Health and Human Services Office of Inspector General Report in Brief December 2020, OEI-06-17-00530 Why OIG Did This Review Incidence of Adverse Events in Indian Health The Office of Inspector General (OIG) conducted this review to Service Hospitals estimate the incidence of patient What OIG Found harm events in Indian Health Key Takeaway An estimated 13 percent of patients Service (IHS) hospitals and to An estimated 13 percent of experienced patient harm events (i.e., assess the extent to which these patients in IHS hospitals harm to a patient as a result of medical events were preventable. experienced patient harm care). Patient harm events include IHS provides comprehensive events, with higher rates of adverse events and temporary harm Federal health services to harm in smaller hospitals. events that occurred during IHS hospital approximately 2.6 million American Within our sample, harm events stays in FY 2017. Indians and Alaska Natives. In were more prevalent among Patients who experienced patient harm fiscal year (FY) 2017, IHS provided older adults and labor and events fell into two categories: acute-care services at 26 hospitals delivery patients, and about half located predominately in remote of events were preventable—i.e., › 4 percent of patients experienced areas across the country. Most of they could have been avoided if adverse events, i.e., harm events these hospitals are small, with few patients had been given better that resulted in a prolonged hospital inpatient beds. Compared to care. We recommend that IHS stay, permanent patient harm, life acute-care hospitals nationally, IHS establish patient harm sustaining intervention, or hospitals have lower patient monitoring and reduction as a contributed to death; and an volume, provide less clinically key priority of its Office of additional › complex care, and have shorter Quality; effectively track and average lengths of stay. monitor harm events; and 8 percent of patients experienced implement quality improvement temporary harm events, i.e., harm Prior OIG reports have identified plans both across IHS and events that did not result in a longstanding challenges to IHS’s targeted to smaller hospitals prolonged hospital stay or cause ability to deliver safe and and patient groups at greater lasting harm. high-quality health care to risk of harm. American Indians and Alaska (Note: Because of rounding the above Natives—a medically vulnerable rates do not add to 13 percent.) population with poorer health Hospital Location and Size. We did not find higher rates of patient harm outcomes and barriers to accessing events in IHS hospitals in the agency’s Great Plains Area—a location of health care, especially among those concern because of past quality-of-care problems. With regard to hospital aged 65 and older. Challenges size, the rate of patient harm was significantly higher among smaller IHS faced by IHS hospitals include hospitals nationwide: staffing shortages and lack of specialty care. IHS hospitals in the › 19 percent of patients experienced patient harm events in smaller IHS agency’s Great Plains Area are a hospitals (those with fewer than 1,000 admissions in FY 2017), whereas › location of particular concern for these issues. 9 percent of patients experienced patient harm events in larger IHS hospitals (those with 1,000 or more admissions in FY 2017). This report is part of an OIG series of reports about adverse events in Event Type and Clinical Category. Most of the patient harm events we health care settings and continues identified in our sample were temporary harm events, and more than half of OIG’s commitment to monitoring patient harm events were related to the use of medication. Few events in our the quality of care at IHS facilities. sample related to procedures or infections. Report in Brief continued Report No. OEI-06-17-00530 How OIG Did This Review Patient Type. The proportion of patients experiencing harm events varied by We selected a stratified random patient type (e.g., age or reason for admission). In our sample, a higher sample of 400 patients, from proportion of older adults (aged 65 and older) and labor and delivery pediatric patients to older adults, patients (any age) experienced harm events, whereas pediatric patients (up to who were admitted to 1 or more of and including 17 years of age) had the 26 IHS hospitals during fewer harm events. We found the Labor and Delivery Patients FY 2017. The final sample following breakdown by patient type A companion report titled consisted of 384 patients because in our sample: Instances of IHS Labor and › of a small number of ineligible 30 percent of older adults Delivery Care Not Following admissions and missing records. experienced patient harm events, National Clinical Guidelines or Best Practices (OEI-06-19-00190) › We drew patients for our sample from six strata that were based on 21 percent of labor and delivery found that over half of the 48 IHS hospital location and size. This patients experienced patient labor and delivery patients in the sample design enabled us to harm events, and sample experienced care that did ensure that the sample included a range of small hospitals, that it did › 5 percent of pediatric patients not follow national clinical guidelines or best practices. experienced patient harm events. not require a high number of records from large hospitals, and (Note: These proportions are unweighted. About 15 percent of adults aged that we could make projections to 18 to 64 experienced patient harm events, but this did not appear to the Great Plains Area. substantially vary from the overall 13 percent harm rate.) We calculated the incidence rate of Most patient harm events in our sample occurred in smaller hospitals or patient harm events in IHS involved either older adults or labor and delivery patients. At least one of hospitals from a review of patients’ these characteristics was present in 68 of the 79 patient harm events. medical records. Clinicians reviewed each patient’s medical Preventability. An estimated 7 percent of patients overall—slightly over half records to identify patient harm of the 13 percent of patients who experienced harm events—experienced events and to assess the extent to events that could have been prevented if the patients had been given better which these events were care. Preventable events were often related to medical errors and preventable. We conducted the substandard care (i.e., the failure to provide care according to national clinical review in two stages. guidelines). We determined that events were not preventable when other factors were involved in the harm, such as patients’ being highly susceptibility Stage 1: Nurses screened the to harm events because of poor health. records for possible patient harm events using a “trigger tool What OIG Recommends and How the Agency Responded method.” A “trigger” is a clinical In an effort to reduce patient harm, OIG recommends that IHS’s Office of clue—for example, documentation Quality establish patient harm monitoring and reduction as a key priority. of a fall—that may indicate harm. IHS should also effectively track and monitor patient harm events using an improved and fully implemented incident reporting system. We further Stage 2: Physician-reviewers recommend that IHS implement quality improvement plans to improve conducted a full review of the patient safety across IHS, including plans that specifically focus on smaller records flagged by nurses as hospitals and patient groups at higher risk of harm. IHS concurred with our containing possible harm events. recommendations and affirmed that patient safety is a high priority for the Physician-reviewers identified harm agency. Actions reported by IHS included enhanced partnerships, such as events and assessed the level of with the Centers for Medicare & Medicaid Services and the Agency for harm, whether events were Healthcare Research and Quality, further hospital adoption of quality preventable, and factors that improvement and compliance plans, and implementation in 2020 of its new contributed to events. incident reporting system, I-STAR, across all IHS Areas and facilities. TABLE OF CONTENTS BACKGROUND 1 Methodology 9 FINDINGS 15 An estimated 13 percent of patients experienced adverse events or temporary harm events 15 during their stays in IHS hospitals Older adults and labor and delivery patients in our sample experienced higher proportions of 20 harm events; pediatric patients had fewer harm events An estimated 7 percent of patients experienced adverse events or temporary harm events that 23 could have been prevented if patients had been given better care RECOMMENDATIONS 26 Establish patient harm monitoring and reduction as a key priority of the Office of Quality 26 Effectively track and monitor patient harm events using an improved incident reporting 27 system Implement quality improvement plans to improve patient safety across IHS, including plans 27 that focus specifically on smaller hospitals and patient groups at higher risk of harm AGENCY COMMENTS AND OIG RESPONSE 28 APPENDICES 30 A. Methodology for Identifying Events and Determining Preventability 30 B. Glossary of Selected Terms 35 C. Trigger Tool Used To Screen for Patient Harm Events 37 D. Estimates, Confidence Intervals, and Key Statistics 41 E. Sample Numbers by Patient and Event Type per Stratum 44 F. Comparing Rates of Adverse Events and Temporary Harm Events to Other Health Care 48 Settings G. Agency Comments 50 ACKNOWLEDGMENTS AND CONTACT 53 ABOUT THE OFFICE OF INSPECTOR GENERAL 54 ENDNOTES 55 BACKGROUND Objectives 1. To estimate the incidence of adverse events and temporary harm events among patients in Indian Health Service (IHS) hospitals. 2. To assess the extent to which these adverse events and temporary harm events were preventable and to identify factors contributing to these events. Adverse Events and Temporary Harm Events in Health Care The term “adverse event” describes harm to a patient as a result of medical care or in a health care setting, including the failure to provide needed care. An adverse event indicates that care resulted in an undesirable clinical outcome—an outcome not caused by underlying disease—that prolonged the patient stay; caused permanent patient harm; required life-saving intervention; or contributed to death. We also identify “temporary harm events,” which are events that resulted in patient harm and required medical intervention but did not cause lasting harm and are often less severe than adverse events. In this report, we sometimes use the term “patient harm events” to refer collectively to adverse events and temporary harm events. Adverse events and temporary harm events include medical errors and general substandard care that result in patient harm, such as failing to recognize and treat patients’ infections. However, adverse events and temporary harm events do not always involve errors, negligence, or poor quality of care, and as a result they are not always preventable—for example, an allergic reaction might not have been preventable if it was unexpected. 1 The Institute for Healthcare Improvement (IHI), a nonprofit advisory group dedicated to improving health and health care worldwide, further explains that “unpreventable events are only an innovation away from being preventable” and that including all causes of harm in research allows for better comparisons over time. 2 All-cause harm includes “any event during the care process that results in harm to a patient, regardless of the cause.” 3 Prior Studies of Adverse Events Reducing the incidence of adverse events is a critical step to improving patient safety and quality care. In 2000, the Institute of Medicine (IOM) report, To Err Is Human: Building a Safer Health System, focused widespread attention on the problem of adverse events. 4 IOM cited two medical record reviews to identify adverse events and assess whether events were preventable. The studies found that between 2.9 and Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Background | 1 3.7 percent of hospitalized patients experienced these events and caused “at least 44,000 and perhaps as many as 98,000 deaths in hospitals each year.”5, 6, 7 Beginning in 2008, the Office of Inspector General (OIG) has released a series of reports regarding adverse events, including five reports that estimated the incidence rates of adverse events in various health care settings.8 In a 2010 study, OIG found that 27 percent of hospitalized Medicare beneficiaries experienced adverse events or temporary harm events and that about half of these events were preventable.9 Each of the OIG studies used a methodology that included screening records for potential patient harm events using the Global Trigger Tool (GTT), followed by a full physician review to identify events. The GTT is a systematic screening of records to look for “triggers” (clinical clues) that may indicate patient harm.10 In a 2012 follow- up report, OIG found that hospital staff did not identify and report 86 percent of adverse events and temporary harm events to their hospitals’ incident reporting systems.11 Other researchers have used variations of GTT methodologies, finding estimated harm rates ranging from 9 percent to 33 percent (see Exhibit 1). Exhibit 1: Selected Patient Safety Research in Hospitals Using the Global Trigger Tool Study Description Harm Rate Office of Inspector OIG randomly selected 780 Medicare beneficiaries from a national sample of Medicare-certified 27% of patients hospitals in October 2008. OIG identified 302 harm events (174 temporary harm events) using (13.5% of patients— 12 General (2010) an OIG-modified version of the GTT. temporary harm) Landrigan, et al. 13 Researchers randomly selected 2,341 adult patient admissions from 10 hospitals between 2002 18% of patient (2010) and 2007. They identified 588 events (including those present on admission (POA)) across admissions (including 423 patient admissions using the IHI GTT. POA) Classen, et al. 14 Researchers randomly selected 795 adult patient admissions from 3 hospitals during 33% of patient (2011) October 2004. They identified 393 events (354 identified using the IHI GTT). admissions Kirkendall, et al. 15 Researchers randomly selected 240 pediatric admissions from a hospital medical center in 2009. 26% of patients (2012) They identified 88 harm events (74 events were not POA) across 62 patients using the IHI GTT. (including POA) Kennerly, et al. 16 Researchers randomly selected 9,017 adult patient encounters from 8 hospitals from a health 21% of admissions (2014) care system between 2007 and 2011. They identified 3,430 events (2,129 events not POA) using (33% of admissions the IHI GTT. including POA) Adler, et al. 17 Researchers randomly selected 21,007 adult patient records from 24 hospitals in a large 26% of patients (2018) multistate health system from 2009 to 2012. They identified 2,579 patients with adverse events (13.4% of patients— and 2,818 patients with temporary harm events using the IHI GTT. temporary harm) Griffey, et al. 18 Researchers randomly selected 2,594 adult patient records from four emergency departments 9% of ED visits (2018) (EDs) from 2016 to 2017. They identified 240 events (72 events involving patient harm and not (including POA) POA) using an ED Trigger Tool. Stockwell, et al. 19 Researchers randomly selected 3,790 pediatric patient records from 16 hospitals from 2007 to 11% of patients* (2018) 2012. They identified 414 adverse events (210 preventable adverse events) using the Global (including POA) Assessment of Pediatric Patient Safety (GAPPS) Trigger Tool. *The incidence rate of 11 percent was not specifically included in the article. We calculated 11 percent using numbers presented in the article. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Background | 2 This OIG study continues our series on patient harm by estimating the incidence of patient harm events in IHS-operated acute-care hospitals, estimating the preventability of these events, and determining the factors contributing to these events. This is the first in the series to sample records from a full range of patient ages, from newborns to older adults. This study also extends our body of work related to quality of care in IHS hospitals. Two OIG reports issued in October 2016 found that IHS was limited in its ability to provide rigorous quality oversight and described challenges that affected IHS hospitals’ abilities to provide quality care and maintain compliance with Federal regulations. 20 An OIG report issued in July 2019 described challenges that IHS faced during the closure and reopening of one IHS hospital emergency department and continued problems maintaining compliance with Federal regulations. 21 Finally, an OIG report issued in August 2019 found that a lack of organizational structure and clarity in understanding hospital performance hindered IHS’s ability to make improvements. 22 Prior congressional testimony and Government Accountability Office (GAO) reports have also raised concerns about the quality of care provided by IHS facilities. 23 Indian Health Service IHS is responsible for providing Federal health services to American Indians and Alaska Natives (AI/ANs), and in fiscal year (FY) 2020 Congress appropriated $6 billion for IHS. 24 IHS’s mission is “to raise the physical, mental, social, and spiritual health of AI/ANs to the highest level.” 25 In partnership with Tribes, IHS provides health care services to approximately 2.6 million AI/ANs living in the United States who are members of the 574 federally recognized Tribes. 26 IHS manages its health care services through 12 Area Offices, each providing support and oversight for the health care delivery sites located within a specified geographic region. 27 The delivery sites may include hospitals, health stations, and other types of facilities. 28 IHS provides health care services directly to AI/ANs through IHS-operated facilities or provides financial support for the Tribes to operate their own health care systems. 29 In FY 2020, about $2.3 billion of IHS’s Federal appropriations was designated to run hospitals and health clinics. 30 Of that, about 40 percent was allocated directly to federally operated hospitals and clinics, and the remaining 60 percent was allocated to individual Tribes or Tribal organizations. 31 IHS-operated facilities may also receive reimbursement from Medicare, Medicaid, and private insurance for services they provide to AI/ANs enrolled in these programs or health plans. 32 Annually, IHS collects approximately $1.2 billion from these three sources. 33 IHS Hospitals In FY 2017 (October 2016 through September 2017), IHS directly operated 26 acute-care hospitals in 7 IHS Areas, mostly in remote locations. (As of July 2020, Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Background | 3 there were 24 IHS hospitals. 34) Five of the 26 IHS hospitals were designated by Centers for Medicare & Medicaid Services (CMS) as critical access hospitals, which receive cost-based Medicare reimbursement to provide essential services in rural communities. 35 IHS hospitals are typically small, with most having fewer than 30 beds and fewer than 1,000 patient admissions annually. In FY 2017, there were a total of 17,210 inpatient admissions to IHS-operated hospitals, averaging out to 662 admissions per hospital. 36 The total average daily census was approximately 158 inpatients across all IHS hospitals, with an average daily census of approximately 6 inpatients per IHS hospital. Collectively, seven hospitals cared for more than three-quarters of the IHS inpatients. 37 See Exhibit 2 for a breakdown by IHS Area of the number of IHS hospitals, the number of inpatient admissions, and the average daily census. Exhibit 2: Patient Volume in IHS Hospitals by Area in FY 2017 IHS-Operated Admissions Average Daily IHS Area Hospitals (Range) Hospital Census* Albuquerque (NM) 4 5–454 1.2 Bemidji (MN) 2 10–62 1.4 Billings (MT) 3 6–644 2.3 Great Plains (ND, SD, NE) 7 2–1,585 3.5 Navajo (AZ, NM) 4 355–3,154 16.5 Oklahoma (OK) 2 365–1,155 8.5 Phoenix (AZ) 4 59–2,801 8.9 Overall Average -- 662 6.0 Source: OIG analysis of patients admitted to IHS hospitals in FY 2017. *Average daily census does not include newborns. In addition to providing acute-care services, IHS hospitals also admitted or delayed discharge for a small number of patients (203 patients) in FY 2017 for nonmedical or “social” reasons. For example, IHS has allowed patients to stay in the hospital longer than medically necessary because their homes were not suitable for recovery. 38 A prior OIG report found that IHS hospitals experienced challenges with discharge planning; staff cited the lack of post-acute care services (e.g., nursing homes, rehabilitation clinics) and difficult living conditions (e.g., no running water or electricity) experienced at home by some IHS patients. 39 Scope of Services. Many IHS hospitals only offer a limited scope of services to patients, often resulting in patient transfers for specialized or more complex care. For example, some IHS hospitals do not have operating rooms; labor and delivery units; or imaging equipment such as computerized tomography (CT) scanners. In situations in which an IHS hospital is unable to provide a patient with medically necessary care, the patient is either transferred to another IHS facility or referred to a non-IHS Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Background | 4 provider. 40 Currently, IHS has only one Level III trauma center capable of providing trauma care to injured patients and improving survival outcomes. 41 Staffing Shortages. IHS hospitals often have difficulty hiring and retaining qualified medical personnel, particularly in remote areas. 42 The Health Resources & Services Administration (HRSA), an agency tasked with improving health care to the medically vulnerable, has designated all IHS hospitals as facilities in areas with shortages of health care professionals. 43 In 2017, IHS reported agencywide vacancy rates of 29 percent for physicians and 27 percent for nurses. 44 A OIG report found that vacancies had a significant impact on the continuity and quality of care provided at IHS hospitals. 45 Hospital administrators reported that staffing shortages sometimes forced them to turn away patients and resulted in compliance issues with Federal requirements. 46 One IHS hospital closed its emergency department in 2015 after numerous quality-of-care problems, largely because of staffing shortages. 47 In a review of this closure, OIG recommended that IHS leadership intensify efforts to develop and implement a staffing program for recruiting and retaining staff and leadership for these remote hospitals. 48 Population Served. The AI/AN population faces persistent disparities in health outcomes. Because of these disparities, HRSA has designated AI/ANs as a medically underserved population. 49 AI/ANs have a lower average life expectancy than other Americans (5.5 years less) and die at higher rates from many common conditions. AI/ANs are 3.2 times more likely to die from diabetes and 4.6 times more likely to die from chronic liver disease and cirrhosis. 50 These disparities are likely attributable to disproportionate poverty, inadequate access to health services, and other factors that leave the AI/AN population vulnerable to poorer health outcomes. 51 The AI/AN population aged 65 and older has health problems (such as diabetes or a previously diagnosed heart attack) at higher rates and more frequently experience barriers to care than the overall U.S. population age 65 and older. 52 AI/ANs in this age group also report having more difficulties with activities of daily living, and such difficulties are often associated with greater health needs. 53 As with the general AI/AN population, these higher rates of poor health are likely attributable to socioeconomic disparities (e.g., household income, employment status, educational attainment, etc.). 54 Comparison to Hospitals Nationwide. In addition to being federally operated, small, and rural, IHS hospitals have lower patient volume, provide less clinically complex care (as measured by hospitals’ case mix index), and have shorter average length of stays than acute-care hospitals nationally. 55 Each of these three metrics indicates that IHS hospitals provide care to patients that is likely less intensive with fewer procedures and lower costs than the services provided in other hospitals. Exhibit 3 on the next page provides a summary of key differences between IHS hospitals and other acute-care hospitals participating in Medicare. 56 Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Background | 5 Exhibit 3: IHS hospitals provided less clinically complex and intensive care than other acute-care hospitals in FY 2017. Sources: IHS, “Fiscal Year 2017 Hospital Inpatient Statistics for IHS and Tribal Sites with Prior Fiscal Year Comparisons,” the Centers for Medicare & Medicaid Services, “Inpatient Prospective Payment System Impact File 2017,” and the Agency for Healthcare Research and Quality, “Overview of U.S. Hospital Stays in 2016: Variation by Geographic Region.” *A hospital’s case mix index measures the clinical complexity of its patient population. A higher number corresponds to greater complexity. We excluded the five IHS critical access hospitals from our analysis because these hospitals do not participate in the Federal payment program which contains this data. HHS Agencies Involved in Patient Safety CMS and the Agency for Healthcare Research and Quality (AHRQ) are the key U.S. Department of Health and Human Services (HHS) agencies tasked with monitoring patient safety and health care quality across the United States, including in IHS hospitals. CMS. CMS is responsible for the regulation and oversight of care provided in hospitals that participate in Medicare and Medicaid. 57 Under its rulemaking authority, CMS has instituted payment policies to reduce adverse events. 58 CMS also manages multiple national-level initiatives designed to supplement hospital-based efforts to improve quality of care and patient safety such as Quality Improvement Organizations (QIOs) and the Partnership for Patients initiative. QIOs bring together beneficiaries, providers, and communities into Quality Innovation Networks (QINs) to share best practices for enhanced care. 59, 60 The Partnership for Patients initiative is a public-private partnership working to “improve the quality, safety and affordability of healthcare for all Americans.” 61 As part of this initiative, CMS awarded contracts to 16 Hospital Improvement and Innovation Networks (HIINs) that work to make care safer and improve care transitions. 62 HIINs provide training to improve patient safety, and track and monitor hospital progress. 63 AHRQ. AHRQ is tasked with improving the quality and safety of the health care system through research and implementation of evidence-based practices. 64 AHRQ Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Background | 6 has several initiatives to assist hospitals in monitoring and preventing adverse events. AHRQ developed common definitions and formats (known as the Common Formats) for hospitals and other organizations to report data on adverse events. 65 AHRQ also manages the Patient Safety Organization (PSO) program. 66 PSOs aggregate and analyze information (including details about adverse events) and conduct other patient safety activities, which include offering expert feedback and advice to hospitals, physicians, and other health care providers to reduce adverse events and improve patient safety. 67 IHS Hospital Quality and Safety Medicare Conditions of Participation IHS instructs its hospitals to “meet the requirements of a nationally recognized accrediting or certifying body.” 68 Accrediting organizations used by IHS must support the reimbursement requirements established by CMS. 69 Among these requirements are the Medicare Conditions of Participation, a set of minimum quality and safety standards. One of the Conditions of Participation requires hospitals to develop and maintain Quality Assessment and Performance Improvement (QAPI) programs. 70 As part of their QAPI programs, hospitals must “track medical errors and adverse patient events, analyze their causes, and implement preventive actions and mechanisms that include feedback and learning throughout the hospital.” 71 Hospital Incident Reporting Systems To meet the QAPI requirements, IHS operates an incident reporting system, called WebCident, to monitor adverse events and other patient safety issues. 72 Physicians, nurses, and other hospital staff report patient safety-related events into WebCident. Patients and their family members may also report events. Reports typically include first-person accounts and other descriptive information about the events. In addition to logging events into WebCident, IHS hospitals may keep their own incident report files that capture similar information. Prior OIG work found that hospitals nationwide use incident reporting systems as the primary tool to track and analyze patient harm events. 73 Although IHS hospitals operate incident reporting systems, it is likely that not all adverse events are recorded in these systems. A GAO study found that IHS providers did not consistently use WebCident to report adverse events in 2017. 74 IHS officials stated that the lack of reporting resulted in a lost opportunity to address deficiencies. Furthermore, OIG learned from interviews with IHS in December 2019 that the WebCident system is no longer supported and lacks the capability to categorize reports, query data, and send automatic notifications when a report is entered into the system. 75 Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Background | 7 In December 2018, IHS awarded a contract for a new adverse event reporting and tracking system to replace WebCident. 76 IHS recently transitioned event reporting from WebCident to the new system, IHS Safety Tracking & Response (I-STAR). 77 The new system has an improved interface and captures data from IHS-operated facilities, including clinical errors and adverse events. Unlike WebCident, I-STAR supports running queries to create reports, allowing for the compilation of aggregate data. Analysis of these data could enable IHS to better identify potential patterns and vulnerabilities. IHS has also conducted agencywide training for I-STAR. 78 As part of this transition, IHS is currently sunsetting WebCident and will retire the system after all reported events in this system are closed. Hospital Participation in Federal Improvement Efforts IHS hospitals participate in several improvement initiatives through Federal programs. In 2016, as part of the Partnership for Patients, CMS awarded several grants to QIN-QIOs and HIINs to support IHS QAPI programs. 79, 80 In 2017, CMS and IHS reported to OIG that IHS hospitals received support for QAPI programs through these initiatives. 81 One example is the Partnership to Advance Tribal Health, a QIN-QIO initiative that supports best practices and other improvements at IHS hospitals. 82 IHS also has several quality improvement initiatives underway as part of its Quality Framework, launched in November 2016. 83 IHS is in the early stages of implementing new policies and programs under its 2018 Quality Framework and 2019 Strategic Plan. 84 The Quality Framework’s initiatives include the National Accountability Dashboard for Quality to capture performance data to monitor and improve quality of care; a patient experience survey; and others. 85 In December 2018, IHS created an Office of Quality to “provide leadership and promote consistency in healthcare quality across the agency.” 86 This office is responsible for developing and monitoring agencywide quality-of-care standards, which at the time of this study had not yet been fully developed. 87 A new effort by this office is the Quality, Assurance, and Risk Management Program (QARM), which is developing new governance processes and oversight systems to review high-risk issues, clinical issues, business impact, and the financial integrity of IHS facilities. 88 As OIG has recommended, in January 2020, IHS announced the development of a national compliance program to improve the care and treatment for the AI/AN population. 89 IHS Hospitals in the Great Plains Area In recent years, inadequacies at IHS hospitals in the agency’s Great Plains Area have been an area of concern. In February 2016, the Senate Committee on Indian Affairs held a hearing on the substandard quality of care in IHS hospitals in the Great Plains Area. Testimony from CMS and Tribal representatives described quality-of-care concerns at several hospitals in the Area, including noncompliance with both the Medicare Conditions of Participation and the requirements of the Emergency Medical Treatment and Labor Act. Findings of noncompliance included deficiencies in critical Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Background | 8 capabilities such as maintaining staff and equipment. 90 Subsequent hearings in 2017 and 2018 raised similar concerns about IHS hospitals, with testimony covering the following issues: problems with access to care, hospital closures, and difficulty— particularly in the Great Plains Area—maintaining staffing. 91 During this timeframe, IHS hospitals in the Great Plains Area continued to struggle with Medicare compliance. CMS terminated one Great Plains Area hospital from the Medicare program in 2015 and another in 2017. 92 To avoid termination, another hospital completed a Systems Improvement Agreement—a corrective action contract that CMS may undertake with hospitals facing termination. 93 A fourth Great Plains Area hospital, which was at risk for termination, voluntarily stopped accepting inpatient admissions. 94 Measuring Patient Harm Researchers and health care entities may adopt different standards for distinguishing degrees of harm and defining what constitutes an adverse event. The National Coordinating Council for Medication Errors Reporting and Prevention (NCC MERP) Index classifies events by level of patient harm, including categories for circumstances that presented a risk but did not cause harm and those that did cause harm. 95 The NCC MERP Index was initially developed to categorize the effect of medication errors, but researchers have modified the Index to measure and distinguish other types of adverse events. For example, IHI uses a modified version of the NCC MERP Index to measure the degree of harm. 96 OIG has also used a modified NCC MERP Index for its adverse events work since 2008. 97 Related Work During the medical record review for this study, nurse-screeners and physician-reviewers identified instances where the care provided to several labor and delivery patients did not follow national clinical guidelines or best practices. In most of these instances, the care did not cause patient harm. A companion report, Instances of IHS Labor and Delivery Care Not Following National Clinical Guidelines or Best Practices (OEI-06-19-00190) describes these instances. Methodology Scope This report estimates the incidence rates of adverse events and temporary harm events experienced by patients in IHS hospitals in FY 2017. These incidence rates are representative of all patients who received acute-care services in IHS hospitals in FY 2017. The incidence rates are composed of all patient harm events, regardless of whether they were preventable. We do not provide a cost estimate for these events because we did not have admission-level cost data for IHS hospitals. We also do not Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Background | 9 provide projections for characteristics, such as severity and preventability, for which we have too few sample events to make reliable projections. Instead, we present this information as numbers of events identified in our sample. Sample Selection and Profile Using encounter data from IHS’s National Data Warehouse, we selected a stratified, random sample of 400 patients from all patients admitted to one or more of the 26 IHS-operated hospitals during FY 2017. 98 We selected this sample from the general population of patients who had IHS hospital stays which included adults (aged 18 to 64), older adults (aged 65 and older), labor and delivery patients (any age), and pediatric patients (up to and including 17 years of age). See Exhibit 4 for the demographics of the study population. Exhibit 4: Demographics of the IHS Patient Population in FY 2017 Source: OIG analysis of encounter data for 14,949 patients admitted to IHS hospitals in FY 2017. We excluded 15 patients from our analysis because they did not have a conventional inpatient stay for acute-care services. Thirteen of these patients were admitted to a drug dependency unit, and the other two patients were admitted as research participants. We excluded an additional admission because the hospital was unable to locate the medical record for the patient. The final sample consisted of 384 patients. We used a stratified sample design with six strata. IHS patients were placed into strata based on hospital location and size so that we could ensure that the sample included a range of small hospitals, that it did not overly burden larger hospitals with a high number of requests for medical records, and that we could make projections to the Great Plains Area. 99 See Exhibit 5 on the next page for a description of the strata populations and sampled patients. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Background | 10 Exhibit 5: Patient Population and Sample Sizes by Stratum (Hospital Group) Stratum number Total patients Sample size* 1 1,337 50 2 552 40 3 274 40 4 349 70 5 2,720 100 6 9,717 100 Source: OIG analysis of encounter data for 14,949 patients admitted to IHS hospitals in FY 2017. *Sixteen of the 400 selected patients were not included in the final analysis because they were either missing records or did not have a conventional inpatient stay for acute-care services. We reviewed all stays during the year for each sampled patient. Of the 384 patients in our sample, 48 had more than one stay at an IHS hospital during FY 2017 (32 patients had 2 stays, and 16 had more than 2 stays). The IHS patients in our sample had a combined 457 hospital stays with discharges in FY 2017 and an average length of stay of 3.5 days. Data Collection We requested complete medical records for the sampled patients’ stays at IHS hospitals. We reviewed the medical records for completeness and made additional requests for any missing components. Identification of Adverse Events and Temporary Harm Events We conducted a two-stage review to identify adverse events and temporary harm events among sampled patients. (See Appendix A for a further description of our methodology for identifying events and determining preventability and see Appendix B for a glossary of selected clinical terms used to describe events.) Nurse Screening. The first stage was a screening process to identify patients who were likely to have experienced harm events during their stays. Three registered nurses used a trigger tool methodology (based on IHI’s GTT) to look for “triggers”— clinical clues—that indicated possible patient harm. The nurses looked for different triggers depending on patient type (e.g., older adults versus labor and delivery patients). If the nurses identified possible patient harm events during the IHS hospital stays, they flagged them for the second stage of review. The flagged records could include more than one possible harm event. Nurses flagged 57 patients’ records and also requested checks for an additional 24 in which they did not identify likely patient harm but found that the case included complexities that warranted additional review. In total, nurses flagged 81 of the 457 admissions in the sample for the second stage of review. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Background | 11 Physician Review. In the second stage of review, physicians reviewed admissions flagged by nurses as well as an additional 100 admissions for quality assurance. Physician-reviewers used the medical records, results from the nurse’s review, and a summary of the encounter data from IHS’s National Data Warehouse to facilitate their reviews. They either confirmed or refuted the nurses’ findings and independently identified any additional patient harm events. For each event, the physician-reviewers followed a structured protocol that required them to describe the event, the relevant evidence in the medical record, the level of harm experienced by the patient, and whether the event was preventable. In total, physicians reviewed 181 admissions. Assessment of Preventability. The physician-reviewers determined whether events were preventable—i.e., those that could have been avoided if the patients had been given better care. They assigned each event to one of five preventability determinations—clearly preventable, likely preventable, likely not preventable, clearly not preventable, or unable to determine. They selected a rationale for each preventability determination from a list of 25 contributing factors based on prior research and experience in OIG studies of adverse events. This list includes factors that contribute to preventable events, such as medical error or inadequate monitoring, and factors that contribute to nonpreventable events, such as whether a patient was particularly susceptible to an event because of the patient’s health status. See Appendix A for further explanation about assessing preventability. Assessment of Severity. As in prior OIG studies, physician-reviewers assigned each event to one of five harm levels using a modified version of the NCC MERP Index for Categorizing Medication Errors. We distinguish between “adverse events” (levels F through I on the index) and “temporary harm events” (level E on the index) to separately identify events that were more likely to affect costs to Medicare and a patient’s length of stay. In addition, we sometimes use the term “patient harm events” to refer collectively to both adverse events and temporary harm events, as both types of events represent harm resulting from medical care. (See Exhibit 6.) Exhibit 6: OIG-Modified Version of the NCC MERP Index for Categorizing Events Event Type Level Description Harm occurred that may have contributed to or resulted in the patient’s I death. H Harm occurred that required intervention to sustain the patient’s life. Adverse Event G Harm occurred that contributed to or resulted in permanent patient harm. Harm occurred that contributed to or resulted in prolonged facility stay, F elevation in level of care, or transfer to another facility. Temporary E Harm occurred that caused temporary harm that required intervention. Harm Event Source: Adapted from the NCC MERP Index for Categorizing Errors. Revised February 20, 2001. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Background | 12 Data Analysis We analyzed the results of the physicians’ reviews and generated national estimates of the incidence rates of patient harm events in IHS hospitals. We also compared incidence rates for IHS hospitals in the Great Plains Area to those of IHS hospitals in all other IHS Areas, and those of small IHS hospitals to those of large IHS hospitals. However, as explained above, we were unable to reliably project estimates for characteristics of events, such as preventability classification, harm level, and event category, because the sample size was insufficient. Instead, we used sample numbers to describe the characteristics of events. (See Appendix D for point estimates, 95-percent confidence intervals, and key statistics and Appendix E for sample counts.) Incidence Analysis. We estimated the incidence rates within the population from which we selected the sample (patients admitted to IHS hospitals during FY 2017), and we provided the percentage of patients who experienced at least one harm event. We estimated the incidence rates for both adverse events and temporary harm events. As an additional measure, we estimated two ratios of incidence density commonly used by hospitals and medical researchers: events per 1,000 patient days and events per 100 hospital admissions. These metrics allow for a clearer comparison of samples with different lengths of stay and when some patients experience multiple events, making harm measurements more comparable across patient types and facilities. (See Appendix F for further explanation of incidence density estimations.) Analysis of Hospital Size and Location. We analyzed the events to identify differences by hospital size and location. For the hospital size analysis, we categorized the hospitals into 2 groups: smaller hospitals (those with fewer than 1,000 admissions in FY 2017) and larger hospitals (those with 1,000 or more admissions in FY 2017). Most IHS hospitals—20 of the 26 IHS hospitals operating at the time of our study—were considered smaller hospitals, and 6 were considered larger hospitals. IHS hospitals in strata 1 and 6 were comprised of larger hospitals; the remaining strata were comprised of smaller hospitals. For the location analysis, we compared patients in hospitals located in the Great Plains Area to patients in hospitals located outside the Great Plains Area. Analysis of Clinical Category and Patient Type. We analyzed the events by clinical category and patient type. For clinical category, physician-reviewers divided events into four categories: medication, patient care, infection, and procedures. For patient type, we classified patients into four groups: adults (aged 18 to 64), older adults (aged 65 and older), labor and delivery patients (regardless of age), and pediatric patients (those up to and including 17 years of age). We made this distinction because the type of care provided to patients differs depending on patient age and reason for admission (e.g., labor and delivery). Preventability Analysis. We estimated the incidence of preventable patient harm events and provide the percentage of patients within the population who had at least Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Background | 13 one preventable event. Similarly, the incidence of nonpreventable patient harm events is the percentage of patients within the population who had at least one nonpreventable event (but no preventable events). We also present the number of preventable events and the number of nonpreventable events we found in the sample. (See Appendix E for the sample counts.) Limitations These results, as with all medical record reviews, are subject to physician interpretation and clinical judgment. Additionally, medical record reviews are dependent on available documentation; any information that was omitted could lead reviewers to miss some events or to make a different assessment as to whether an event was preventable. Therefore, it is unlikely that the reviewers identified all adverse events and temporary harm events within our sample of patients at IHS hospitals. In addition, we counted only patient harm events that occurred during the inpatient stays or a contiguous emergency department or other outpatient department visit. We did not include harm events that occurred in other health care settings preceding or following the patients’ arrival at the hospital (i.e., present-on-admission events). We also note that the GTT screening methodology used in the first stage of the review is not a comprehensive medical record review. Screeners may have missed some events because their review focused on a specific set of triggers. Analysis in a prior OIG study found that compared to a comprehensive medical record review by physicians, nurses using the GTT screening methodology identified 93 percent of patients with events. 100 Finally, we were unable to project event-level characteristics and instead present relevant findings in terms of sample numbers. We also cannot compare most of the results of this study with those of other studies on adverse events because of the differences in population and settings. We included a descriptive comparison in one case: results for IHS patients 65 and over as compared to our results for Medicare patients in acute-care hospitals nationally and other settings, given their similar age and conditions. We were unable to make a statistical comparison of those results. Standards We conducted this study in accordance with the Quality Standards for Inspection and Evaluation issued by the Council of the Inspectors General on Integrity and Efficiency. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Background | 14 FINDINGS An estimated 13 percent of patients experienced adverse events or temporary harm events during their stays in IHS hospitals Of the nearly 15,000 patients admitted to IHS hospitals in FY 2017, an estimated 13 percent (about 1 in 8) experienced at least 1 patient harm event during their stays. This projects to 1,840 patients experiencing harm as a result of medical care received in IHS hospitals during this period. Patients who experienced patient harm events fell into two categories: › 4 percent of patients experienced adverse events (harm events that resulted in a prolonged hospital stay, permanent patient harm, life-sustaining intervention, or contributed to death), and an additional › 8 percent of patients experienced temporary harm events (harm events that required medical intervention but did not lead to a prolonged hospital stay or cause lasting harm). a Harm rates may be affected in part by patient type, the complexity of care provided, and patients’ length of stay. Prior OIG work on patient harm has focused on Medicare patients, who are older than the general population and require more complex care. 101 A prior OIG study of adverse events among hospitalized Medicare patients found that 27 percent of Medicare beneficiaries experienced harm. Given the differences in populations and samples, we determined that it would not be valid to compare rates across studies. Research from outside OIG has estimated harm rates ranging from 9 to 33 percent for adult and pediatric patients at non-IHS hospitals. 102 See Appendix F for information about limitations to comparing rates of patient harm events in IHS hospitals to those in other health care settings. The incidence rate of patient harm in IHS’s Great Plains Area was similar to that of other IHS Areas The rate of patient harm events in hospitals IHS’s Great Plains Area was not significantly different from the rate of patient harm in the remainder of IHS hospitals. As previously discussed, IHS hospitals in the Great Plains Area are of concern because of past quality and safety issues. Fourteen percent of patients in the Great Plains Area experienced at least one adverse or temporary harm event, and 12 percent of patients outside of the Great Plains Area experienced a harm event; this difference was not statistically significant (see Exhibit 7 on the next page). 103 _____________________________________________________________________________________________________________________________ a The combined total of patients who experienced adverse events (4 percent) and temporary harm events (8 percent) does not total 13 percent because of rounding. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Findings | 15 Exhibit 7: Patients in IHS’s Great Plains Area had rates of adverse events and temporary harm events that were similar to those for patients in other IHS hospitals (n=384). Source: OIG analysis of 384 IHS patients admitted to IHS hospitals in FY 2017. Notes: We found no statistically significant difference in the rate for hospitals in IHS’s Great Plains Area and the rate all other IHS hospitals (p=.5422). In addition, the rates of temporary harm events and adverse events for IHS hospitals in the Great Plains Area do not total the rate of the two types combined because of rounding. Patients in smaller IHS hospitals were more likely to experience adverse events and temporary harm events than patients in larger IHS hospitals The rate of patient harm among smaller IHS hospitals (hospitals with fewer than 1,000 admissions in FY 2017) was higher than the rate we observed in larger hospitals. Nineteen percent of patients in smaller hospitals experienced patient harm events, compared to 9 percent of patients in larger hospitals (see Exhibit 8 on the next page). 104 A prior OIG report found that providers in some IHS hospitals struggle to maintain clinical competence as a result of low patient censuses and limited scopes of services, with providers having less opportunity to exercise their skillsets. 105 Although the prior work did not distinguish between smaller and larger IHS hospitals, this issue may be compounded in smaller IHS hospitals, particularly those in remote locations. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Findings | 16 Exhibit 8: Patients in smaller IHS hospitals were more likely to experience an adverse or temporary harm event than patients in larger IHS hospitals (n=383*). Source: OIG analysis of 384 IHS patients admitted to IHS hospitals in FY 2017. *For one patient with two hospitalizations during the year, one admission was ineligible for this particular analysis because the patient did not receive acute-care services. As a result, the n here is 383 rather than 384 patients. Note: We found a statistically significant difference in the rate of adverse and temporary harm events between smaller IHS hospitals and larger IHS hospitals (p=.0094). Most of the 79 patient harm events in our sample were temporary harm events Physician-reviewers identified a total of 79 patient harm events; 19 of these events were adverse events, resulting in prolonged hospital stays and other serious consequences, and 60 were temporary harm events that required medical intervention but did not cause lasting harm. Sixty-one of the 384 patients in our sample experienced harm, with a small number of sampled patients (11 of the 61 patients) experiencing multiple unrelated harm events during their stays. (See Exhibit 9 for the number of adverse events and temporary harm events identified in our sample by harm level.) Exhibit 9: Most events were classified as temporary harm (n=79 events). Number of Event Type Level of Harm Events F level: Resulted in prolonged hospital stay, an elevation in the 13 level of care, or resulted in transfer to another facility G level: Contributed to or resulted in permanent patient harm 1 Adverse Events H level: Required intervention to sustain the patient’s life 4 I level: Contributed to or resulted in patient death 1 Temporary Harm Events E level: Resulted in temporary harm and required intervention 60 Total All levels of harm 79 Source: OIG analysis of 384 patients admitted to IHS hospitals in FY 2017. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Findings | 17 More than half of the patient harm events in our sample related to the use of medication We found that—as in previous OIG studies of patient harm—medications and patient care led to the highest numbers of harm events. Medication-related events were the most common patient harm events in our sample (41 of 79 events). Patients who experienced these events often suffered from oversedation, confusion, and hallucinations. Medication-related events also included hypoglycemic episodes in patients with diabetes and problems with blood pressure management (e.g., hypotension while on medication). Patient care events were the next most common type of harm event in our sample (29 of 79 events). This category pertains to the daily care of patients, which is often performed by nurses. Patient care events commonly involved intravenous catheter infiltration (leaking of fluid or medication into surrounding tissue) that resulted in burns, swelling, and pain (nine events). Other patient care events involved postpartum hemorrhage; pressure injuries; and skin tears, abrasions, and breakdowns. Six events in our sample related to infections or procedures. Two of the six infections involved serious surgical site wounds, with one leading to permanent harm and the other necessitating life-saving intervention. Only three events in our sample related to medical procedures, possibly because IHS hospitals perform few surgeries. (As previously noted, some IHS hospitals lack operating rooms, so patients requiring surgical intervention are transferred to other locations.) Procedure-related events included a postoperative atrial fibrillation (irregular heartbeat) and a procedure-related laceration during labor and delivery. See Exhibit 10 (on the next page) for the list of events identified within each of the four clinical categories (medication, patient care, infections, and procedures). Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Findings | 18 Exhibit 10: More than half of the patient harm events in our sample related to the use of medication (n=79 events). Adverse Events and Temporary Harm Events by Clinical Category Number of Events Medication 41 Delirium or other change in mental status 12 Hypoglycemia (low blood sugar) 7 Hypotension/hypertension (low blood pressure/high blood pressure) 6 Allergic reaction to medication 3 Diarrhea while on medication 3 Tachysystole (excessive uterine contractions) and related conditions 3 Nausea and vomiting 2 Cardiac arrhythmia 1 Excessive bleeding while on multiple blood thinners 1 Interstitial nephritis (inflammation of the kidney) 1 Prolonged constipation while on opioids 1 Seizure leading to fall and head injury 1 Patient Care 29 Infiltration, burn, or vein inflammation from intravenous catheter 9 Postpartum hemorrhage 4 Pressure injury 3 Skin tear, abrasion, or breakdown 3 Fluid/electrolyte disorders 2 Hypotension 2 Allergic reaction 1 Diabetic ketoacidosis 1 Exacerbation of abnormal liver condition 1 Fall that led to contamination and infection 1 Life-threatening oxygen desaturation 1 Severe hypertension/preeclampsia 1 Infections 6 Soft tissue or other nonsurgical infections 4 Surgical site infections 2 Procedures 3 Atrial fibrillation 1 Procedure-related laceration 1 Urinary retention 1 Source: OIG analysis of 384 IHS patients admitted to IHS hospitals in FY 2017. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Findings | 19 Older adults and labor and delivery patients in our sample experienced higher proportions of harm events; pediatric patients had fewer harm events Proportionally, we found more harm events among older adults (aged 65 and older) and labor and delivery patients in our sample, and fewer events among pediatric patients (17 and younger). Given the small number of events, we were unable to project the rates of adverse events and temporary harm events for subgroups of patients determined by age or patient type. Adults ages 18 to 64 who were not labor and delivery patients appeared to have a similar proportion of harm (15 percent) as the overall incidence rate (13 percent). See Exhibit 11 for the number and unweighted percentage of patients who had adverse events and temporary harm events within the three subgroups. Exhibit 11: Older adults and labor and delivery patients experienced more adverse events and temporary harm events than other patients in our sample (n=384). Source: OIG analysis of 384 IHS patients admitted to IHS hospitals in FY 2017. Note: These percentages are unweighted and not projectable to all IHS patients in these groups. Thirty percent of older adults (aged 65 and older) in our sample experienced an adverse event or temporary harm event In our sample, we found that 18 of 61 older adults (30 percent) experienced an adverse event or temporary harm event during their stays in IHS hospitals. As previously mentioned, our prior study of adverse events among Medicare beneficiaries, typically age 65 and older, found that 27 percent experienced adverse events or temporary harm events. 106 Older adults often experienced more than a single patient harm event, with a total of 29 adverse events and temporary harm events experienced by these 18 patients. As with the broader sample of patients, harm events among older adults were most commonly related to medications (15 of 29 events) and patient care (9 of 29 events). Many of the medication-related events related to delirium or other changes in mental status (8 events) with most of these events involving the administration of opioids Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Findings | 20 leading to confusion and hallucinations. Patient care events included issues such as pressure injuries (three events) and problems with intravenous catheters (two events). One factor that may have contributed to the higher proportion of harm events among older adults in our sample is the length of their hospital stays. The average length of stay for older adults in our sample was 4.8 days, compared to 3.5 days for sampled patients overall. Patient Story 1 An older adult patient with multiple medical problems and a history of heart failure was admitted to the hospital for severe leg swelling and cellulitis (skin inflammation). During the hospital stay, the patient was given excessive medication for cramps and insomnia (a total of 3 doses of 2 different benzodiazepines over 12 hours). The patient woke during the night confused (a known side effect of the medication) and fell out of bed. A nurse found the patient confused and on the floor minutes later. The attending physician later evaluated the patient and discontinued the medication. About one-fifth of labor and delivery patients in our sample experienced adverse events or temporary harm events We found that 10 of 48 labor and delivery patients in our sample (21 percent) experienced at least one adverse event or temporary harm event. Two of these patients experienced 2 harm events each, resulting in a total of 12 events experienced by labor and delivery patients. Four patients experienced postpartum hemorrhage that resulted in temporary harm. As described in the companion report about instances of care not meeting national clinical guidelines or best practices for labor and delivery, we found that postpartum hemorrhage was unusually common in our sample—experienced by one-third of labor and delivery patients (16 of 48). 107 For reference, research suggests that 1 to 3 percent of labor and delivery patients nationwide experience postpartum hemorrhage. 108 For two of the four patients who experienced a postpartum hemorrhage, our physician-reviewer determined the care (or lack of care) provided may have exacerbated the hemorrhage and resulted in excessive blood loss. The two other patients who experienced postpartum hemorrhage both had clinical complications (preeclampsia) that required their labor to be induced. Postpartum hemorrhage is a known risk factor associated with induced labor; therefore, our review determined that appropriate care was provided, and these two events were not preventable. 109 The other 12 hemorrhages did not result in harm to the mother or newborn. Among the other eight harm events involving labor and delivery patients were three patients with tachysystole (excessive uterine contractions) as a result of medications used to induce labor (oxytocin or misoprostol) and two patients with hypotension (dangerously low blood pressure) as a result of epidural anesthesia (for pain relief). Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Findings | 21 Patient Story 2 A woman, admitted for labor and delivery, was induced using oxytocin. After induction, the patient experienced adequate uterine activity for a progression of labor, but staff did not appear to recognize this and unnecessarily increased the dosage of oxytocin which resulted in tachysystole (excessive uterine contractions). The increase in induction medication was not consistent with clinical guidelines and could have resulted in additional harm to the uterus or the newborn. Relatively few (4 of 81) pediatric patients in our sample experienced adverse events and temporary harm events We found that a small proportion (5 percent) of pediatric patients in our sample experienced an adverse or temporary harm event. This 5 percent was composed of four sampled patients. Three of the four patients had harm events that were related to patient care, and the fourth had an event related to medication. None of these patients had more than one harm event. The average length of stay for sampled pediatric patients was 2.2 days. Patient Story 3 A child was admitted for chronic respiratory disease and required supplemental oxygen. The attending physician prescribed a corticosteroid (anti-inflammatory medication) for treatment of the child’s respiratory symptoms. The staff gave the child more than five times the prescribed dose of the medication (125 mg instead of 20 mg). The child developed diarrhea, which was caused by the higher than appropriate dose. After discovering the error, staff contacted poison control and provided supportive care for the overdose. Most patient harm events in our sample occurred within smaller hospitals or involved older adults or labor and delivery patients Hospital size and patient type may be characteristics of interest when assessing patient harm. As mentioned above, patients who received care in smaller hospitals were more likely to experience patient harm, with 54 of the 79 events occurring in smaller hospitals. Older adults within our sample experienced more harm events per capita than any other patient type, with a total of 29 events. Although the sample included fewer labor and delivery patients, this group also experienced a high number of harm events per capita, with a total of 12 events. Given that patients could belong to more than one of these subgroups (e.g., older patients could be treated in smaller hospitals), these characteristics were present in 68 of the 79 patient harm events identified. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Findings | 22 An estimated 7 percent of patients experienced adverse events or temporary harm events that could have been prevented if patients had been given better care An estimated 7 percent of patients overall—nearly half of the 13 percent of patients who experienced harm events—experienced events that were preventable. Across all patient harm events, physicians determined that about half of the events in our sample (38 of 79) were preventable, i.e., they could have been avoided if patients had received better care. The other half of events (40 of 79) were determined to have been nonpreventable. Most of the more serious adverse events in our IHS sample were preventable (15 of 19 adverse events), whereas less than half of temporary harm events were considered preventable (23 of 60 events). Exhibit 12 below shows the number of events assigned to each preventability classification. (Our sample size was insufficient to project these estimates; see Appendix E for more information.) Exhibit 12: About half of the patient harm events identified in our sample were considered preventable (n=79). Preventability assessment Events within the sample Preventable – Harm could have been avoided through improved 38 (48%) assessment or alternative actions Likely preventable 29 Clearly preventable 9 Not preventable – Harm could not have been avoided given the complexity 40 (51%) of the patient’s condition or care required Likely not preventable 34 Clearly not preventable 6 Unable to determine preventability 1 (1%) Source: OIG analysis of medical records for 384 patients admitted to IHS hospitals in FY 2017. Note: These percentages are unweighted and not projectable to all events. Preventable events in our sample involved substandard treatment, medical errors, and lack of monitoring Half of the preventable events in our sample (19 of 38) were the result of substandard treatment—i.e., the failure to adhere to national clinical guidelines and current best practices when delivering care. Other common factors that contributed to preventable events included medical errors, lack of adequate monitoring, and failure by clinicians to provide necessary treatment. These contributing factors are not necessarily mutually exclusive, and our physicians-reviewers often identified multiple factors as having contributed to preventable events. Exhibit 13 on the next page shows the most common factors that contributed to preventable events. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Findings | 23 Exhibit 13: Substandard treatment contributed to half of preventable adverse events and temporary harm events in our sample (n=38). Source: OIG analysis of 384 patients admitted to IHS hospitals in FY 2017. *Reviewers often selected more than one rationale per event. Note: These percentages are unweighted and not projectable to all preventable events. Substandard treatment contributed to 19 events in our sample. These include delirium or other changes in mental status from oversedation; tachysystole from unnecessarily high doses of medication (misoprostol) used to induce labor; and failure to properly treat infections. For example, in one event, the overly aggressive use of antihypertensive blood pressure medication led to hypotension and kidney injury. In another event, a subtherapeutic dose of an antiepileptic medication contributed to a seizure leading to a fall with injury while the patient was left unattended. Patient Story 4 One preventable event involved a delay in diagnosis and treatment of symptomatic hypercalcemia (high calcium levels), a condition that is often painful and can be quickly treated with saline. The patient, who was ultimately diagnosed with a blood cell cancer, was admitted for abdominal pain and had significant hypercalcemia, unexplained anemia, and kidney failure. Providers at the hospital failed to diagnose the hypercalcemia and did not recognize the symptoms of the cancer despite a classic presentation of the disease. They transferred the patient to a non-IHS hospital for imaging related to the abdominal pain 2 days after admission. Staff at the second hospital correctly identified the hypercalcemia and diagnosed the patient with a blood cell cancer. The physician-reviewers determined that the missed diagnosis of hypercalcemia was clearly preventable given the information available. The reviewers categorized the reasons for this harm as both substandard care and error in clinical judgment. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Findings | 24 Nonpreventable events in our sample often involved patients who were in poor health and therefore highly susceptible to harm events Almost half of the nonpreventable harm events in our sample (19 of 40 nonpreventable events) occurred because the patients’ poor health status left them susceptible to harm events. The other two most frequently cited factors used to classify events as nonpreventable were that the event occurred despite proper assessment and that the procedures being followed; and the provider could not have anticipated the event with information available at the time. See Exhibit 14 for the most common factors related to nonpreventable harm events. Exhibit 14: Nonpreventable events in our sample often related to patient susceptibility due to poor health (n=40). Source: OIG analysis of 384 patients admitted to IHS hospitals in FY 2017. *Reviewers often selected more than one rationale per event. Note: These percentages are unweighted and not projectable to all preventable events. Patient susceptibility due to poor health often included older adults with multiple comorbidities (13 of 19 nonpreventable events). These nonpreventable events involved delirium due to medication following major surgeries and procedures; hypotension while on medication to treat pre-existing health issues (e.g., congestive heart failure); pressure injuries due to immobility with those having a terminal condition; and skin tears or breakdowns among those with frequent stools. Patient Story 5 One nonpreventable event involved an adult patient with liver cancer who was transferred to an IHS hospital for care. The patient had a history of diabetes, required a clear liquid diet, and was hypermetabolic (experiencing an increased rate of metabolic activity) which reduced the patient’s insulin production. Less than a day after the transfer, the patient received their regular dose of insulin and then experienced a life-threatening hypoglycemic event. The physician-reviewers determined that the hypoglycemic event was likely not preventable because the patient’s liver cancer and associated weight loss may have impacted their ability to process insulin and glucose. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Findings | 25 RECOMMENDATIONS The rates of patient harm and preventable events confirm that IHS hospitals have both the need and opportunity to improve patient care and safety. An estimated 13 percent of hospitalized IHS patients experienced adverse events or temporary harm events during their stays in FY 2017, with patients in smaller IHS hospitals more likely to experience patient harm than those in larger IHS hospitals. In our sample, we identified proportionally more harm among older adults (age 65 and older) and labor and delivery patients than among pediatric or young adult patients. Our physician-reviewers determined that half of the patient harm events in our sample were preventable—i.e., they could have been avoided if the patients had been given better care. IHS has ongoing and new initiatives designed to improve its quality of care. Recent efforts have included creating an Office of Quality and a Quality Framework to guide the organization’s improvement activities. We recommend that IHS: Establish patient harm monitoring and reduction as a key priority of the Office of Quality As mentioned previously, IHS created an Office of Quality in late 2018 to guide the organization’s improvement efforts. IHS is also in the early stages of implementing new policies and programs under its 2018 Quality Framework and 2019 Strategic Plan. However, it is unclear how it will incorporate patient safety into these efforts. IHS should ensure that patient safety, including identification of patient harm events, is a key priority of the new Office of Quality. As part of this role, the Office of Quality should monitor patient harm events in IHS’s incident reporting system to identify opportunities for training and quality improvement efforts. The Office’s identification of harm events could help hospital administrators and providers set goals for improvement, direct resources, and assess the effectiveness of prevention strategies. As part of its focus on patient safety, the Office of Quality should also produce educational correspondence or materials for hospitals and Area Offices that include a definition of “adverse event”; a list of potential patient harm events to educate staff on the range of harm patients can experience; evidence-based best practices for reducing harm in IHS hospitals; and best practices for improving staff identification and tracking of adverse events. The Office of Quality should also support Area Offices and hospital leadership in actively disseminating this information to IHS hospital providers to ensure widespread implementation of harm detection and safety practices. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Recommendations | 26 Effectively track and monitor patient harm events using an improved incident reporting system Prior OIG work found that hospitals nationwide use incident reporting systems as the primary tool to track and analyze patient harm events. IHS has invested in a new incident reporting system that has the potential to advance hospitals’ abilities to use patient harm events to identify opportunities for improvement. In announcing the system in 2018, IHS outlined a number of goals for the system, including improving management of incidents and ease of use. A December 2019 OIG report recommended that IHS ensure that the new reporting system is effective and allows staff to query and aggregate incident data. 110 Although the rollout of the new incident reporting system was delayed, IHS transitioned to its new system, I-STAR, in August 2020. Depending on the quality and the extent of its use, IHS’s new incident reporting system could enable IHS to effectively monitor harm events, facilitate learning from these harm events, and identify areas for improvement efforts. To ensure I-STAR allows IHS to effectively track and monitor patient harm events, IHS should continue to develop and institute policies and agencywide training for this new system. Implement quality improvement plans to improve patient safety across IHS, including plans that focus specifically on smaller hospitals and patient groups at higher risk of harm IHS should design and implement quality improvement plans to improve patient safety and reduce the incidence of adverse and temporary harm events across IHS. This will require IHS to identify and address patient safety disparities for patient populations that may be at greater risk of harm. Our analysis found that patients in smaller hospitals were more likely to experience harm events than those in larger hospitals. We also found that older adults and labor and delivery patients experienced proportionately more harm events than other patient groups. In addition to focusing on the subset of hospitals and patient groups that we identified through our analysis for this report, IHS should work on an ongoing basis to identify facilities and patient groups at greater risk. This effort should be evidence-based, with the goal of improving patient outcomes. IHS’s efforts to focus quality improvement plans on these hospitals and patient groups should include provisions unique to these specific challenges For example, a quality improvement plan for smaller hospitals could address maintaining clinical competence in the absence of a high volume of cases; a plan for older adults could focus on common comorbidities such as diabetes and heart conditions; and a plan for labor and delivery patients could include implementation of recommendations in our companion report, such as use of best practices in diagnosing and treating postpartum hemorrhage. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Recommendations | 27 AGENCY COMMENTS AND OIG RESPONSE IHS concurred with our recommendations and affirmed that patient safety is a high priority for the agency. IHS described actions it has taken to improve patient safety since 2017, including efforts toward implementing the recommendations. OIG values the steps that IHS has taken and will monitor progress in implementing these recommendations as IHS continues its efforts to improve patient safety. (For the full text of IHS’s comments, see Appendix G.) In response to our first recommendation that IHS establish patient harm monitoring and reduction as a key priority of the Office of Quality, IHS reported that it has partnered with several HHS agencies, including AHRQ, CMS, and the Centers for Disease Control and Prevention (CDC), in its efforts to improve patient safety. For example, IHS and CMS established all-cause harm reduction as a priority of the Partnership to Advance Tribal Health—an initiative that supports best practices and improvements at IHS hospitals. This prioritization is an important first step. As IHS further develops new safety initiatives, it should monitor results and ensure that the Office of Quality has clear responsibility for monitoring harm and taking action, and that its efforts result in reducing harm. Regarding staff education efforts, the Office of Quality manages a web-based Quality Portal to support information sharing among IHS hospital and health center staff and leadership. IHS reported it is also implementing trainings such as AHRQ’s Team STEPPS program—a teamwork system designed to improve patient safety— and has conducted trainings with the CDC and the American Hospital Association to improve infection control practices at IHS facilities. Regarding medication safety, IHS established a program to provide clinicians with knowledge, tools, and resources to reduce the risks associated with medications, and for risks related to opioids IHS recruited Area Office mentors to improve appropriate use of Naloxone (a medication used to treat opioid overdoses). In addition to these efforts, the Office of Quality should continue to identify areas for educational outreach to hospitals and health clinics, and develop trainings and materials to address these areas. The educational materials should directly address harm identification and best practices for identifying and reducing harm. In response to our second recommendation that IHS effectively track and monitor patient harm events using an improved incident reporting system, IHS reported that it has fully implemented its new incident reporting system, I-STAR, across all IHS Areas and facilities. IHS reported that the Office of Quality monitors reporting from the I-STAR system to identify adverse events and to address challenges in entering information into the system, and works with IHS Area Offices and facility staff to optimize I-STAR, provide training, and improve reporting. As IHS gains experience Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Agency Comments and OIG Response | 28 using its new I-STAR system, we will look to IHS for evidence that it is effectively using this system to track and monitor patient harm events. In response to our third recommendation that IHS implement quality improvement plans to improve patient safety, IHS stated that its hospitals complete QAPI plans in accordance with CMS regulations and accreditation standards, and confirmed that these plans target patient safety and adverse events. It further stated that surveys of IHS hospitals and health centers have found full compliance with these standards. In 2020, IHS established a national compliance program as a component of the IHS Enterprise Risk Management Program. The program’s activities included oversight reviews of high-risk subject areas for all IHS Area Offices. This included reviews of IHS hospital Governing Board minutes to ensure that patient safety performance improvement measures were included in QAPI plans, and that when a suspected or potential patient safety event occurs, QAPI plans are reviewed to ensure performance improvement measures are incorporated in IHS hospitals. As IHS continues to develop QAPI plans, it should identify opportunities to focus these plans on smaller hospitals and patient groups at higher risk of harm. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Agency Comments and OIG Response | 29 APPENDIX A Methodology for Identifying Events and Determining Preventability We conducted a two-stage medical record review to identify adverse events and temporary harm events experienced by patients in our sample. In the first stage, one of three registered nurses (referred to as “nurse-screeners”) identified possible patient harm events during the IHS hospital stays and flagged the medical records of those patients. Records flagged by nurse-screeners were reviewed in the second stage of the medical record review. The second stage included comprehensive medical record reviews conducted by one of the six contracted physicians. Every record was reviewed by a nurse-screener and, if flagged, reviewed by a physician. Using patient characteristics and diagnosis codes, we grouped patients into one of four categories at the time of sample selection: labor and delivery (regardless of age), pediatrics (aged up to 17), adults (aged 18 to 64), and older adults (aged 65 and older). Pediatric patients and labor and delivery patients were assigned to physicians with specialized experience with those demographics. Additionally, one nurse-screener who is a perinatal expert was responsible for screening all labor and delivery cases. Stage One: Nurse Screening To identify patients who were likely to have experienced events during their stays, nurse-screeners reviewed medical records for the IHS hospital stays using a trigger tool. The protocol required nurse-screeners to look for “triggers” that indicate possible patient harm. A trigger is a clinical clue (e.g., a laboratory test showing low blood glucose, or a patient care event such as a fall) that requires the nurse-screener to explore the medical record to determine whether adverse or temporary harm events likely occurred. It could be the harm itself, such as a pressure injury, or a reference that indicates possible harm, such as transfer to a higher level of care. Three registered nurses used different trigger tool modules to review medical records depending on the patient’s age and reason for admission (e.g., labor and delivery). (See Appendix C for a list of the triggers used to identify events.) › Adult and older patients—Nurse-screeners used a modified version of the Institute for Healthcare Improvement’s (IHI’s) Global Trigger Tool (GTT) to review all adult patients (aged 18 and over). This tool includes triggers in five categories: patient care, intensive care, medication, surgery, and long-term/non-acute-care. The long-term/non-acute triggers were included to facilitate identification of harm in patients who remained in an IHS hospital Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix A | 30 after they no longer required inpatient acute care, as IHS patients may be admitted or have their stay extended for social reasons or if there is not a post-acute-care facility nearby. The triggers were selected from among those developed for OIG studies of adverse events in post-acute-care settings. › Labor and delivery patients—For labor and delivery patients, nurse-screeners followed the same OIG-modified GTT as was used on adult patients. An additional module of perinatal triggers was also incorporated. The perinatal triggers were drawn from the IHI free-standing Perinatal Trigger Tool with some additional triggers suggested by OIG-contracted clinicians who are experts in perinatal trigger tools. › Pediatric patients—For pediatric patients, nurse-screeners used the OIG’s modified Global Assessment of Pediatric Patient Safety (GAPPS) Trigger Tool. 111 This tool was developed by the Center of Excellence for Pediatric Quality Measurement and funded by AHRQ and CMS pediatric quality programs. 112 This tool includes triggers in six categories: medications/fluids, hospital care environment, health care-associated infections, transfer/outcomes, surgery, and neonatal/pediatric intensive care. A seventh module of long-term/non-acute triggers was added to better recognize harm in pediatric patients who remained in an IHS hospital after they no longer required acute-care services. In addition, we modified the tool for consistency with other OIG reviewer guidance, such as including Stage 1 pressure injuries as adverse events and not including adverse events that were present on admission. For each possible event, the screeners recorded a description of the event, the level of harm, and relevant evidence in the medical record. Using the GTT methodology, nurse-screeners flagged 57 admissions for referral to the second stage of review and also requested checks for an additional 24 in which they did not identify likely patient harm but found that the case included complexities that warranted additional review. Altogether, nurses flagged 81 of the 457 admissions in the sample for the second stage of review. The flagged admissions could include more than one possible event. The screening process enabled us to reduce the number of records requiring second-level review of the full medical records by a physician. As in the other OIG studies of adverse event incidence, physician-reviewers indicated that the results of the stage-one screening helped them to readily identify potential events for consideration. Stage Two: Physician Review One of six physicians reviewed the medical records for each of the 81 admissions flagged by the nurse screeners. As part of our quality assurance process, physicians reviewed an additional 100 admissions that were not referred by nurses. Altogether, physicians reviewed 181 admissions. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix A | 31 Physicians conducted comprehensive reviews of the medical records. The physician-reviewers used the medical records, nurse-screener results, and a summary of the encounter data from IHS’s National Data Warehouse to facilitate their reviews. They independently identified adverse events and temporary harm events and also confirmed or dismissed the possible harm events flagged by the screeners in the first stage of review. For each event identified, the physician-reviewers followed a structured protocol that required them to describe the event, the relevant evidence in the medical record, the level of harm experienced by the patient (i.e., severity), and whether the event was preventable. When an initial event caused a series of related events, physicians collapsed the events into a “cascade” and counted it as a single event. For labor and delivery adverse events, physicians identified events that occurred in the delivery room as attributable to the mother’s care and not that of the newborn. The physician-reviewers represented a variety of specializations and experience including cardiology, infectious disease/internal medicine, orthopedics, neurology/physical medicine and rehabilitation, pediatrics, and obstetrics-gynecology. All of the physician-reviewers were experienced with trigger tools. Four of the six had served as physician-reviewers in prior OIG studies of adverse events. In addition, our obstetrician-gynecologist was an expert in the perinatal trigger tool and our pediatrician was one of the developers of the GAPPS Trigger Tool. Assessment of Severity. As in prior OIG studies, physician-reviewers assigned each event to one of five levels of harm using a modified version of the NCC MERP Index. We distinguished between “adverse events” (levels F through I on the index) and “temporary harm events” (level E on the index) to separately identify events that were more likely to affect cost and length of stay. In addition, we sometimes use the term “patient harm events” as a combination of both adverse events and temporary harm events because both types represent harm resulting from medical care (see Exhibit 6 on page 12). Assessment of Preventability. Physicians assigned each event to one of five preventability determinations and identified one or more factors that contributed to each event. (See the five-point scale in Exhibit A-1 on the next page.) Physicians also selected a rationale for each determination based on a list of 25 contributing factors gleaned from prior research and experience in OIG studies of adverse events. 113 Because physicians identified many factors, we present only the most common contributing factors in our analysis rather than an exhaustive list of each factor. Contributing factors varied depending on the circumstances of each event. For example, preventable events may be related to substandard treatment, medical error, and inadequate monitoring depending on the factors involved. Nonpreventable events may be related to a patient’s diagnosis or treatment being unusual or complex and thereby making care difficult, or a patient being highly susceptible to harm because of poor health. These factors are not necessarily exclusive of each other and their definitions are often subjective. For example, substandard care generally refers to the failure to adhere to professional standards of practice in the delivery of care, Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix A | 32 but practitioners may not always agree on what this includes, and their determination depends on the strength of the evidence available to substantiate whether a patient received substandard care. As a result, preventability determinations are necessarily subjective and required the physicians to use clinical experience and judgment. Physicians based decisions on the circumstances of the specific case and also considered accepted standards of care; the expected frequency of certain events; guidance developed during the review process; and group discussion of the patients and events. Physicians were allowed to choose multiple contributing factors for the rationale behind their determinations. Assessing an event as clearly preventable or clearly not preventable required a greater degree of certainty on the part of the reviewer. The expanded scale enabled physicians to make more precise determinations, while our primary statistics collapse clearly and likely into the larger categories of preventable or not preventable. Exhibit A-1: Preventability Determinations Preventability determination Description Patient harm could definitely have been avoided through improved Clearly preventable assessment or alternative actions. Patient harm could have been avoided through improved assessment Likely preventable or alternative actions. Patient harm could not have been avoided given the complexity of the Likely not preventable patient’s condition or the care required. Patient harm could definitely not have been avoided given the Clearly not preventable complexity of the patient’s condition or the care required. Physicians were unable to determine preventability because of Unable to determine incomplete documentation or case complexity. Efforts To Improve Consistency and Quality of Reviews To promote consistency and accuracy across reviews, we issued a study-specific guidance document for improved decision-making, we provided training to all reviewers, we facilitated consensus calls with the physician-reviewers, and we conducted quality assurance reviews. Guidance Document. We provided reviewers with a guidance document that included event definitions and considerations for specific types of events and included a list of frequently asked questions. We created the guidance document to align with clinical research literature; professional and government guidelines (such as evidence-based practices); decisions made in prior OIG studies; and consultations with subject-matter experts. The document also provides instructions that are Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix A | 33 applicable to a wide range of events, including how to assess event timing, underlying disease, related events, and recurring events: › Present on admission—We excluded events that occurred before the patient entered the IHS hospital or that were attributable to care provided prior to admission. › Underlying disease—We excluded events that were part of the underlying disease process unless there was an omission of care resulting in an exacerbation of the underlying disease. › Related events—When an initial event caused a series of related and dependent events, we combined the events into a “cascade” event and counted it as a single event. › Recurring events—When an event recurred during an IHS hospital stay (e.g., two episodes of hypoglycemia), we considered the timeframe and the circumstances of the event. We counted recurring events as a single event if they happened under similar circumstances or were less than 7 days apart. We counted them as separate events if the circumstances that led to the events were substantially different and the events were more than 7 days apart. 114 Training. We provided two trainings for each reviewer regarding the OIG methodology for identifying and classifying adverse and temporary harm events. The training included a review of the guidance document and an explanation of the protocol questions. Each reviewer also performed pre-test reviews and received feedback on the results of those reviews. Consensus Calls. We facilitated regular conference calls to further promote consistency across physician-reviewers. During these calls, physician-reviewers discussed events that were complex, difficult to assess, involved issues outside their area of expertise, or had possible implications for other cases. Once physicians completed the medical record reviews, we held one additional call, during which physicians reviewed subsets of events to ensure that the event determinations and classifications of severity and preventability were made in a manner consistent with other similar events. The goal of these calls was to reach consensus and to establish consistency among the reviewers. Quality Assurance Reviews. We compared the identified events, harm-level determinations, and preventability determinations across groups and looked for deviations from our physician guidance document. We also conducted separate quality assurance reviews and discussed any inconsistencies with the reviewers. These included re-reviews of readmissions and admissions involving patient deaths, as well as re-reviews to check for missed events. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix A | 34 APPENDIX B Glossary of Selected Terms Adverse event—Harm to a patient as a result of medical care or in a health care setting, including the failure to provide needed care. (Adverse events are Levels F through I on the OIG-modified NCC MERP Index.) Atrial fibrillation—A quivering or irregular heartbeat that can lead to blood clots, stroke, heart failure, or other heart-related complications. Cardiac arrhythmia—An irregular heart rate or rhythm. Cascade—A chain of events initiated by an unexpected result or other incident that may result in patient harm. Cellulitis—Spreading inflammation of tissue, most commonly the skin, caused by a bacterial infection. Delirium—Mental disturbance characterized by acute confusion, disordered speech, and hallucinations. Hypercalcemia—A condition in which the blood contains an unusually high level of calcium. Hypermetabolism—An abnormal increase in the metabolic rate (i.e., an abnormal increase in the body's cellular chemical activity). Hypertension—Condition of abnormally high blood pressure. Hypoglycemia—Condition of abnormally low-level blood sugar (glucose). Hypotension—Condition of abnormally low blood pressure. Interstitial nephritis—A kidney disorder characterized by swelling in between the kidney tubules. Ketoacidosis—A severe diabetic complication that occurs when the body produces high levels of acids in the blood due to the body’s rapid breakdown of fat as fuel. Opioid—A class of drugs most-often prescribed to treat moderate to severe pain, notable for their addictive potential. Patient harm event—Any harm to a patient as a result of medical care. This term encompasses both adverse events (Levels F through I on the OIG-modified NCC MERP Index) and temporary harm events (Level E on this index). Perinatal—The time period immediately before or after birth, typically within a number of weeks. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix B | 35 Postpartum hemorrhage—Excessive bleeding immediately following birth, characterized by a decrease in blood pressure, an increase in heart rate, and a decrease in red blood count. Preeclampsia—A pregnancy complication characterized by unusually high blood pressure and protein in the urine. Tachysystole—Excessive contractions of the uterus during labor and delivery. Temporary harm—Harm to a patient that required intervention but did not cause lasting harm. Classified as Level E on the OIG-modified NCC MERP Index. Urinary retention—Difficulty urinating and completely emptying the bladder. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix B | 36 APPENDIX C Trigger Tool Used To Screen for Patient Harm Events For this study, OIG and its contracted clinical consultants developed and used different trigger tools specific to IHS hospital stays based on the IHI GTT and the GAPPS Trigger Tool. To develop these trigger tools, we reviewed and selected triggers from the IHI GTT, GAPPS Trigger Tool, and from among the triggers that were included in prior OIG studies. We chose triggers that the clinicians determined to be most applicable to different types of IHS hospital stays. We used the OIG-modified GTT to review adult (aged 18 to 64), older adult (aged 65 and older), and labor and delivery (regardless of age) patient records (see Exhibit C-1). Perinatal triggers (included below) were used only for labor and delivery patients. For pediatric patients, we used the OIG-modified GAPPS Trigger Tool to review records for patients up to and including 17 years of age at admission (see Exhibit C-2). Exhibit C-1: OIG-Modified Trigger Tool Worksheet for Adult and Labor and Delivery Patients Care Triggers Care Triggers (continued) C1 Acute mental status change C17 Any procedure complication C2 Transfusion or use of blood products C18 Urinary retention Code; cardiac or pulmonary arrest; or rapid C3 C19 New onset diarrhea response team activation C4 Positive culture C20 Prolonged constipation Studies for emboli, pulmonary embolus (PE), or deep vein thrombosis (DVT), such as D-Dimer, C5 C21 Care—other CT pulmonary angiogram (CTPA), or lung ventilation-perfusion scan C6 Death Intensive Care Unit Triggers C7 Drop in hemoglobin/hematocrit I1 Pneumonia onset C8 Patient fall or other trauma I2 Readmission to intensive care C9 Pressure injuries or other skin breakdown I3 In-unit procedure C10 Readmission within fiscal year 2017 I4 Intubation/reintubation C11 Restraint use I5 Intensive care unit—other C12 Healthcare-associated infections Medication Triggers Total white blood cells (WBC) <3000 (or C13 M1 Clostridioides difficile positive stool >12,000) C14 New or increased diuretics M2 Abnormal electrolytes Hospital stroke or transient ischemic attack C15 M3 Partial thromboplastin time >100 seconds (TIA) at IHS hospital C16 Transfer to higher level of care M4 International normalized ratio (INR) >6 Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix C | 37 Medication Triggers (continued) Perinatal Triggers (continued) Glucose < 50, glucagon or dextrose supplement M5 For newborn infants (and other pediatric P9 Instrumented delivery patients), glucose <40 mg/dl during first year of life; and <50 mg/dl after first year, glucagon or dextrose supplement Rising serum creatinine, decreasing urine M6 output, glomerular filtration rate (GFR), or acute P10 General anesthesia dialysis M7 Vitamin K administration (phytonadione) P11 Cord gases ordered M8 Diphenhydramine use P12 Gestational diabetes M9 Flumazenil use P13 Terbutaline administration Administration of uterotonic agents (such as M10 Naloxone use P14 methylergonovine, and 15-methyl-prostaglandin in the postpartum period) M11 Antiemetic administration P15 Corticosteroid administration Labetalol, hydralazine or nifedipine M12 Sodium polystyrene (kayexalate administration) P16 administration M13 Abrupt onset hypotension P17 Unplanned Caesarean section M14 Abnormal drug levels P18 Perinatal—other M15 Abrupt medication stop Surgical Triggers M16 Thrombocytopenia S1 Return to surgery M17 Use of traditional herbs, rituals, or botanicals S2 Change in surgical procedure Unplanned admission to intensive care post- M18 Medication—other S3 operation Intubation/reintubation/ Perinatal Triggers S4 bipap in post-anesthesia care unit (PACU) P1 Delivery prior to 39 weeks gestation S5 Unplanned X-ray intra-op or in PACU P2 Apgar <7 at 5 min. S6 Intra-op or post-op death Admission to neonatal intensive care unit P3 S7 Mechanical ventilation >24 hours post-op (NICU) >24 hours Intra-op epinephrine, norepinephrine, naloxone, P4 Transfer to a higher level of care S8 or flumazenil Abnormal postoperative troponin level, P5 3rd or 4th degree lacerations S9 including highly sensitive troponin T (hs-cTnT), above the upper limit of normal P6 Prolonged fetal heart rate decelerations S10 Removal, injury, or repair of organ P7 Platelet count <50,000 S11 Any operative complication P8 Specialty consult (Intentionally blank) Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix C | 38 Long-term/Non-acute Care Triggers Long-term/Non-acute Medication Triggers Antibiotics started while in long-term or non- LT-C1 Insertion or use of urinary catheter LT-M1 acute status Acute deterioration while in long-term or non- Starting or increasing pain medication needs LT-C2 LT-M2 acute status while in long-term or non-acute status Diagnostic radiology or imaging studies while Administration of parenteral fluid while in long- LT-C3 LT-M3 in long-term or non-acute status term or non-acute status Long-term/non-acute care medication module— LT-C4 Long-term/non-acute care—other LT-M4 other Exhibit C-2: OIG-Modified GAPPS Trigger Tool Medications/Fluids Triggers Hospital Care Environment Triggers (continued) Ped_M1 Warfarin triggers: INR >6 Ped_H5 Embolus/thrombus documentation Ped_M2 Serum creatinine doubling Ped_H6 Pediatric hospital care environment—other Nephrotoxin use (e.g., aminoglycosides, Ped_M3 cyclosporine, tacrolimus, vancomycin) and Healthcare-associated Infection Triggers doubling creatinine (Cr) Elevated drug levels (antiepileptics): Positive Clostridioides difficile test Ped_M4 phenytoin (>30 mcg/ml) or abnormally low Ped_I1 (>4th calendar day from admission) (<10 mcg/ml) Elevated drug levels (antiepileptics): Ped_M5 oxcarbazepine (>45 mcg/ml) or abnormally Ped_I2 Oral vancomycin low (<3 mcg/ml) Ped_M6 Total bilirubin >25 mg/dl (<28 days old) Ped_I3 Positive blood culture Hepatotoxic medications and elevated liver Ped_M7 Ped_I4 Positive urine culture enzymes (AST, ALT) >3 x normal Hypoglycemia < 40 mg/dl during the first Positive respiratory or gastrointestinal (GI) Ped_M8 year of life; <2 mmol/L or 50 mg/dl) after Ped_I5 viral infection (on or after the 3rd calendar day the first year of life from admission) Ped_M9 Abrupt medication stops Ped_I6 Surgical site infection Ped_M10 Flumazenil administration Ped_I7 Healthcare-associated infection module—other Opiate-related constipation with Ped_M11 Hospital Transfer/Outcomes Triggers intermittent laxative use Ped_M12 Naloxone administration Ped_T1 Readmission within fiscal year 2017 Any code or arrest, or rapid response team Ped_M13 Pediatric medication—other Ped_T2 activation Hospital Care Environment Triggers Ped_T3 All inpatient deaths Ped_H1 Patient fall Ped_T4 Hospital transfer/outcomes module – other Infiltrations: infiltration/extravasation or Ped_H2 NICU/PICU Triggers phlebitis documentation Readmission to ICU within 24 hours after Ped_H3 Infiltrations: hyaluronidase administration Ped_N1 discharge/transfer Ped_H4 Pressure injury documentation (≥ Stage 1) Ped_N2 Transfer to higher level of care Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix C | 39 NICU/PICU Triggers (continued) Surgical Triggers (continued) Ped_N3 Unplanned endotracheal extubating Ped_S7 Pediatric surgical module—other Failed endotracheal extubation Ped_N4 (reintubation within 24 hours of planned Long-term/Non-acute Care Triggers extubation) Racemic epinephrine administration Ped_N5 (patients mechanically ventilated within last LT-C1 Insertion or use of urinary catheter 24 hours) NICU/Pediatric intensive care unit (PICU) Acute deterioration while in long-term or non- Ped_N6 LT-C2 module—other acute status Surgical Triggers LT-C2 LT-C3 Drop of hemoglobin (Hgb) or hematocrit Ped_S1 LT-C4 Long-term/non-acute care—other (Hct) of >25% in less than 24 hours Mechanical ventilation >48 hours Ped_S2 Long-term/Non-acute Medication Triggers postoperatively Operative time >6 hours (non-cardiac Antibiotics started while in long-term or non- Ped_S3 LT-M1 patients) acute status Intraoperative epinephrine, norepinephrine, Starting or increasing pain medication needs Ped_S4 LT-M2 or phenylephrine (non-cardiac patients) while in long-term or non-acute status Administration of parenteral fluid while in Ped_S5 Return to surgery LT-M3 long-term or non-acute status Long-term/non-acute care medication Ped_S6 Change in procedure LT-M4 module—other Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix C | 40 APPENDIX D Estimates, Confidence Intervals, and Key Statistics The estimates included in this report are based on a sample of 384 patients admitted to IHS hospitals during FY 2017. We also provide estimates of incidence rates for IHS hospitals in the Great Plains Area, IHS hospitals in all other Areas, small IHS hospitals (those with less than 1,000 admissions in FY 2017), and large IHS hospitals (those with 1,000 or more admissions in FY 2017). These estimates were weighted by the six strata in our sample. However, given the small number of patient harm events, we were unable to provide reliably projectable estimates for each preventability classification, harm level, event category, and for certain event details. (See Appendix E for sample counts related to these analyses). See Exhibit D-1 for patient-level statistics for all of IHS and Exhibit D-2 for selected patient-level statistics for the hospitals in the Great Plains Area, hospitals outside the Great Plains Area, small hospitals, and large hospitals. Exhibit D-1: Patient-Level Estimates, Confidence Intervals, and Key Statistics for the Population of Patients in IHS Hospitals (n=384) 95-Percent 95-Percent Confidence Interval Confidence Interval* Estimated Sample Percentage Lower Upper Number of Lower Upper Estimate Description Size (n) of Patients Bound Bound Patients Bound Bound Events Experienced by All Patients At least one adverse or 384 12.6% 8.9% 17.5% 1,840 1,216 2,465 temporary harm event At least one adverse event 384 4.5%† 2.4% 8.2% 649 245 1,054 At least one temporary harm 384 9.4% 6.3% 13.7% 1,369 834 1,905 event An adverse event only 384 3.2% 1.5% 6.7% 471 121 821 A temporary harm event only 384 8.2% 5.3% 12.3% 1,191 692 1,690 An adverse event and 384 1.2% 0.4% 3.9% 178 -- 389 temporary harm event Preventable adverse events or 384 7.3% 4.6% 11.5% 1,070 577 1,563 temporary harm events Preventable adverse events 384 3.6% 1.8% 7.0% 525 168 882 Preventable temporary harm 384 4.6% 2.5% 8.3% 669 269 1,070 events Preventable temporary harm 384 3.7% 1.9% 7.0% 545 193 897 events only Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix D | 41 Continued from previous page Adverse events or temporary harm events resulting in 384 0.5% 0.2% 1.4% 71 -- 146 transfer A cascade adverse event or 384 1.9% 0.7% 5.1% 281 1 561 temporary harm event Source: OIG analysis of IHS hospital stays for 384 patients in FY 2017. *Given the small proportions, confidence intervals for projected totals exceed 30-percent relative precision. †This estimate rounds to 4 percent in the report because its value is 4.4502. Exhibit D-2: Patient-Level Estimates, Confidence Intervals, and Key Statistics for Patient Populations in the Great Plains Area, Outside the Great Plains Area, in Small Hospitals, and in Large Hospitals (n=384) 95-Percent 95-Percent Confidence Interval Confidence Interval* Estimated Sample Percentage Lower Upper Number of Lower Upper Estimate Description Size (n) of Patients Bound Bound Patients Bound Bound Events Experienced by Patients in Hospitals in the Great Plains Area At least one adverse or 187 14.4% 9.7% 21.0% 352 216 487 temporary harm event At least one adverse event 187 2.9% 1.0% 8.1% 70 -- 145 At least one temporary harm 187 12.6% 8.3% 18.7% 308 182 435 event An adverse event only 187 1.8% 0.5% 6.1% 44 -- 98 A temporary harm event only 187 11.5% 7.5% 17.2% 282 165 398 Preventable adverse events or 187 9.9% 5.9% 16.0% 242 122 362 temporary harm events Preventable adverse events 187 2.9% 1.0% 8.1% 70 -- 145 Preventable temporary harm 187 7.0% 3.9% 12.2% 172 75 269 events Preventable temporary harm 187 7.0% 3.9% 12.2% 172 75 269 events only Events Experienced by Patients in Hospitals Outside the Great Plains Area At least one adverse or 197 12.3% 8.1% 18.2% 1,489 879 2,098 temporary harm event At least one adverse event 197 4.8% 2.4% 9.3% 579 181 977 At least one temporary harm 197 8.7% 5.3% 14.1% 1,061 540 1,582 event A temporary harm event only 197 7.5%† 4.4% 12.6% 909 425 1,394 Preventable adverse events or 197 6.8% 3.8% 12.0% 828 350 1,306 temporary harm events Preventable adverse events 197 3.7% 1.7% 7.9% 455 106 804 Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix D | 42 Continued from previous page Preventable temporary harm 197 4.1% 1.9% 8.8% 497 109 886 events Preventable temporary harm 197 3.1% 1.2% 7.5% 373 35 711 events only Events Experienced by Patients in Small Hospitals‡ At least one adverse or 236 18.5% 13.5% 24.9% 708 491 925 temporary harm event At least one adverse event 236 4.7% 2.3% 9.4% 180 54 307 At least one temporary harm 236 15.2% 10.7% 21.2% 582 384 781 event A temporary harm event only 236 13.8% 9.6% 19.5% 528 340 716 Preventable adverse events or 236 9.2% 5.8% 14.3% 353 195 511 temporary harm events Preventable adverse events 236 4.0% 1.8% 8.4% 153 37 269 Preventable temporary harm 236 5.9% 3.4% 10.1% 227 105 349 events Preventable temporary harm 236 5.2% 3.0% 9.0% 200 89 311 events only Events Experienced by Patients in Large Hospitals‡ At least one adverse or 147 9.4% 5.3% 16.0% 1,008 451 1,566 temporary harm event At least one adverse event 147 4.1% 1.7% 9.5% 442 61 823 At least one temporary harm 147 6.4% 3.2% 12.3% 690 225 1,154 event A temporary harm event only 147 5.3% 2.5% 10.9% 566 141 991 Preventable adverse events or 147 5.5% 2.6% 11.1% 593 165 1,021 temporary harm events Preventable adverse events 147 3.2% 1.2% 8.3% 345 11 679 Preventable temporary harm 147 3.2% 1.2% 8.3% 345 11 679 events Preventable temporary harm 147 2.3% 0.7% 6.9% 248 -- 525 events only Source: OIG analysis of IHS hospital stays for 384 patients in FY 2017. *Given the small proportions, confidence intervals for projected totals exceed 30-percent relative precision. †This estimate rounds to 7 percent in the report because its value is 7.4882. ‡For one patient with two hospitalizations during the year, one admission was ineligible for this particular analysis because the patient did not receive acute-care services. As a result, the n here is 383 rather than 384 patients. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix D | 43 APPENDIX E Sample Numbers by Patient and Event Type per Stratum Our patient-level and event-level sample numbers and percentages are based on a sample of 384 patients admitted to IHS hospitals during FY 2017. We are unable to reliably project certain sample numbers, so we instead present evidence of the sample frequencies by patient and event type per the six strata (numbered from 1 to 6). See Exhibit 5 on page 11 for the patient population and sample sizes in each stratum. Patient type is distinguished by adult (aged 18 to 64), older adult (aged 65 and older), pediatric (those up to and including 17 years of age), and labor and delivery (regardless of age). Event type includes severity (on the OIG-modified NCC MERP Index), preventability, and clinical category (medication, patient care, infections, and procedures). Exhibit E-1 shows the number of patients in our sample, the number of patients who experienced harm events, and the number of events by patient type per stratum. Exhibit E-2 shows the number of events by patient type and clinical category. Exhibit E-3 shows the number of events by event type (e.g., severity, clinical category, and preventability) per stratum. Exhibit E-4 shows the number of events by clinical category and subcategory per stratum. Exhibit E-1: Number of Patients, Number of Patients Who Experienced Harm Events, and Number of Events by Patient Type per Stratum (n=384) Stratum Patient Type Total (1) (2) (3) (4) (5) (6) Patients in Sample Adults* 194 27 16 22 38 49 42 Older Adults 61 1 7 7 9 26 11 Labor and Delivery 48 11 9 4 0 6 18 Pediatric 81 11 8 7 10 19 26 Total Patients 384 50 40 40 57 100 97 Patients Who Experienced Harm Events Adults 29 5 3 3 7 8 3 Older Adults 18 0 1 2 2 10 3 Labor and Delivery 10 0 4 1 0 1 4 Pediatric 4 0 0 1 3 0 0 Total Patients Who 61 5 8 7 12 19 10 Experienced Harm Events Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix E | 44 Continued from previous page Events in Sample Adults 34 8 5 3 7 8 3 Older Adults 29 0 2 2 4 13 8 Labor and Delivery 12 0 4 1 0 1 6 Pediatric 4 0 0 1 3 0 0 Total Events 79 8 11 7 14 22 17 Source: OIG analysis of IHS hospital stays for 384 patients in FY 2017. *We do not count as adults two patients who had initial stays as labor and delivery patients and subsequent stays as adult patients. Exhibit E-2: Number of Events by Patient Type and Clinical Category (n=384) Clinical Category Total Patient Type Events Medication Patient Care Infections Procedures Events in Sample Adults 34 22 10 1 1 Older Adults 29 15 9 4 1 Labor and Delivery 12 3 7 1 1 Pediatric 4 1 3 0 0 Total Events 79 41 29 6 3 Source: OIG analysis of IHS hospital stays for 384 patients in FY 2017. Exhibit E-3: Number of Events by Event Type per Stratum (n=384) Stratum Total Event-level Description Events (1) (2) (3) (4) (5) (6) Events in Sample Adverse and temporary harm events 79 8 11 7 14 22 17 Adverse events 19 3 0 1 2 7 6 Temporary harm events (E-level harm) 60 5 11 6 12 15 11 Cascade temporary or adverse events 7 1 2 0 1 1 2 Transfer to an acute-care hospital because of an 5 1 0 1 2 1 0 adverse or temporary harm event Severity Level on NCC-MERP for Adverse Events F-level harm—An event that resulted in a prolonged stay or became primary reason for 13 2 0 1 2 5 3 treatment G-level harm—An event that contributed to or 1 0 0 0 0 0 1 resulted in permanent patient harm Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix E | 45 Continued from previous page H-level harm—An event that required 4 1 0 0 0 2 1 intervention to sustain the patient’s life I-level harm—An event that contributed to 1 0 0 0 0 0 1 death Clinical Category for Adverse Events and Temporary Harm Events Medication-related adverse and temporary 41 7 5 5 8 7 9 harm events Patient-care-related adverse and temporary 29 0 6 2 6 11 4 harm events Infection-related adverse and temporary harm 6 1 0 0 0 2 3 events Procedure-related adverse and temporary harm 3 0 0 0 0 2 1 events Preventability Classification for Adverse Events and Temporary Harm Events Adverse and temporary harm events that were 38 5 6 4 5 10 8 preventable Adverse and temporary harm events that were 40 3 5 3 9 11 9 not preventable Adverse and temporary harm events—unable to 1 0 0 0 0 1 0 determine Adverse events that were preventable 15 3 0 1 2 6 3 Adverse events that were not preventable 4 0 0 0 0 1 3 Temporary harm events that were preventable 23 2 6 3 3 4 5 Temporary harm events that were not 36 3 5 3 9 10 6 preventable Source: OIG analysis of IHS hospital stays for 384 patients in FY 2017. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix E | 46 Exhibit E-4: Number of Events by Clinical Category and Subcategory (n=384) Adverse Events and Temporary Total Stratum Harm Events by Clinical Category Events (1) (2) (3) (4) (5) (6) Medication 41 7 5 5 8 7 9 Delirium/change in mental status 12 1 2 1 3 2 3 Hypoglycemia 7 1 1 1 1 2 1 Hypotension/hypertension 6 2 1 1 0 1 1 Allergic reaction 3 0 0 1 2 0 0 Diarrhea 3 2 0 1 0 0 0 Tachysystole 3 0 1 0 0 0 2 Nausea and vomiting 2 0 0 0 0 2 0 Cardiac arrhythmia 1 0 0 0 0 0 1 Excessive bleeding 1 0 0 0 1 0 0 Interstitial nephritis 1 0 0 0 0 0 1 Prolonged constipation 1 1 0 0 0 0 0 Seizure 1 0 0 0 1 0 0 Patient Care 29 0 6 2 6 11 4 Infiltration from intravenous catheter 9 0 3 0 3 3 0 Postpartum hemorrhage 4 0 1 1 0 0 2 Pressure injury 3 0 0 0 1 2 0 Skin tear/abrasion 3 0 0 0 0 3 0 Fluid/electrolyte disorders 2 0 0 1 0 1 0 Hypotension 2 0 2 0 0 0 0 Allergic reaction 1 0 0 0 1 0 0 Diabetic ketoacidosis 1 0 0 0 0 0 1 Exacerbation of abnormal liver condition 1 0 0 0 1 0 0 Fall 1 0 0 0 0 1 0 Oxygen desaturation 1 0 0 0 0 1 0 Hypertension/preeclampsia 1 0 0 0 0 0 1 Infections 6 1 0 0 0 2 3 Soft tissue infections 4 1 0 0 0 2 1 Surgical site infections 2 0 0 0 0 0 2 Procedures 3 0 0 0 0 2 1 Atrial fibrillation 1 0 0 0 0 1 0 Laceration 1 0 0 0 0 1 0 Urinary retention 1 0 0 0 0 0 1 Source: OIG analysis of IHS hospital stays for 384 patients in FY 2017. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix E | 47 APPENDIX F Comparing Rates of Adverse Events and Temporary Harm Events to Other Health Care Settings We are unable to conduct a reliable comparison between the incidence rates found in this study and the 27 percent rate of adverse and temporary harm events found in the prior OIG study of adverse events among hospitalized Medicare patients. Many factors can affect harm rates other than the care provided. Other factors to consider include: › Differences in the population of patients reviewed—Prior OIG studies included only Medicare beneficiaries (typically age 65 and older) who are potentially more susceptible to harm; this study, in contrast, spanned the full range of patient ages, including children and young adults who may be less susceptible to harm. › Differences in the type of care provided—IHS hospitals typically provide less complex care than other hospitals across the nation, resulting in fewer opportunities for potential harm. For example, a patient requiring complex surgery may be transferred out of the IHS system for the surgery. › Differences in average length of stay—Hospitalized IHS patients have shorter average lengths of stay than patients at other hospitals, further reducing opportunities for harm. To account for the last factor (length of stay), hospitals commonly measure adverse events by incidence density, which is the number of events by patient days or by hospital admissions to adjust for the period during which patients are observed. For example, incidence density is often used in measuring healthcare-acquired infections because risk can increase with the length of exposure to the healthcare environment. 115 The Institute for Healthcare Improvement (IHI), a nonprofit advisory group to hospitals, cites advantages to using incidence density metrics over standard incidence rates that measure the percent of admissions with adverse events. 116 IHI reports that measuring total events by patient days or hospital admissions also enables hospitals to count multiple events experienced by the same patient. The sample of 384 IHS hospital patients that were admitted to an IHS hospital in FY 2017 included 457 total hospital stays (admissions) and a total of 1,587 days in the hospital (patient days). We calculated patient days by subtracting the admission date for each IHS hospital stay from its discharge date. Using the incidence density measure, we found 46 patient harm events per 1,000 patient days in IHS hospitals. Exhibit F-1 on the next page provides the estimated ratios for adverse and temporary harm events per 1,000 patient days and per 100 admissions. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix F | 48 Exhibit F-1: Rates of Adverse Events and Temporary Harm Events in the Sample by Patient Days and Hospital Admissions (n=384) 95-Percent 95-Percent Confidence Interval Confidence Interval Per 1,000 Lower Upper Per 100 Lower Upper Category Patient Days Bound Bound Admissions Bound Bound Adverse events 14.6 6.8 22.4 5.6 2.2 9.1 Temporary harm events 31.2 20.7 41.8 10.7 6.3 15.1 Adverse events and temporary harm events 45.8 31.2 60.4 16.4 10.5 22.3 combined Source: Office of Inspector General Analysis of IHS hospital stays for 384 patients in FY 2017. Using incidence densities from prior OIG work as benchmarks, IHS hospitals appear to have fewer patient harm events per 1,000 patient days than other acute-care hospitals, but more than post-acute-care settings (such as nursing homes), which typically provide less complex care. However, a direct comparison of the incidence density in IHS to incidence density in other acute-care hospital settings is not appropriate given the time gap in measurements and differences in the length of the review period (1 year versus 1 month). As a result, we only present these prior rates for reference and avoid statistical comparisons. Exhibit F-2 shows the rates of harm events per 1,000 patient days for acute-care hospitals, IHS hospitals, long-term care hospitals, rehabilitation hospitals, and skilled nursing facilities. Exhibit F-2: Rates of Adverse Events and Temporary Harm Events per 1,000 Patient Days Across Health Care Settings 95-Percent Confidence Interval Healthcare Setting Sample Size (n) Per 1,000 Patient Days Lower Bound Upper Bound Acute-care hospitals 780 69.4 61.2 77.5 IHS hospitals 384 45.8 31.2 60.4 Long-term care hospitals 587 38.0 33.9 42.1 Rehabilitation hospitals 417 29.3 24.3 34.3 Skilled nursing facilities 653 24.3 20.4 28.1 Sources: Office of Inspector General analysis of IHS hospital stays for 384 patients in FY 2017, long-term care hospital stays for 587 Medicare beneficiaries in March 2014, rehabilitation hospital stays for 417 Medicare beneficiaries in March 2012, skilled nursing facility stays for 653 Medicare beneficiaries in August 2011, and acute-care hospital stays for 780 Medicare beneficiaries in October 2008. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix F | 49 APPENDIX G Agency Comments Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix G | 50 Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix G | 51 Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Appendix G | 52 ACKNOWLEDGMENTS AND CONTACT Acknowledgments Jesse Valente and Jennifer Hagen served as team leaders for this study and Demetrius Martinez served as lead analyst. Office of Evaluation and Inspections staff who provided support include Joe Chiarenzelli, Althea Hosein, Christine Moritz, and Troy Yamaguchi. This report was prepared under the direction of Ruth Ann Dorrill, Regional Inspector General for Evaluation and Inspections in the Dallas regional office, and Amy Ashcraft, Deputy Regional Inspector General. Contact To obtain additional information concerning this report, contact the Office of Public Affairs at Public.Affairs@oig.hhs.gov. OIG reports and other information can be found on the OIG website at oig.hhs.gov. Office of Inspector General U.S. Department of Health and Human Services 330 Independence Avenues, SW Washington, DC 20201 Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Acknowledgments and Contact | 53 ABOUT THE OFFICE OF INSPECTOR GENERAL The mission of the Office of Inspector General (OIG), as mandated by Public Law 95- 452, as amended, is to protect the integrity of the Department of Health and Human Services (HHS) programs, as well as the health and welfare of beneficiaries served by those programs. This statutory mission is carried out through a nationwide network of audits, investigations, and inspections conducted by the following operating components: The Office of Audit Services (OAS) provides auditing services for HHS, either by conducting audits with its own audit resources or by overseeing audit work done by others. Audits examine the performance of HHS programs and/or its grantees and contractors in carrying out their respective responsibilities and are intended to provide independent assessments of HHS programs and operations. These audits help reduce waste, abuse, and mismanagement and promote economy and efficiency throughout HHS. The Office of Evaluation and Inspections (OEI) conducts national evaluations to provide HHS, Congress, and the public with timely, useful, and reliable information on significant issues. These evaluations focus on preventing fraud, waste, or abuse and promoting economy, efficiency, and effectiveness of departmental programs. To promote impact, OEI reports also present practical recommendations for improving program operations. The Office of Investigations (OI) conducts criminal, civil, and administrative investigations of fraud and misconduct related to HHS programs, operations, and beneficiaries. With investigators working in all 50 States and the District of Columbia, OI utilizes its resources by actively coordinating with the Department of Justice and other Federal, State, and local law enforcement authorities. The investigative efforts of OI often lead to criminal convictions, administrative sanctions, and/or civil monetary penalties. The Office of Counsel to the Inspector General (OCIG) provides general legal services to OIG, rendering advice and opinions on HHS programs and operations and providing all legal support for OIG’s internal operations. OCIG represents OIG in all civil and administrative fraud and abuse cases involving HHS programs, including False Claims Act, program exclusion, and civil monetary penalty cases. In connection with these cases, OCIG also negotiates and monitors corporate integrity agreements. OCIG renders advisory opinions, issues compliance program guidance, publishes fraud alerts, and provides other guidance to the health care industry concerning the anti-kickback statute and other OIG enforcement authorities. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 About the OIG | 54 ENDNOTES 1 Wachter, R.M., Understanding Patient Safety, 2nd edition, McGraw-Hill, 2012, p.15. 2Griffin, F.A., and Resar, R.K., IHI Global Trigger Tool for Measuring Adverse Events (Second Edition), Institute for Healthcare Improvement Innovation Series 2009, p. 5. 3CMS, “AHRQ Common Formats - Information for Hospitals and State Survey Agencies (SAs) - Comprehensive Patient Safety Reporting Using AHRQ Common Formats,” S&C: 13-19-Hospitals, March 15, 2013. Accessed at https://www.cms.gov/Medicare/Provider-Enrollment-and-Certification/SurveyCertificationGenInfo/Downloads/Survey-and- Cert-Letter-13-19.pdf on December 14, 2018. 4The Institute of Medicine is now the Health and Medicine Division of the National Academies. See Kohn, et al., To Err Is Human: Building a Safer Health System, A Report of the Committee on Quality of Health Care in America, Institute of Medicine, National Academy Press, 2000. 5 Ibid. 6Brennan, et al., “Incidence of Adverse Events and Negligence in Hospitalized Patients Results of the Harvard Medical Practice Study I,” The New England Journal of Medicine, Volume 324, Number 6, February 1991. 7 Thomas, et al., “Incidence and Types of Adverse Events and Negligent Care in Utah and Colorado,” Medical Care, Volume 38, Issue 3, 2000. 8 OIG released 13 reports on adverse events in hospitals from 2008–2018, including reports about the incidence of adverse events, methods for identifying adverse events, State reporting systems, and public disclosure of event information. All reports are available at https://oig.hhs.gov. 9 OIG, Adverse Events in Hospitals: National Incidence Among Medicare Beneficiaries, OEI-06-09-00090, November 2010. 10 For more information on the GTT method used by the OIG, see pages 28 to 29 of this report. See also Griffin, F.A. and Resar, R.K., op. cit. 11 OIG, Hospital Incident Reporting Systems Do Not Capture Most Patient Harm, OEI-06-09-00091, January 2012. 12 OIG, Adverse Events in Hospitals: National Incidence Among Medicare Beneficiaries, OEI-06-09-00090, November 2010, op. cit. 13 Landrigan,et al., “Temporal Trends in Rates of Patient Harm Resulting from Medical Care,” The New England Journal of Medicine, Volume 363, Number 22, November 2010. 14 Classen, et al., “‘Global Trigger Tool’ Shows That Adverse Events In Hospitals May Be Ten Times Greater Than Previously Measured,” Health Affairs, Volume 30, Number 4, April 2011. 15 Kirkendall,et al., “Measuring Adverse Events and Levels of Harm in Pediatric Inpatients With the Global Trigger Tool,” Pediatrics, Volume 130, Number 5, November 2012. 16 Kennerly,et al., “Characterization of Adverse Events Detected in a Large Health Care Delivery System Using an Enhanced Global Trigger Tool over a Five-Year Interval,” Health Services Research, Volume 49, Number 5, October 2014. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Endnotes | 55 17 Adler, et al., “Impact of Inpatient Harms on Hospital Finances and Patient Clinical Outcomes,” Journal of Patient Safety, Volume 14, Number 2, June 2018. 18Griffey, et al., “Multicenter Test of an Emergency Department Trigger Tool for Detecting Adverse Events,” Journal of Patient Safety, published ahead of print on July 2018. 19 Stockwell, et al., “Adverse Events in Hospitalized Pediatric Patients,” Pediatrics, Volume 142, number 2, August 2018. 20OIG, Indian Health Service Hospitals: More Monitoring Needed to Ensure Quality Care (OEI-06-14-00010) and Indian Health Service Hospitals: Longstanding Challenges Warrant Focused Attention to Support Quality Care (OEI-06-14-00011). OIG, Case Study: Indian Health Service Management of Rosebud Hospital Emergency Department Closure and Reopening, 21 OEI-06-17-00270. 22 OIG, Organizational Challenges to Improving Quality of Care in IHS Hospitals, OEI-06-16-00390, August 2019. 23U.S. House of Representatives Committee on Appropriations, Management Challenges at the Departments of Labor, Health and Human Services, and Education and the Social Security Administration: Views from the Inspectors General, March 9, 2017. Accessed at https://www.govinfo.gov/content/pkg/CHRG-115hhrg27208/pdf/CHRG-115hhrg27208.pdf on February 11, 2019. U.S. Senate Committee on Indian Affairs, Reexamining the Substandard Quality of Indian Health Care in the Great Plains, February 3, 2016. Accessed at https://www.indian.senate.gov/hearing/oversight-hearing-reexamining-substandard-quality- indian-health-care-great-plains on February 11, 2019. GAO, Indian Health Service: Actions Needed to Improve Oversight of Quality of Care, January 2017. Accessed at https://www.gao.gov/assets/690/681951.pdf on February 11, 2019. 24 IHS, Justification of Estimates for Appropriations Committees, FY 2021, p. CJ-8. Accessed at https://www.ihs.gov/sites/budgetformulation/themes/responsive2017/display_objects/documents/FY_2021_Final_CJ-IHS.pdf on June 24, 2020. 25 IHS, “Agency Overview.” Accessed at https://www.ihs.gov/aboutihs/overview/ on July 25, 2019. 26 IHS, Justification of Estimates for Appropriations Committees, FY 2021, op. cit., pp. CJ-1 and CJ-2. 27 IHS, “Locations.” Accessed at https://www.ihs.gov/locations/ on November 14, 2019. 28 IHS, “IHS Profile,” April 2020. Accessed at https://www.ihs.gov/newsroom/factsheets/ihsprofile/ on June 26, 2020. 29Under the Indian Self-Determination and Education Assistance Act, P.L. No. 93-638, IHS contracts and compacts with Tribes or Tribal organizations to deliver services. 30 IHS, Justification of Estimates for Appropriations Committees, FY 2021, op. cit., p. CJ-8. 31 Ibid., p. CJ-68. 32Social Security Act §§ 1880(a) and 1911(a), P.L. No. 94-437, Indian Health Care Improvement Act, as amended (codified at 25 U.S.C. §1621e). 33 IHS, Justification of Estimates for Appropriations Committees, FY 2021, op. cit., p. CJ-8. 34IHS currently operates 24 hospitals. Tribes took over management and operation of Winnebago Hospital in July 2018 after the sample was selected and the Rapid City Hospital converted from a hospital to a health center in 2019. See https://www.ihs.gov/locations/ for a list of IHS facilities and hospitals. Critical access hospitals are designated by CMS to provide critical services to rural communities. To qualify, these facilities 35 must meet certain conditions. For more information, see IHS, “Critical Access Hospitals,” Accessed at Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Endnotes | 56 https://www.ihs.gov/sites/newsroom/themes/responsive2017/display_objects/documents/factsheets/CriticalAccessHospitals.pd f on March 9, 2020. 36 IHS,“Inpatient Summary Data for Indian Health Service Hospitals By Area and Facility – October through September Fiscal 2017,” internal memorandum, April 9, 2018. 37 The seven hospitals are located in: Great Plains (1), Navajo (3), Oklahoma (1), and Phoenix (2). 38 IHS, “Overview of the Indian Health Service” presentation by the IHS Chief Medical Officer, August 3, 2018. OIG, Indian Health Service Hospitals: Longstanding Challenges Warrant Focused Attention to Support Quality Care, 39 OEI-06-14-00011, October 2016, op. cit. 40Care provided by non-IHS providers is funded through the Purchased/Referred Care (PRC) program. IHS, “Purchased/Referred Care (PRC).” Accessed at https://www.ihs.gov/prc/ on July 25, 2019. 41IHS, “First IHS facility designated as a Level III Trauma Center,” July 26, 2013. Accessed at https://www.ihs.gov/newsroom/pressreleases/2013pressreleases/firstihslevel3traumacenter/ on January 23, 2019. 42GAO, Indian Health Service Agency Faces Ongoing Challenges Filling Provider Vacancies, GAO-18-580, August 2018. Accessed at https://www.gao.gov/assets/700/693940.pdf on January 4, 2019. 43 HRSA,“Health Professional Shortage Areas (HPSAs),” last modified May 2019. Accessed at https://bhw.hrsa.gov/shortage- designation/hpsas on December 20, 2019. 44 GAO, Indian Health Service Agency Faces Ongoing Challenges Filling Provider Vacancies, op. cit., pp. 11 and 13. 45OIG, Indian Health Service Hospitals: Longstanding Challenges Warrant Focused Attention, OEI-06-14-00011, October 2016, op. cit. 46 Ibid. 47 OIG, Case Study: Indian Health Service Management of Rosebud Hospital Emergency Department Closure and Reopening, OEI-06-17-00270, op. cit. 48 Ibid. 49 HRSA,“American Indian/Alaska Native Health,” last modified April 2017. Accessed at https://www.hrsa.gov/public- health/community/indian-health/shortage.html on December 20, 2019. 50IHS, Indian Health Disparities, April 2018. Accessed at https://www.ihs.gov/newsroom/factsheets/disparities/ on July 25, 2019. 51 Ibid. 52 Boccuti, C., Swoope, C., and Artiga, S., The Role of Medicare and the Indian Health Service for American Indians and Alaska Natives Health, Access and Coverage, Kaiser Family Foundation, Publication No. 8676, 2014. 53 Ibid. 54 Goins,R.T., et al., Lifelong Disparities among Older American Indians and Alaska Natives, American Association of Retired Persons (AARP) Public Policy Institute, Research Report 2015-08, November 2015. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Endnotes | 57 55 Ahospital’s case mix index is an indicator of “the clinical complexity and resource consumption of the population of patients within a hospital.” A higher number generally indicates greater clinical complexity and resource intensity within a hospital. The case mix index is calculated from Medicare Severity Diagnosis Related Group (MS-DRG) codes and represents the average MS-DRG weight for a hospital. For more information about case mix index, see C.M. Mendez, et al., “Impact of Hospital Variables on Case Mix Index as a Marker of Disease Severity,” Population Health Management, February 2014, Vol. 17(1), pp. 28–34, DOI:10.1089/pop.2013.0002. 56IHS, “IHS Profile,” April 2020. Accessed at https://www.ihs.gov/newsroom/factsheets/ihsprofile/ on June 26, 2020 and email correspondence between IHS and OIG on July 18, 2019. 57 Social Security Act §§ 1861(e) and 1905(a), and 42 CFR Part 482. 58 These include CMS’s hospital-acquired condition payment policies, see CMS, “HAC Regulations and Notices.” Accessed at https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/HospitalAcqCond/HAC-Regulations-and-Notices on June 26, 2020. CMS also aims to improve care with 16 initiatives, see CMS, “Our 16 Strategic Initiatives.” Accessed at https://www.cms.gov/about-cms/story-page/our-16-strategic-initiatives.html on October 28, 2019. 59 Social Security Act § 1153 (codified at 42 U.S.C. § 1320c–2) authorizes the Secretary to contract with QIOs. 60 CMS,“QIO Overview,” last modified September 2018. Accessed at https://www.cms.gov/Medicare/Quality-Initiatives-Patient- Assessment-Instruments/QualityImprovementOrgs/index.html on November 1, 2019. 61CMS, “About the Partnership.” Accessed at https://partnershipforpatients.cms.gov/about-the- partnership/aboutthepartnershipforpatients.html on January 7, 2019. 62 CMS integrated the work of the Partnership for Patients’ Hospital Engagement Network (HEN) with the QIN-QIO program to initiate HIINs in 2016, awarding 16 HIIN contracts covering more than 4,000 hospitals. See https://partnershipforpatients.cms.gov/about-the-partnership/hospital-engagement- networks/thehospitalengagementnetworks.html accessed on November 1, 2019. 63 CMS, “Hospital Improvement Innovation Networks.” Available at https://partnershipforpatients.cms.gov/about-the- partnership/hospital-engagement-networks/thehospitalengagementnetworks.html. Accessed February 7, 2019. 64 AHRQ, “About AHRQ.” Accessed at https://www.ahrq.gov/cpi/about/index.html on October 28, 2019. 65 AHRQ, “Common Formats.” Accessed at https://www.pso.ahrq.gov/common on October 30, 2019. 66 AHRQ, “Patient Safety Organizations Program.” Accessed at https://www.ahrq.gov/cpi/about/otherwebsites/pso.ahrq.gov/index.html on October 28, 2019. 67 PatientSafety & Quality Improvement Act of 2005 (P. L. 109-41 enacted July 19, 2005); 42 CFR pt. 3. HHS promulgated the Patient Safety and Quality Improvement Act of 2005 Final Rule to implement the Patient Safety and Quality Improvement Act. 68IHS, Circular No. 97-01. Accessed at http://www.ihs.gov/ihm/index.cfm?module=dsp_ihm_circ_main&circ=ihm_circ_9701 on January 4, 2019. 69 Social Security Act §§ 1880(a) and 1865 (a)(1). 70 42 CFR § 482.21. 71 42 CFR § 482.21(c)(2). Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Endnotes | 58 72IHS is replacing WebCident with a new incident reporting system. See Adverse Event Reporting System - Web Based COTS, Solicitation No. 18-236-SOL-00026. Accessed at https://www.fbo.gov/index.php?s=opportunity&mode=form&id=61f8aec7ed164041b4f993d658763679&tab=core&_cview=0 on July 18, 2019. 73 OIG, Hospital Incident Reporting Systems Do Not Capture Most Patient Harm, OEI-06-09-00091, January 2012, op. cit. 74GAO, Indian Health Service Actions Needed to Improve Oversight of Quality of Care, GAO-17-181, January 2017. Accessed at https://www.gao.gov/assets/690/681951.pdf on January 4, 2019. OIG, IHS Has Strengthened Patient Abuse Policies, but Must Fully Integrate Them into Practice and Organizational Culture, 75 OEI-06-19-00330, December 2019. 76Gilbert, J., “HHS’ Indian Health Service awards 5-year contract for Adverse Event Reporting System,” FedHealthIT, December 13, 2018. Accessed at https://www.fedhealthit.com/2018/12/hhs-indian-health-service-awards-5-year-contract-for-adverse- event-reporting-system/ on May 13, 2019. 77 IHS, “IHS Safety Tracking & Response (I-STAR) Update,” June 2020. 78 Ibid. 79CMS, “IHS and CMS partnership to strengthen hospital care quality,” May 13, 2016. Accessed at https://www.ihs.gov/newsroom/pressreleases/2016pressreleases/ihs-and-cms-partnership-to-strengthen-hospital-care- quality/ on January 7, 2019. 80CMS, “CMS and Indian Health Service expand collaboration to improve health care in hospitals,” November 10, 2016. Accessed at https://www.ihs.gov/newsroom/pressreleases/2016pressreleases/cms-and-indian-health-service-expand- collaboration-to-improve-health-care-in-hospitals/ on January 7, 2019 81 CMS Final Management Decision to OIG, February 21, 2017. 82 IHS,“Partnership to Advance Tribal Health (PATH),” October 2018. Accessed at https://www.ihs.gov/sites/quality/themes/responsive2017/display_objects/documents/FactSheet_PATH_October2018.pdf on October 22, 2019. 83 IHS, “Quality at IHS.” Accessed at https://www.ihs.gov/quality/ on January 7, 2019. 84 IHS, Strategic Plan FY 2019-2023, lasted updated February 27, 2019. 85IHS, “National Accountability Dashboard for Quality,” February 2018. Accessed at https://www.ihs.gov/newsroom/factsheets/nationalqualityaccountability/ on January 7, 2019. IHS, “Patient Experience of Care Survey,” October 2017. Accessed at https://www.ihs.gov/newsroom/factsheets/patientexperienceofcaresurvey/ on January 7, 2019. 86IHS, “Indian Health Service Announces New Office of Quality,” December 21, 2018. Accessed at https://www.ihs.gov/newsroom/pressreleases/2018pressrAeleases/indian-health-service-announces-new-office-of-quality/ on January 7, 2019. 87 GAO, High-Risk Series Substantial Efforts Needed to Achieve Greater Progress on High-Risk Areas, GAO-19-157SP, March 2019, p. 130. Accessed at https://www.gao.gov/assets/700/697245.pdf on March 20, 2019. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Endnotes | 59 88Tribal Self-Governance Advisory Committee, “2020 Tribal Self-Governance Consultation Conference,” January 2020. Accessed at https://www.tribalselfgov.org/wp-content/uploads/2020/01/TSGAC-packet-January-2020.pdf on July 10, 2020. 89 IHS, “Stand Up of IHS Compliance Program,” internal email memorandum, January 21, 2020. 90U.S. Senate Committee on Indian Affairs, Reexamining the Substandard Quality of Indian Health Care in the Great Plains, February 3, 2016. Accessed at https://www.indian.senate.gov/hearing/oversight-hearing-reexamining-substandard-quality- indian-healthcare-great-plains on June 28, 2019. 91 U.S. Senate Committee on Indian Affairs, Oversight Hearing on the President's FY 2019 Budget Request for Indian Programs, Senate, 115th Congress (2018), April 11, 2018. Accessed at https://www.indian.senate.gov/hearing/oversight-hearing- presidents-fy2019-budget-request-indian-programs-0 on August 19, 2019. 92 CMS terminated Winnebago Hospital in 2015 and Pine Ridge Hospital in 2017. 93 Rosebud Hospital completed a Systems Improvement Agreement in 2018. 94 IHS,Indian Health Service Report on Proposed Closure of the Sioux San Hospital Emergency and Inpatient Departments, July 2017. Accessed at https://www.ihs.gov/sites/newsroom/themes/responsive2017/display_objects/documents/ReportToCongress_SiouxSanHospital _07282017.pdf on November 4, 2019. 95NCC MERP, Index for Categorizing Medication Errors, 2001. Accessed at http://www.nccmerp.org/sites/default/files/indexColor2001-06-12.pdf on September 11, 2017. 96 Griffin, F.A., op. cit., p 5. 97 OIG, Adverse Events in Hospitals: Case Study of Incidence Among Medicare Beneficiaries in Two Counties, OEI-06-08-00220, December 2008. 98 The IHS National Data Warehouse gathers, stores, reports, and allows easy access to accurate historical data. For more information, see IHS, “National Data Warehouse (NDW).” Accessed at https://www.ihs.gov/ndw/ on December 19, 2019. 99 Three patients had stays in more than one IHS hospital across strata during FY 2017. 100OIG, Adverse Events in Skilled Nursing Facilities: National Incidence Among Medicare Beneficiaries, OEI-06-11-00370, February 2014, p. 35. 101 See OIG, Adverse Events in Hospitals: National Incidence Among Medicare Beneficiaries, OEI-06-09-00090, November 2010. 102 See endnotes 12 to 19. 103 The incidence rate of adverse or temporary harm events at IHS hospitals in GPA was not statistically different from the rate at IHS hospitals outside GPA (p=.5422). 104 The incidence rate of adverse or temporary harm events at smaller IHS hospitals is significantly higher than the rate at larger IHS hospitals (p=.0094). In addition, the rate of temporary harm events only is significantly higher at smaller hospitals than larger hospitals (p=.0036). 105OIG, Indian Health Service Hospitals: Longstanding Challenges Warrant Focused Attention, OEI-06-14-00011, October 2016, op. cit. 106 OIG, Adverse Events in Hospitals: National Incidence Among Medicare Beneficiaries, OEI-06-09-00090, November 2010. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Endnotes | 60 OIG, Instances of IHS Labor and Delivery Care Not Following National Clinical Guidelines or Best Practices, OEI-06-19-00190, 107 December 2020. 108 Centers for Disease Control and Prevention, Reproductive Health: Data on Selected Pregnancy Complications in the United States. Accessed at https://www.cdc.gov/reproductivehealth/maternalinfanthealth/pregnancy-complications-data.htm#post on July 12, 2019; Callaghan, W.M., et al., “Trends in postpartum hemorrhage: United States, 1994-2006,” American Journal of Obstetrics and Gynecology, vol. 202, issue 4, 2010. 109 Mayo Clinic, “Inducing labor: When to wait, when to induce.” Accessed at https://www.mayoclinic.org/healthy- lifestyle/labor-and-delivery/in-depth/inducing-labor/art-20047557 on May 1, 2020. 110 OIG, Indian Health Service Has Strengthened Patient Abuse Policies, but Must Fully Integrate Them into Practice and Organizational Culture, OEI-06-19-00330, December 2019, op. cit. 111 Global Assessment of Pediatric Patient Safety (GAPPS): A Pediatric Trigger Tool for Measuring Adverse Events, Manual of Operations. Boston, MA: Center of Excellence for Pediatric Quality Measurement; February 2016. 112 Ibid. 113We have an “other” option for each preventability determination category. (There are 22 total without the “other” category, 25 total with the “other” category for each preventability determination, and 23 total with the “other’ category only once.) 114 One exception is for pediatric patients where multiple similar event types were counted as separate events. 115K.M. Arias, Outbreak Investigation, Prevention, and Control in Health Care Settings: Critical Issues in Patient Safety (Second Edition), Jones and Bartlett Publishers, 2009. 116 Griffin, F.A., op. cit., pp. 13 and 14. Incidence of Adverse Events in Indian Health Service Hospitals OEI-06-17-00530 Endnotes | 61