1 U.S. Department of Health and Human Services Office of Inspector General Patient Safety Organizations: Hospital Participation, Value, and Challenges OEI-01-17-00420 Suzanne Murrin September 2019 Deputy Inspector General for Evaluation and Inspections oig.hhs.gov 1 Report in Brief U.S. Department of Health and Human Services September 2019 OEI-01-17-00420 Office of Inspector General Patient Safety Organizations: Hospital Why OIG Did This Review Participation, Value, and Challenges Researchers have estimated that over The Agency for Healthcare Research and 200,000 people die each year because Quality’s (AHRQ’s) voluntary Patient Safety Key Takeaway of medical errors in hospitals. Organization (PSO) program is the first and Many hospitals that Learning from those and other, only nationwide program that offers legal participate in the Patient nonfatal events to improve patient protections for providers to disclose and Safety Organization program safety is the goal of the PSO program. learn from patient safety events. An find that it has improved Hospitals’ descriptions of their organization must meet AHRQ’s criteria to patient safety. However, experiences with the program provide be federally listed as a PSO. The program is challenges have slowed insight into the program’s progress also the only program to establish a progress toward a national toward facilitating national learning Network of Patient Safety Databases (NPSD) system of learning to from patient safety events. This to enable learning on a national scale about improve patient safety. review is the first to explore the extent the causes of such events. to which hospitals participate in the PSO program and their perspectives What OIG Found on its values and challenges. It builds Over half of general acute-care hospitals work with a PSO, and nearly all of them on previous Office of Inspector find it valuable. Among hospitals that work with a PSO, 80 percent find that General work from 2010 that found the PSO’s feedback and analysis on patient safety events have helped prevent 27 percent of hospitalized Medicare future patient safety events. beneficiaries experienced harm because of medical care. OIG However, the PSO program faces challenges. Hospitals that do not participate recommended, among other things, do not perceive the PSO program to be distinct from other patient safety that AHRQ encourage hospitals to efforts. Nearly all of these hospitals cited redundancy relative to other patient participate in the PSO program. safety efforts as a reason they do not participate. Uncertainty over the program’s legal protections and determining what information is protected can How OIG Did This Review be challenging for hospitals. This may discourage them from disclosing data to We selected a random sample of their respective PSOs or participating at all. Although the Common Formats 600 general acute-care hospitals to (standard methods for reporting patient safety data) enable AHRQ to survey and achieved a 79-percent aggregate and analyze data, requiring them for the NPSD may slow its response rate. We asked them progress. Forty-two percent (31 of 74) of PSOs cannot contribute to the NPSD detailed questions about their because they do not use the Common Formats. Challenges with the Common experiences in working with federally Formats reflect the limits of using a standardized approach to capturing patient listed PSOs and their perceived value safety data. Finally, AHRQ provides technical assistance that PSOs find helpful, of the program. We also surveyed all but its guidance falls short of meeting PSOs’ needs. federally listed PSOs, achieving What OIG Recommends a 90-percent response rate. We AHRQ should do more to support and promote the PSO program. Specifically, asked them detailed questions about the Office of Inspector General recommends that AHRQ (1) develop and their experiences in working with execute a communications strategy to increase nonparticipating hospitals’ hospitals and with AHRQ. Finally, we awareness of the PSO program and the program’s value to participants; (2) take interviewed AHRQ staff and reviewed steps to encourage PSOs to participate in the NPSD, including accepting data data on AHRQ’s oversight of the into the NPSD in other formats in addition to the Common Formats; and program from 2009 through 2017. (3) update guidance for PSOs on processes for listing PSOs. AHRQ concurred with our first and third recommendations and partially concurred with our second recommendation. Full report can be found at oig.hhs.gov/oei/reports/oei-01-17-00420.asp TABLE OF CONTENTS BACKGROUND 1 Methodology 6 Key Characteristics of the PSO Program 9 FINDINGS Over half of hospitals work with a PSO, and nearly all of them find the relationship valuable 10 Hospitals that do not participate in the program do not perceive it to be distinct from other 11 patient safety efforts Uncertainty over the program’s legal protections and determining what information is 12 protected can be challenging for hospitals Although the Common Formats enable AHRQ to aggregate and analyze data, requiring them 13 for the NPSD may slow its progress AHRQ provides technical assistance that PSOs find helpful, but its guidance on the program 15 falls short of meeting PSOs’ needs CONCLUSION AND RECOMMENDATIONS 16 AHRQ should develop and execute a communications strategy to increase hospitals’ 16 awareness of the program and its value to participants AHRQ should take steps to encourage PSOs to participate in the NPSD, including 17 accepting data into the NPSD in other formats in addition to the Common Formats AHRQ should update guidance for PSOs on the initial and continued listing processes 18 AGENCY COMMENTS AND OIG RESPONSE 19 APPENDICES Appendix A: Statistics for Responses to Select Items From Surveys 20 Appendix B: Agency Comments 32 ACKNOWLEDGMENTS 37 ENDNOTES 38 BACKGROUND Objectives 1. To determine the extent to which hospitals participate in the PSO program. 2. To describe hospital perspectives on the value of the PSO program. 3. To describe challenges to the PSO program. 4. To assess the Agency for Healthcare Research and Quality’s oversight of the PSO program. Researchers have estimated that over 200,000 people die each year because of medical errors in hospitals. 1 The aim of the Patient Safety and Quality Improvement Act of 2005 (the Patient Safety Act, or the Act) is to improve patient safety by encouraging learning from these and other, nonfatal events.2 The Act created the Patient Safety Organization (PSO) program and established PSOs to collect, aggregate, and analyze patient safety information submitted by providers. All aspects of participation in the PSO program are voluntary.3 To address providers’ fears that such information would be used against them, the Act also established the first and only comprehensive, nationwide confidentiality and privilege protections (hereinafter, legal protections) for certain patient safety information that providers submit to PSOs.4 The Act also requires a national Network of Patient Safety Databases (NPSD) to aggregate and analyze nonidentifiable patient safety data and make it available for researchers. 5 The Institute of Medicine—now the Health and Medicine Division of the National Academies—called for such a reporting and learning system in its landmark 1999 report To Err Is Human: Building a Safer Health System.6 In 2010, the Office of Inspector General (OIG) found that 27 percent of hospitalized Medicare beneficiaries experienced harm because of medical care.7 In that report we recommended that the Agency for Healthcare Research and Quality (AHRQ) should, among other activities, enhance its efforts to identify adverse events, in part by continuing to encourage hospitals to participate in the Patient Safety Organization (PSO) program. This study examines the extent to which hospitals have participated in and received value from the PSO program; identifies challenges associated with it; and assesses AHRQ’s oversight of the program. It contributes to OIG’s body of work on patient safety. The PSO Program In 2006, the Secretary of Health and Human Services delegated most authorities under the Patient Safety Act to AHRQ. The Secretary delegated the responsibility for interpreting and enforcing the legal protections of the Patient Safety Organizations: Hospital Participation, Value, and Challenges 1 OEI-01-17-00420 Patient Safety Act to the Office for Civil Rights (OCR).8 In 2008, the Department of Health and Human Services (HHS) published a final rule implementing the Act.9 Patient Safety Organizations Central to the PSO program are organizations that meet AHRQ’s requirements to be federally listed as PSOs. PSOs are private organizations—which may be consulting firms, health care providers, or other entities—that serve as patient safety experts for health care providers that choose to work with them (hereinafter referred to as PSOs’ members).10, PSOs may be nonprofit or for-profit. A PSO’s members may also choose to disclose information about patient safety events and other patient safety information to the PSO. A PSO is required to perform certain patient safety activities to be federally listed as a PSO. These activities include, but are not limited to, efforts to improve patient safety and the quality of health care delivery; the collection and analysis of Patient Safety Work Product (PSWP), which we discuss in more detail below; and procedures to preserve the confidentiality of PSWP. PSOs may perform these activities in various ways, such as analyzing data to identify the causes of patient safety events; developing recommendations to prevent future events and improve patient safety; and facilitating the sharing of best practices among providers to enhance learning.11 As of July 2019, 83 organizations were listed with AHRQ as federally listed PSOs. Patient Safety Work Product and Its Legal Protections The Patient Safety Act established legal protections for certain information—when it meets the definition of PSWP—that providers disclose to PSOs.12 In general, the Act defines PSWP as including any data, reports, records, memoranda, analysis, or statements that are assembled or developed by a provider for reporting to a PSO and are reported to a PSO; or are developed by a PSO for conducting patient safety activities.13 The Act also excludes medical records, among other records, from the definition of PSWP. PSWP is not generally subject to subpoena or discovery in criminal, civil, or administrative proceedings, including disciplinary action against a provider.14, 15 Additionally, PSWP is not subject to disclosure under the Freedom of Information Act.16 According to HHS, these protections alleviate “concerns about such information being used against a provider, such as in litigation.”17 Network of Patient Safety Databases The Patient Safety Act directed the Secretary of HHS to develop the NPSD to enable national learning about patient safety events. According to the Act, the NPSD should have the capacity to accept, aggregate, and analyze other entities.18 AHRQ and others are to use the NPSD data to analyze national and regional statistics, including trends and patterns of health care Patient Safety Organizations: Hospital Participation, Value, and Challenges 2 OEI-01-17-00420 errors.19 AHRQ is to report on the findings from this analysis.20 An intermediary known as the PSO Privacy Protection Center (PSOPPC) renders data nonidentifiable before it reaches the NPSD (see Exhibit 1). A contractor operates both the PSOPPC and the NPSD for AHRQ. In a 2010 report, the Government Accountability Office found that AHRQ was in the process of developing the NPSD and expected it to be ready to receive data from hospitals by February 2011.21 The PSOPPC, which renders data nonidentifiable, was ready to receive data in 2012. In 2017, AHRQ told us that although some PSOs had submitted data, the PSOPPC had not yet released data to the NPSD because its process for rendering data nonidentifiable limited the utility of the data. Furthermore, AHRQ also noted that the process may limit the quantity of data that could be made available to the public. The PSOPPC tested methods for preparing data for the NPSD, and AHRQ launched the public-facing NPSD website on June 21, 2019. Exhibit 1. PSWP flows from providers and PSOs to the NPSD. Patient Safety Organizations: Hospital Participation, Value, and Challenges 3 OEI-01-17-00420 Common Formats As permitted by the Patient Safety Act, AHRQ developed common definitions and formats—known as the Common Formats—for reporting patient safety event data. It did so in collaboration with the National Quality Forum and with input from stakeholders including PSOs and providers. The Common Formats make it possible for AHRQ and others to aggregate and analyze patient safety event data. As of 2019, AHRQ has released Common Formats for reporting events that occur in three settings of care: acute-care hospitals, community pharmacies, and skilled nursing facilities. AHRQ continues to develop new versions of the Common Formats. PSOs are not required to use the Common Formats. However, although PSOs may collect data in any standardized format that permits valid comparisons of similar cases among similar providers, the NPSD accepts only data in the Common Formats. Federally listed PSOs must use either the Common Formats or an alternative system of formats and definitions, or provide a clear explanation why it is not practical or appropriate to do either.22 AHRQ Oversight of PSO Listing and Certification The Patient Safety Act directed the Secretary of HHS to compile and the PSO Program maintain a list of PSOs. Accordingly, an entity wishing to be listed as a PSO must submit a certification to AHRQ attesting that it has policies and procedures in place to perform the patient safety activities described in the Act. It must also attest to meeting additional criteria described in the Act, which include having a qualified workforce, not being a health insurer, and having at least two bona fide contracts with providers every 2 years.23 AHRQ reviews and verifies the certification and ensures that the entity understands the implications of becoming a PSO. If AHRQ accepts the entity’s certifications, AHRQ will list the entity as a PSO.24 After its initial certification, a PSO must recertify with AHRQ every 3 years.25 AHRQ calls its review of a PSO’s recertification a continued listing review (see Exhibit 2 on the next page). AHRQ may also conduct an announced or unannounced compliance review of a PSO to verify compliance with the Act and the final rule that implemented the Act. If AHRQ finds that a PSO is not in compliance, AHRQ may file a notice of preliminary finding of deficiency with the PSO, requiring it to remedy the specified deficiencies. Patient Safety Organizations: Hospital Participation, Value, and Challenges 4 OEI-01-17-00420 Exhibit 2. AHRQ performed its oversight responsibilities as set forth in the Patient Safety Act.26 AHRQ is also responsible for delisting PSOs that do not meet requirements of the Act. Delisting refers to the loss of a PSO’s federally listed status, and AHRQ may delist a PSO for three reasons: the PSO voluntarily relinquishes its status as a PSO; the PSO’s listing expires; or AHRQ revokes the PSO’s listing for cause. AHRQ may revoke a PSO’s listing for cause if the PSO fails to correct a deficiency.27 Technical Assistance The Patient Safety Act authorized the Secretary of HHS to “provide technical assistance to [PSOs], including convening annual meetings for [PSOs] to discuss methodology, communication, data collection, or privacy concerns.”28 Accordingly, AHRQ hosts an annual meeting for PSOs. Topics at the 2018 meeting included sessions in which PSOs shared successful practices and discussions on how AHRQ could improve and support the PSO program. AHRQ also provides technical assistance by responding to inquiries from PSOs and making resources available on its website.29 Litigation Regarding The legal protections for PSWP have been tested through the courts, with varied outcomes. For example, in 2012, an Illinois appellate court affirmed the Act’s Legal a trial court’s decision that pharmacy incident reports constituted PSWP and Protections for were protected under the Act.30 However, since this case, some State courts PSWP have found that the Act does not protect certain information. In 2014, the Kentucky Supreme Court held that adverse event reports created to comply with State laws are not protected. 31 The Florida Supreme Court came to a similar conclusion in 2017. 32 The U.S. Supreme Court denied petitions to review both the Kentucky and Florida cases. 33, 34 Patient Safety Organizations: Hospital Participation, Value, and Challenges 5 OEI-01-17-00420 In 2016, the Kentucky Supreme Court heard another case on the scope of the privileges provided by the Act. It ruled that documents collected, maintained, or developed for the sole purpose of reporting to a PSO are privileged. The Court also clarified that providers may store information in their respective patient-safety evaluation systems but that doing so does not relieve providers from their State and Federal reporting requirements.35 Such cases spurred debate among stakeholders over what constitutes PSWP. In 2016, HHS issued guidance to clarify the definition, stating that “information prepared for purposes other than reporting to a PSO is not PSWP.”36 HHS’s guidance did not settle the uncertainty over PSWP for some. Providers may be reluctant to disclose data that they are uncertain will meet the definition of PSWP and be protected in the State and Federal courts. Methodology Scope This report is based on the results of surveys that we sent to all PSOs listed with AHRQ as of April 2018 and to a nationally representative sample of general acute-care hospitals that participated with Medicare as of February 2018. Our findings on the PSOs’ characteristics, services, and their interactions with AHRQ encompass all PSOs that responded to our survey. Our findings on PSOs’ experiences working with hospitals reflect only those PSOs that identified themselves as working with hospitals. Our findings on AHRQ’s oversight considered the agency’s activities from 2008 through 2017. Data Sources and Analysis To conduct this study, we relied on multiple data sources. PSO Survey We sent an electronic survey to all 82 PSOs listed on the AHRQ website at the time of our survey. PSOs could respond to the survey from May 1, 2018, through June 15, 2018; 74 PSOs responded, a 90-percent response rate. The survey included questions related to PSO characteristics, services PSOs offer, and challenges PSOs face. Hospital Survey We selected a nationally representative, simple random sample of 600 general acute-care hospitals to ask about their experiences with federally listed PSOs. We selected the random sample from among all 3,400 general acute-care hospitals that participated in Medicare in 2018. Of the original 600 hospitals in our sample, we found that 2 were closed, bringing our total sample of eligible hospitals to 598. Patient Safety Organizations: Hospital Participation, Value, and Challenges 6 OEI-01-17-00420 We sent an electronic survey to the sampled hospitals between May 16, 2018, and July 23, 2018; 474 hospitals responded, a 79-percent response rate. We requested information on whether hospitals work with a federally listed PSO; why they did or did not; and what value and challenges they perceive from the PSO program if they do. AHRQ Data We requested data from AHRQ on its oversight of the PSO program from its start through 2017, including the following: the numbers of initial and continued PSO listings; the number of delisted PSOs and related information; the number and outcomes of compliance reviews; and the number of times that AHRQ provided technical assistance to PSOs, and the nature of that technical assistance. Stakeholder Interviews We conducted interviews with staff from a purposive sample of 9 hospitals and 12 PSOs. We use the data from these interviews as examples and to provide context, but do not use them to generalize to all hospitals or PSOs. We also interviewed AHRQ staff and others, including an attorney who works with PSOs and a representative from a professional association for PSOs. Analysis We analyzed data from our PSO survey to describe PSOs’ characteristics, services offered, and experiences working with AHRQ. We also analyzed data from that survey to describe the subset of PSOs that work with hospitals. We produced estimates from our hospital survey data to describe the experiences of general acute-care hospitals with listed PSOs. Some questions on our surveys offered response options on a 3-point or 4-point ranked scale. For example, for some questions, respondents could choose “very important,” “somewhat important,” “slightly important,” or “not important.” For others, they could choose from “major challenge,” “minor challenge,” or “not a challenge.” We report our findings by aggregating all categories that positively identify something either as challenging or as important, for example. Finally, we used data from our interviews with hospitals and PSOs to add context to our survey data and to gain additional detail on areas of interest that PSOs and hospitals identified. Limitations We did not independently verify the survey responses that PSOs and hospitals provided, nor did we independently verify the data that AHRQ provided on its oversight activities. Data from the PSO survey represent the views and experiences of the 74 responding PSOs rather than all 82 PSOs. Patient Safety Organizations: Hospital Participation, Value, and Challenges 7 OEI-01-17-00420 In addition, because we limited the scope of our evaluation of the value of the PSO program to general acute-care hospitals participating in Medicare, this study does not reflect the experiences of other types of providers that work with PSOs. Standards We conducted this study in accordance with the Quality Standards for Inspection and Evaluation issued by the Council of the Inspectors General on Integrity and Efficiency. Patient Safety Organizations: Hospital Participation, Value, and Challenges 8 OEI-01-17-00420 Key Characteristics of the PSO Program Not all PSOs are the same. Although they generally offer a similar array of services, they differ in other ways. For example, PSOs vary by size, profit status, and specialty. Exhibit 3. PSOs by the Numbers Patient Safety Organizations: Hospital Participation, Value, and Challenges 9 OEI-01-17-00420 FINDINGS Fifty-nine percent of general acute-care hospitals participating in Medicare Over half of work with a PSO. More than two-thirds of those hospitals (68 percent) have hospitals work with done so for 5 years or fewer. Among the most important reasons why a PSO, and nearly all hospitals choose to work with a PSO are the opportunity to improve patient of them find the safety (with 94 percent of hospitals citing it as very important in their decision to work with a PSO); the opportunity to learn from PSOs’ analysis relationship of patient safety data (with 87 percent citing it as very important), and the valuable privilege and confidentiality protections for PSWP (with 83 percent citing this reason as very important). Among hospitals that work with a PSO, nearly all (97 percent) find it valuable to work with a PSO and half rate it as very valuable. Hospitals find that working with a PSO improved patient safety Among hospitals that work with PSOs, 80 percent find that feedback and analysis on patient safety events have helped prevent future events, and 72 percent find that such feedback has helped 80% them understand the causes of events. For example, one hospital told us that its PSO alerted its members about a malfunction with a certain medical device. This hospital was able of hospitals that to identify the device and resolve the work with a PSO malfunction. Although it can be difficult to found that the PSO's identify events and quantify improvement, feedback and 63 percent of hospitals that work with PSOs believe that feedback and analysis from a PSO analysis was helpful has made a measurable improvement in patient to prevent future safety.37 patient safety events PSOs offer hospitals analysis and feedback of patient safety in several ways, including root-cause analyses of specific events and analysis of data aggregated from their members. A PSO may use its analysis of aggregate data to show members how their data compare to those of their peers; this service is known as benchmarking. Among hospitals that receive a benchmarking service, nearly all (96 percent) find it helpful. Hospitals find PSO services related to knowledge-sharing and learning to be helpful Among hospitals who say working with a PSO has been very valuable, nearly half volunteered that the value is in the ability to learn from other organizations. Working with a PSO allows hospitals to draw on the shared Patient Safety Organizations: Hospital Participation, Value, and Challenges 10 OEI-01-17-00420 knowledge of their fellow member-hospitals through peer-to-peer learning that would not otherwise be available to them. One Hospital’s Perspective A service called safe tables is one example of how PSOs facilitate peer-to-peer learning “Learning from other organizations in a in a confidential environment. safe environment has been extremely PSOs use safe tables to bring helpful. It assists us with identifying together staff from their risks we may not have considered and provider members—either in decreases the chance of a harm person or virtually—to discuss occurring to our patients...” patient safety topics, such as adverse events that have occurred at member institutions. PSO staff facilitate the meetings, which may include analysis of the causes of adverse events and possible solutions for preventing them in the future. Both hospitals and PSOs find that safe tables are a valuable service. One PSO said that safe tables are among the most valuable services it offers, noting that its membership had quadrupled since it began offering them: “[E]ven though [providers] are hesitant to submit adverse events [to PSOs], they will talk in a protected environment.” A physician we interviewed called safe tables “priceless,” and noted that such discussions can change a hospital’s culture. Nearly all (95 percent) hospitals that work with a PSO found that their PSOs have helped improve the culture of safety at their facilities. A culture of safety is one that (among other key features) enables individuals to report errors without fear of reprimand and to collaborate on solutions.38 Hospitals that do For hospitals that do not participate in the PSO program, a perception that the program is redundant relative to other patient safety efforts is not participate in an important factor for 97 percent of such hospitals and a very important the program do not factor for 70 percent. For example, most hospitals that do not work with perceive it to be a PSO are working with a non-PSO entity (79 percent) to improve patient safety. Similarly, most (82 percent) believe that a PSO’s functions are distinct from other redundant to their internal efforts to improve patient safety. patient safety Furthermore, hospitals perceive PSO reporting 97% efforts as being redundant to Federal and State reporting of patient safety data. Although the data that hospitals send to PSOs may be similar to what they send to these reporting of hospitals that do systems, reporting to a PSO does not exempt not work with a PSO them from government reporting said redundancy was requirements. About 80 percent of hospitals an important factor that do not work with a PSO cite Federal or Patient Safety Organizations: Hospital Participation, Value, and Challenges 11 OEI-01-17-00420 State redundancies in reporting as a factor in that decision. Perceived redundancies in reporting and overlap with other patient safety programs may foster the impression that a PSO creates extra work, detracting from the value that hospitals perceive in working with a PSO. In fact, workload is an important factor for 87 percent of One PSO’s Perspective hospitals that do not participate in the program. Perceived lack Because the PSO program “overlaps of value is also an important with other initiatives… it can be factor in the decision not to challenging to recruit and engage work with a PSO for about members who have limited time and two-thirds (67 percent) of the resources.” hospitals that do not participate. These perceptions may help explain why some PSOs fail to recruit even two providers—the minimum required by the Act—and relinquish their listing as a result. Finally, among hospitals that do not work with a PSO, lack of familiarity with the program was a factor for almost two-thirds (61 percent). Some hospitals volunteered that they wanted to learn more about the program. Uncertainty over the The Act enables the PSO program to offer legal protections for certain patient safety data that other programs cannot. AHRQ’s website includes program’s legal resources for understanding those protections. However, despite these protections and available resources, uncertainty over the Act’s legal protections for PSWP is determining what a challenge for 27 percent of hospitals that work with a PSO, and a major information is challenge for 24 percent. protected can be Concern over the protections may be heightened for providers in States where such protections have been challenged in court. For example, one challenging for PSO told us that some hospitals in Florida, where protections have been hospitals challenged in court, do not report patient safety information because of their uncertainty over legal protections. A clear understanding of the Patient Safety Act’s definitions is vital, because the legal protections apply only to information that meets the definition of PSWP. Fifty-seven percent of hospitals that work with PSOs found determining what constitutes PSWP to be a challenge; however, 43 percent did not find it challenging. Similarly, 56 percent of hospitals that work with PSOs find interpreting HHS guidance on the definition of PSWP to be challenging while 44 percent do not. Hospitals’ concerns over data protections may keep some hospitals from disclosing data to their respective PSOs and others from working with a PSO at all. Uncertainty over data protections was a factor for nearly three-quarters of hospitals that choose not to work with a PSO. Patient Safety Organizations: Hospital Participation, Value, and Challenges 12 OEI-01-17-00420 Hospitals’ concerns over the legal protections create challenges for the PSOs that work with them. In fact, 48 of the 56 PSOs that work with hospitals find that hospitals’ concerns over protections are challenging and 24 find them very challenging. Although the One goal of the Patient Safety Act is to improve patient safety by (in part) using the NPSD to gather and aggregate data for national research and Common Formats learning. Although the Act permits AHRQ to develop Common Formats, enable AHRQ to AHRQ faces a challenge universal to developing any standard—that aggregate and a singular approach cannot fit every situation. Indeed, AHRQ intends for the Common Formats to facilitate national-level data aggregation and analyze data, analysis, rather than to meet the unique needs of every provider and PSO. requiring them for The Act does not require providers and PSOs to use the Common Formats, the NPSD may slow but AHRQ requires data that PSOs submit to the NPSD to be in the its progress Common Formats. Forty-two percent of PSOs surveyed cannot contribute to the NPSD because they do not use the Common Formats Among the 74 PSOs that responded to our survey, 42 percent (31 of 74) neither accept data from members in the Common Formats nor translate data into the Common Formats (through a process called mapping). For some PSOs’ members, using the Common Formats is not an option because none exist for the type of events they experience. In fact, nearly two-thirds of PSOs (46 of 74) reported that the lack of Common Formats for the data their members collect is a challenge to submitting data for the NPSD. The Common Formats The Act provided for development of common definitions and formats, known as the Common Formats, for reporting patient safety event data. The Common Formats enable AHRQ to aggregate and analyze patient safety data that PSOs submit to the NPSD. Only 12 percent of PSOs (9 of 74) that use the Common Formats use them exclusively. Most PSOs that use them told us that they also accept data in other formats, such as those used by their members’ risk management systems. PSOs’ accepting data in multiple formats makes it easier for their members to submit data. Forty percent of PSOs that accept data in other formats (26 of 65) map the data into the Common Formats or engage a third party to do so, but many (47 of 74 PSOs) report that this process is challenging. In the past few years, AHRQ has made progress in getting PSOs to submit data to the NPSD, with the number of records growing from 740,000 in 2017 to 1.8 million in 2019. According to AHRQ, 18 PSOs have submitted data to Patient Safety Organizations: Hospital Participation, Value, and Challenges 13 OEI-01-17-00420 the NSPD, with 3 PSOs submitting the bulk (87 percent) of the records. Because PSOs vary in the numbers of and types of providers they serve, some are likely to submit more data than others. In any case, the number of records the NPSD has received from a limited number of PSOs shows the potential for data aggregation if more PSOs submitted data. Challenges with the Common Formats reflect the limits of using a standardized approach to capturing patient safety data Despite having opportunities to provide input on the design of the Common Formats, over half of PSOs (40 of 74) told us that they do not capture enough information and nearly as many (31 of 74 PSOs) said they capture too much. Furthermore, most PSOs (56 of 74) reported that the Common Formats are not useful for certain patient safety events. In some cases, such as anesthesia-related events, the Common Formats do not collect the type of information that PSOs find useful for learning from these events. One PSO that works with specialty hospitals told us that the Common Formats are designed for general acute-care hospitals, and as a result the Common Formats do not capture the type of information that a specialty hospital might find useful. For example, a rehabilitation hospital would find it useful to know contextual details that are specific to the rehabilitation setting, such as a fall’s having occurred during a routine physical therapy session. One general acute-care hospital told us that these types of limitations with the Common Formats led it to use the “other” category to describe as many as half of its events. A PSO told us that as many as 80 percent of the patient safety events it receives fall into the “other” category. One consequence of this is that the Common Formats’ recording of an “other” event does not capture enough information, or the right type of information, to make the data useful. Indeed, AHRQ told us that incomplete event data and having too many events described as “other” limits the usefulness of the data for analysis and learning. The challenges with the Common Formats highlight the difficulty of developing a standard for the range of patient safety events that PSOs and their members face. This may explain why PSOs choose not to use the Common Formats and why more than half cite as a challenge the lack of clarity on how their submitting data for the NPSD would be valuable either to them as PSOs (42 of 74 PSOs) or valuable to their members (44 of 74 PSOs). Patient Safety Organizations: Hospital Participation, Value, and Challenges 14 OEI-01-17-00420 AHRQ provides Nearly all (43 of 47) PSOs that sought technical assistance from AHRQ over the past year found it helpful, and over half of those (27 of 47) said it was technical assistance very helpful. Requests for technical assistance related to the PSO program that PSOs find increased from 350 in 2009 to 1,134 in 2017. One PSO we interviewed said helpful, but its that AHRQ is quick to respond to requests for technical assistance. guidance on the Similarly, PSOs find AHRQ’s other avenues of providing technical support helpful. For example, nearly all PSOs that attended AHRQ’s annual meeting program falls short (60 of 64 PSOs) found it to be helpful, as did nearly all of the PSOs that had of meeting PSOs’ used AHRQ’s website resources over the preceding year (66 of 70 PSOs). needs One PSO described the annual meeting as “a wonderful opportunity to share information with other PSOs and learn from other programs’ patient safety activities.” Several PSOs noted that AHRQ’s website provides valuable information, with one PSO saying that it and its members “frequently accessed and One PSO’s Perspective utilized” AHRQ’s website resources. “Resources provided through AHRQ However, PSOs struggle to have been supportive and educational interpret AHRQ’s expectations in the day-to-day work of a PSO.” for the processes of initial listing and continued listing. Although PSOs that completed the initial listing process and PSOs that had recently completed the continued listing process generally found AHRQ to be helpful with these processes, some PSOs reported challenges in interpreting AHRQ’s expectations for meeting the requirements.39 For example, nearly two-thirds of PSOs (44 of 72) found it challenging to interpret AHRQ’s expectations for initial listing, with 18 of these reporting that it was a major challenge.40 PSOs that underwent the continued listing process also reported challenges. Of the PSOs that underwent the process in the preceding year, nearly two-thirds (27 of 44) found it challenging to interpret AHRQ’s expectations for the process, and 23 percent of these (10 of 44) reported that it was a major challenge. Some PSOs provided examples of challenges such as difficulty in determining exactly what they needed to prepare for the continued listing process, and AHRQ’s being inconsistent in its expectations. Although AHRQ provides technical assistance to PSOs, HHS delegated the responsibility for interpretation and enforcement of the legal protections to OCR. Therefore, AHRQ does not provide legal guidance to PSOs on the definition of PSWP. Accordingly, AHRQ told us that it refers PSOs with complex questions about legal protections to OCR or brokers a call between the PSO and OCR. The fact that hospitals and PSOs both cited the issue of the protections as a challenge suggests that additional support from AHRQ and OCR might be beneficial. Patient Safety Organizations: Hospital Participation, Value, and Challenges 15 OEI-01-17-00420 CONCLUSION AND RECOMMENDATIONS Congress intended for the PSO program to be unique and powerful among patient safety programs. It is the first and only nationwide program that offers legal protections for providers to disclose patient safety events and learn from them. Where providers were once reluctant to discuss patient safety events for fear of litigation, they may now seek expert analysis from PSOs and discuss these events with peers that are fellow PSO members. Furthermore, through the NPSD, the PSO program is the only comprehensive program that aims to enable learning on a national scale about the causes of patient safety events. The PSO program has the potential to improve health care. Indeed, this review shows that the program has made progress in its first decade. AHRQ has invested in developing and revising the Common Formats, and in creating the NPSD. Over half of hospitals work with a PSO; those hospitals find their participation valuable, with many reporting measurable improvement in patient safety. The number of records in the NPSD is growing, and AHRQ has launched a public-facing website for sharing NPSD data. However, despite this progress, the PSO program faces challenges. A lack of hospital familiarity with the program hinders PSOs’ ability to recruit more hospitals, and concerns over the program’s legal protections may keep hospitals from fully engaging with PSOs. Furthermore, PSOs have not universally adopted the Common Formats. Ultimately, these challenges have slowed AHRQ’s progress toward creating a robust NPSD. As a result, the PSO program has yet to realize its promise of enabling learning and advances in patient safety on a national scale. For the PSO program to fully realize its potential, AHRQ should do more to support and promote the program. Therefore, we recommend that AHRQ: Develop and execute a communications strategy to increase hospitals’ awareness of the program and its value to participants Lack of familiarity with and misperceptions of PSOs among hospitals are challenges that PSOs still face, 10 years after the program began. Therefore, AHRQ should do more to promote the program by developing and executing a communications strategy. In doing so, AHRQ could work to engage provider associations, professional societies, risk management organizations, and other stakeholder organizations. As part of this outreach, AHRQ should explain how aspects like the legal protections and shared learning make working with a PSO different from other quality- and safety-related initiatives. Regarding the legal protections for PSWP, AHRQ Patient Safety Organizations: Hospital Participation, Value, and Challenges 16 OEI-01-17-00420 could reach out to OCR to discuss how the two can improve stakeholders’ understanding of the legal protections. Such discussion could include assessing the potential for formalizing a pathway for PSOs and their members to contact OCR for timely, case-by-case guidance on the legal protections. AHRQ could take advantage of the launch of the public-facing NPSD website and use it to promote the PSO program more broadly. Take steps to encourage PSOs to participate in the NPSD, including accepting data into the NPSD in other formats in addition to the Common Formats Nearly 10 years after OIG encouraged AHRQ to invest in the Common Formats, a number of PSOs (31 of 74) still do not use them. However, the NPSD accepts only data that is in the Common Formats, limiting its ability to aggregate data on a scale that would fulfill the promise of national learning. For this reason, in addition to accepting data in the Common Formats, AHRQ should consider accepting data to the NPSD in other formats as well. In doing so, AHRQ could prioritize accepting data in existing reporting formats such as those used by State and other reporting systems. This might yield large gains in data for the NPSD while reducing redundancies in the reporting workload for providers. Furthermore, AHRQ should also explore advanced technologies that may enable the NPSD to accept and analyze unstructured data in the future. Beyond accepting data in additional formats, AHRQ should take further steps to encourage providers and PSOs to submit data to the NPSD. Such steps might include: 1. Developing a campaign to encourage providers and PSOs to address a specific, high-priority type of patient safety event. Central to the campaign would be submitting a critical mass of data about the event to the NPSD for analysis. AHRQ could use the resulting learning to provide feedback on preventing the event as an example of the NPSD’s value. 2. Collecting and analyzing data on reasons why PSOs do not submit data to the NPSD. AHRQ could use that information to develop next steps for addressing challenges that PSOs face beyond what we identified within this report. Such steps could increase the likelihood that PSOs will contribute data to the NPSD and offer a quicker path to fulfilling the promise of national learning envisioned within the Act. Patient Safety Organizations: Hospital Participation, Value, and Challenges 17 OEI-01-17-00420 Update guidance for PSOs on the initial and continued listing processes AHRQ released a self-assessment tool—its only comprehensive guide on eligibility, listing, operational, and other requirements for PSOs—in September 2009, less than a year after publishing the final rule implementing the Patient Safety Act. Since issuing the self-assessment tool, AHRQ has had nearly 10 years of experience in working with PSOs with varying business models and approaches to meeting the requirements of the Patient Safety Act and the final rule. To provide better guidance for PSOs on the initial and continued listing processes, AHRQ should first consider whether a self-assessment tool is the best format for guiding PSOs through these processes and whether this tool is adequate guidance on its own. AHRQ should then update the tool and/or produce additional guidance as appropriate. Patient Safety Organizations: Hospital Participation, Value, and Challenges 18 OEI-01-17-00420 AGENCY COMMENTS AND OIG RESPONSE AHRQ concurred with our first and third recommendations, and partially concurred with our second recommendation. Regarding our first recommendation, AHRQ said that it will develop and execute a communications strategy to increase hospitals’ awareness of the PSO program and its value. The strategy will include a review of AHRQ’s website and resources, and—subject to available resources—outreach to organizations of providers and other stakeholders. Also as part of its strategy, AHRQ will discuss with OCR how to improve stakeholder understanding of the PSO program’s legal protections. Regarding our second recommendation, AHRQ concurred with taking steps to encourage PSOs to participate in the NPSD, but it did not concur with accepting data to the NPSD in other formats. AHRQ stated that, subject to available resources, it will consider developing a campaign to focus on collecting data on a specific event type to encourage NPSD participation, and that it will discuss that possibility with PSOs at its 2020 PSO Annual Meeting. AHRQ will also explore the use of advanced technologies that might make it possible for the NPSD to accept unstructured data. AHRQ identified challenges to accepting data into the NPSD in existing formats in addition to the Common Formats but stated it could consider doing so should technological and other factors make it feasible. Finally, regarding our third recommendation, AHRQ stated that it will revise its PSO self-assessment tool to more clearly link the contents to additional resources and tools it has developed based on experience with the PSO listing process. For the full text of AHRQ’s comments, see Appendix B. Patient Safety Organizations: Hospital Participation, Value, and Challenges 19 OEI-01-17-00420 APPENDIX A: Statistics for Responses to Select Items From Surveys Survey of Hospitals Sample Point 95% confidence Description size estimate interval PSO participation Percentage of hospitals that work with a PSO 474 59.5% 55.3–63.5% Percentage of hospitals that do not work with 474 40.5% 36.5–44.7% a PSO Number of years the hospital has worked with a PSO Percentage of hospitals that have worked with 282 8.5% 5.9–12.1% a PSO for less than 1 year Percentage of hospitals that have worked with 282 59.2% 53.8–64.4% a PSO for 1 to 5 years Percentage of hospitals that have worked with 282 32.3% 27.4–37.5% a PSO for 6 years or more Importance of opportunity to improve patient safety in deciding to work with a PSO Percentage of hospitals that found opportunity 282 94.0% 90.8–96.1% to improve patient safety very important Percentage of hospitals that found opportunity 282 5.0% 3.1–7.9% to improve patient safety somewhat important Percentage of hospitals that found opportunity 282 1.1% 0.4–3.0% to improve patient safety slightly important Percentage of hospitals that found opportunity 282 0% 0.1–2.2% to improve patient safety not important Patient Safety Organizations: Hospital Participation, Value, and Challenges 20 OEI-01-17-00420 Sample Point 95% confidence Description size estimate interval Importance of opportunity to learn from analysis of aggregate data from providers in deciding to work with a PSO Percentage of hospitals that found opportunity to 282 86.5% 82.4–89.8% learn from analysis very important Percentage of hospitals that found opportunity to 282 7.7% 7.7–14.5% learn from analysis somewhat important Percentage of hospitals that found opportunity to 282 2.8% 1.5–5.3% learn from analysis slightly important Percentage of hospitals that found opportunity to 282 0% 0.1–2.2% learn from analysis not important Importance of privilege and confidentiality protections for PSWP in deciding to work with a PSO Percentage of hospitals that found PSWP privilege 282 82.6% 78.1–86.4% and confidentiality protections very important Percentage of hospitals that found PSWP privilege 282 14.9% 11.4–19.2% and confidentiality protections somewhat important Percentage of hospitals that found PSWP privilege 282 1.8% 0.8–4.0% and confidentiality protections slightly important Percentage of hospitals that found PSWP privilege 282 0.7% 0.2–2.5% and confidentiality protections not important Value of Working with a PSO Percentage of hospitals that found working with a 282 51.8% 46.3–57.2% PSO very valuable Percentage of hospitals that found working with a 282 37.6% 32.5–43.0% PSO somewhat valuable Percentage of hospitals that found working with a 282 7.8% 5.3–11.3% PSO slightly valuable Percentage of hospitals that found working with a 282 2.8% 1.5–5.3% PSO not valuable Patient Safety Organizations: Hospital Participation, Value, and Challenges 21 OEI-01-17-00420 Sample Point 95% confidence Description size estimate interval Usefulness of PSO analysis in preventing future patient safety events Percentage of hospitals that found working with a PSO useful to preventing future patient safety 282 80.1% 75.5–84.1% events Percentage of hospitals that found working with 282 19.9% 15.9–24.6% a PSO not useful to preventing future patient safety events Usefulness of PSO analysis in understanding the causes of patient safety events Percentage of hospitals that found analysis from a PSO useful to understanding the cause of 282 71.6% 66.5–76.3% patient safety events Percentage of hospitals that found analysis from 282 28.4% 23.7–33.5% a PSO not useful to understanding the cause of patient safety events Measurable improvement in patient safety from PSO analysis Percentage of hospitals that found PSO analysis resulted in measurable improvement in patient 282 62.8% 57.4–67.9% safety Percentage of hospitals that found PSO analysis 282 37.2% 32.2–42.6% did not result in measurable improvement in patient safety Helpfulness of PSO analysis of PSWP from its members [i.e., benchmarking] to hospitals Percentage of hospitals that found PSO analysis 249 96.0% 93.0–97.7% of PSWP helpful Percentage of hospitals that found PSO analysis 249 4.0% 2.3–7.0% of PSWP not helpful Patient Safety Organizations: Hospital Participation, Value, and Challenges 22 OEI-01-17-00420 Sample Point 95% confidence Description size estimate interval Helpfulness of PSO service of cultivating a culture of safety to hospitals Percentage of hospitals that found PSO service of 265 95.1% 92.0–97.0% cultivating a culture of safety helpful Percentage of hospitals that found PSO service of 265 4.9% 3.0–8.0% cultivating a culture of safety not helpful Helpfulness of PSO service of safe tables/member convenings to hospitals Percentage of hospitals that found PSO service of 207 94.7% 91.0–96.9% safe tables/member convenings helpful Percentage of hospitals that found PSO service of 207 5.3% 3.1–9.0% safe tables/member convenings not helpful Lack of familiarity with the PSO program as a reason for not working with a PSO Percentage of hospitals that found lack of familiarity with the PSO program an important 192 60.9% 54.4–67.1% reason Percentage of hospitals that found lack of 192 39.1% 32.9–45.6% familiarity with the PSO program not an important reason At least one form of redundancy as a reason for not working with a PSO Percentage of hospitals that found at least one 192 69.8% 63.4–75.5% form of redundancy a very important reason Percentage of hospitals that found at least one 192 20.3% 15.5–26.1% form of redundancy a somewhat important reason Percentage of hospitals that found at least one 192 7.3% 4.5–11.5% form of redundancy a slightly important reason Percentage of hospitals that found at least one 192 2.6% 1.2–5.8% form of redundancy not an important reason Already working with another entity to improve patient safety (including nonlisted PSOs) as a reason for not working with a PSO Percentage of hospitals that found working with another entity to improve patient safety an 192 79.2% 73.3–84.0% important reason Percentage of hospitals that found working with 192 20.8% 16.0–26.7% another entity to improve patient safety not an important reason Patient Safety Organizations: Hospital Participation, Value, and Challenges 23 OEI-01-17-00420 Sample Point 95% confidence Description size estimate interval Redundancy to internal efforts as a reason for not working with a PSO Percentage of hospitals that found redundancy 192 81.8% 76.1–86.3% to internal efforts an important reason Percentage of hospitals that found redundancy 192 18.2% 13.7–23.9% to internal efforts not an important reason Redundancy to Federal reporting as a reason for not working with a PSO Percentage of hospitals that found redundancy 192 83.3% 77.8–87.7% to Federal reporting an important reason Percentage of hospitals that found redundancy 192 16.7% 12.3–22.2% to Federal reporting not an important reason Redundancy to State reporting as a reason for not working with a PSO Percentage of hospitals that found redundancy 192 79.7% 73.9–84.5% to State reporting an important reason Percentage of hospitals that found redundancy 192 20.3% 15.5–26.1% to State reporting not an important reason Workload as a reason for not working with a PSO Percentage of hospitals that found workload is 192 87.0% 81.9–90.8% an important reason Percentage of hospitals that found workload is 192 13.0% 9.2–18.1% not an important reason Lack of value as a reason for not working with a PSO Percentage of hospitals that found lack of value 192 67.2% 60.7–73.1% is an important reason Percentage of hospitals that found lack of value 192 32.8% 27.0–39.3% is not an important reason Patient Safety Organizations: Hospital Participation, Value, and Challenges 24 OEI-01-17-00420 Sample Point 95% confidence Description size estimate interval Uncertainty over privilege and confidentiality protections as a challenge to working with a PSO Percentage of hospitals that found uncertainty over privilege/confidentiality protections to be a major 282 23.8% 19.5–28.7% challenge Percentage of hospitals that found uncertainty over privilege/confidentiality protections to be a minor 282 27.0% 22.4–32.0% challenge Percentage of hospitals that found uncertainty over privilege/confidentiality protections not to be a 282 49.3% 43.9–54.7% challenge Determining what constitutes PSWP as a challenge to working with a PSO Percentage of hospitals that found determining 282 57.1% 51.7–62.4% what constitutes PSWP a to be a challenge Percentage of hospitals that found determining 282 42.9% 37.6–48.4% what constitutes PSWP not to be a challenge Interpreting the 2016 HHS guidance document as a challenge to working with a PSO Percentage of hospitals that found interpreting the 282 56.0% 50.6–61.3% 2016 HHS guidance document to be a challenge Percentage of hospitals that found interpreting the 282 44.0% 38.7–49.4% 2016 HHS guidance document not to be a challenge Uncertainty of privilege and confidentiality protections as a reason for not working with a PSO Percentage of hospitals that found uncertainty of privilege and confidentiality protections an 192 74.5% 68.3–79.8% important reason Percentage of hospitals that found uncertainty of 192 25.5% 20.2–31.7% privilege and confidentiality protections not an important reason Patient Safety Organizations: Hospital Participation, Value, and Challenges 25 OEI-01-17-00420 Survey of PSOs Description Percentage Number/Total PSO nonprofit status Percentage of PSOs that are nonprofit 62.2% 46/74 37.8% 28/74 Percentage of PSOs that are not nonprofit Health care provider status of PSO or its parent company Percentage of PSOs or parent companies that are health 33.8% 25/74 care providers Percentage of PSOs or parent companies that are not health 66.2% 49/74 care providers Offering aggregate analysis of PSWP across providers Percentage of PSOs that offer aggregate analysis of PSWP 94.6% 70/74 across providers Percentage of PSOs that do not offer aggregate analysis of 5.4% 4/74 PSWP across providers Helping cultivate a culture of safety Percentage of PSOs that cultivate a culture of safety 91.9% 68/74 8.1% 6/74 Percentage of PSOs that do not cultivate a culture of safety Offering at least one learning-based service Percentage of PSOs that offer at least one learning-based 89.2% 66/74 service Percentage of PSOs that do not offer at least one learning- 10.8% 8/74 based service Offering safe tables or safety huddles Percentage of PSOs that offer either safe tables, safety 73.0% 54/74 huddles, or both Percentage of PSOs that offer neither safe tables nor 27.0% 20/74 safety huddles Patient Safety Organizations: Hospital Participation, Value, and Challenges 26 OEI-01-17-00420 Description Percentage Number/Total Working with at least one general acute-care hospital Percentage of PSOs that work with at least one general 75.7% 56/74 acute-care hospital Percentage of PSOs that do not work with at least one 24.3% 18/74 general acute-care hospital Number of general acute-care hospitals with which PSO works 0 24.3% 18/74 1 to 9 17.6% 13/74 10 to 49 28.4% 21/74 50 to 99 14.9% 11/74 100 or more 14.9% 11/74 Working with more than one provider type Percentage of PSOs that work with more than one provider 82.4% 61/74 type Percentage of PSOs that do not work with more than one 17.6% 13/74 provider type Hospital concerns about data protections as a challenge to PSOs Percentage of PSOs that find hospital concerns about data 85.7% 48/56 protections to be a challenge Percentage of PSOs that find hospital concerns about data 14.3% 8/56 protections not to be a challenge Hospitals not submitting data as a challenge to PSOs Percentage of PSOs that find hospitals not 62.5% 35/56 submitting data to be a challenge Percentage of PSOs that find hospitals not 37.5% 21/56 submitting data not to be a challenge Patient Safety Organizations: Hospital Participation, Value, and Challenges 27 OEI-01-17-00420 Description Percentage Number/Total Accepting data from members in Common Formats and/or translating data into Common Formats through mapping Percentage of PSOs that neither accept CF nor map 41.9% 31/74 Percentage of PSOs that do not accept CF but map 9.5% 7/74 Percentage of PSOs that accept CF but do not map 23.0% 17/74 Percentage of PSOs that accept CF and map 25.7% 19/74 A lack of Common Formats relevant to the data that PSOs’ members collect as a challenge to PSOs Percentage of PSOs that found a lack of Common Formats relevant to the data that PSOs’ members collect to be a 62.2% 46/74 challenge Percentage of PSOs that found a lack of Common Formats 37.8% 28/74 relevant to the data that PSOs’ members collect not to be a challenge Format of patient safety reports accepted by PSOs Percentage of PSOs that accept patient safety reports in 12.2% 9/74 AHRQ's Common Formats Percentage of PSOs that accept patient safety reports in a 51.4% 38/74 format other than the CF Percentage of PSOs that accept patient safety reports in both 36.5% 27/74 Mapping data to the Common Formats Percentage of PSOs that map data to the Common 40.0% 26/74 Formats Percentage of PSOs that do not map data to the 60.0% 39/74 Common Formats Mapping data onto the Common Formats as a challenge to PSOs Percentage of PSOs that found mapping data onto 63.5% 47/74 the Common Formats to be a challenge Percentage of PSOs that found mapping data onto 36.5% 27/74 the Common Formats not to be a challenge Patient Safety Organizations: Hospital Participation, Value, and Challenges 28 OEI-01-17-00420 Description Percentage Number/Total Common Formats not capturing enough information as a challenge to PSOs Percentage of PSOs that found that the Common Formats do 54.1% 40/74 not capture enough information to be a challenge Percentage of PSOs that found that the Common Formats do 46.0% 34/74 not capture enough information not to be a challenge Common Formats capturing too much information as a challenge to PSOs Percentage of PSOs that found that the Common Formats 41.9% 31/74 capture too much information to be a challenge Percentage of PSOs that found that the Common Formats 58.1% 43/74 capture too much information not to be a challenge Usefulness of the Common Formats for certain types of patient safety events as a challenge to PSOs Percentage of PSOs that found usefulness of the Common 75.7% 56/74 Formats for certain types of patient safety events to be a challenge Percentage of PSOs that found usefulness of the Common 24.3% 18/74 Formats for certain types of patient safety events not to be a challenge Lack of clarity on how submitting to the NPSD would provide value to this PSO as a challenge Percentage of PSOs that found a lack of clarity on how submitting to the NPSD would provide value to this PSO to 56.8% 42/74 be a challenge Percentage of PSOs that found a lack of clarity on how 43.2% 32/74 submitting to the NPSD would provide value to this PSO not to be a challenge Patient Safety Organizations: Hospital Participation, Value, and Challenges 29 OEI-01-17-00420 Description Percentage Number/Total Lack of clarity on how submitting to the NPSD would provide value to PSOs’ members as a challenge Percentage of PSOs that found a lack of clarity on how submitting to the NPSD would provide value to PSOs’ 59.5% 44/74 members to be a challenge Percentage of PSOs that found a lack of clarity on how 40.5% 30/74 submitting to the NPSD would provide value to PSOs’ members not to be a challenge Helpfulness of AHRQ with technical assistance over the past year Percentage of PSOs that found AHRQ to be very helpful with 57.5% 27/47 technical assistance Percentage of PSOs that found AHRQ to be somewhat 21.3% 10/47 helpful with technical assistance Percentage of PSOs that found AHRQ to be slightly helpful 12.8% 6/47 with technical assistance Percentage of PSOs that found AHRQ to not be helpful with 8.5% 4/47 technical assistance Helpfulness of AHRQ with their annual meeting over the past year Percentage of PSOs that found AHRQ’s annual meeting to be 93.8% 60/64 helpful Percentage of PSOs that found AHRQ’s annual meeting to 6.3% 4/64 not be helpful Helpfulness of AHRQ with website resources over the past year Percentage of PSOs that found AHRQ to be helpful with 94.3% 66/70 website resources Percentage of PSOs that found AHRQ to not be helpful with 5.7% 4/70 website resources Patient Safety Organizations: Hospital Participation, Value, and Challenges 30 OEI-01-17-00420 Description Percentage Number/Total Helpfulness of AHRQ with technical assistance over the past year Percentage of PSOs that found AHRQ to be very helpful 57.5% 27/47 with technical assistance Percentage of PSOs that found AHRQ to be somewhat 21.3% 10/47 helpful with technical assistance Percentage of PSOs that found AHRQ to be slightly helpful 12.8% 6/47 with technical assistance Percentage of PSOs that found AHRQ to not be helpful 8.5% 4/47 with technical assistance Helpfulness of AHRQ with their annual meeting over the past year Percentage of PSOs that found AHRQ’s annual meeting to 93.8% 60/64 be helpful Percentage of PSOs that found AHRQ’s annual meeting to 6.3% 4/64 not be helpful Helpfulness of AHRQ with website resources over the past year Percentage of PSOs that found AHRQ to be helpful with 94.3% 66/70 website resources Percentage of PSOs that found AHRQ to not be helpful 5.7% 4/70 with website resources Interpreting AHRQ's expectations for initial listing as a challenge to PSOs Percentage of PSOs that found interpreting AHRQ’s 25.0% 18/72 expectations for initial listing to be a major challenge Percentage of PSOs that found interpreting AHRQ’s 36.1% 26/72 expectations for initial listing to be a minor challenge Percentage of PSOs that found interpreting AHRQ’s 38.9% 28/72 expectations for initial listing not to be a challenge Interpreting AHRQ's expectations for continued listing in the past year as a challenge to PSOs Percentage of PSOs that found interpreting AHRQ’s 22.7% 10/44 expectations for continued listing to be a major challenge Percentage of PSOs that found interpreting AHRQ’s 38.6% 17/44 expectations for continued listing to be a minor challenge Percentage of PSOs that found interpreting AHRQ’s 38.6% 17/44 expectations for continued listing not to be a challenge Patient Safety Organizations: Hospital Participation, Value, and Challenges 31 OEI-01-17-00420 APPENDIX B: Agency Comments Patient Safety Organizations: Hospital Participation, Value, and Challenges 32 OEI-01-17-00420 Patient Safety Organizations: Hospital Participation, Value, and Challenges 33 OEI-01-17-00420 Patient Safety Organizations: Hospital Participation, Value, and Challenges 34 OEI-01-17-00420 Patient Safety Organizations: Hospital Participation, Value, and Challenges 35 OEI-01-17-00420 Patient Safety Organizations: Hospital Participation, Value, and Challenges 36 OEI-01-17-00420 ACKNOWLEDGMENTS Elizabeth Sandefer served as the team leader for this study. Others in the Office of Evaluation and Inspections who conducted the study include Malaena Taylor and Shanna Weitz. Office of Evaluation and Inspections staff who provided support include Althea Hosein, Christine Moritz, and Mike Novello. This report was prepared under the direction of Joyce Greenleaf, Regional Inspector General for Evaluation and Inspections in the Boston regional office, and Kenneth Price, Deputy Regional Inspector General. To obtain additional information concerning this report or to obtain copies, contact the Office of Public Affairs at Public.Affairs@oig.hhs.gov. Patient Safety Organizations: Hospital Participation, Value, and Challenges 37 OEI-01-17-00420 ENDNOTES 1 James, J.T. 2013. “A New, Evidence-Based Estimate of Patient Harms Associated with Hospital Care.” Journal of Patient Safety, Vol. 9(3): 122-128. 2 73 Fed. Reg. 70732 (Nov. 21, 2008). 3 Ibid. 4 42 U.S.C. § 299b-22. 5 42 U.S.C. § 299b-23. 6 Institute of Medicine, To Err Is Human: Building A Safer Health System, November 1999. Accessed at http://nationalacademies.org/hmd/Reports/1999/To-Err-is-Human-Building-A-Safer-Health-System.aspx on September 15, 2017. 7 About 13.5 percent of hospitalized beneficiaries experienced adverse events resulting in serious harm, and an additional 13.5 percent experienced events that resulted in temporary harm. See OIG, Adverse Events in Hospitals: A National Incidence Among Medicare Beneficiaries, November 2010. Accessed at https://oig.hhs.gov/oei/reports/oei-06-09-00090.pdf on September 18, 2017. 8 71 Fed. Reg. 28701 (May 17, 2006). 9 73 Fed. Reg. 70732 (Nov. 21, 2008). 10 AHRQ, “Frequently Asked Questions,” Patient Safety Organization (PSO) Program. Accessed at https://www.pso.ahrq.gov/faq#BenefitstoHealthcareProviders on September 4, 2019. 11 Ibid. 12 The Office for Civil Rights is responsible for overseeing the confidentiality protections for PSWP. 13 42 U.S.C. § 299b-21(7)(A)(i). 14 42 CFR § 3.204(a)(1). 15 42 CFR § 3.204(a)(2). 16 42 CFR § 3.204(a)(3). 17 81 Fed. Reg. 32655 (May 24, 2016). 18 42 U.S.C. § 299b-23(a). 19 42 U.S.C. § 299b-23(c). 20 The Act directs the Secretary to include information resulting from analyses of the NPSD in HHS’s annual quality reports. 21 Government Accountability Office, Patient Safety Act: HHS Is in the Process of Implementing the Act, So Its Effectiveness Cannot Yet Be Evaluated. January 2010. Available online at http://www.gao.gov/new.items/d10281.pdf. 22 42 CFR 3.102(b)(2)(iii). 23 P.L. No. 109-41 § 2, PHSA, § 924, 42 U.S.C. § 299b-24. 24 42 CFR 3.104. 25 P.L. No. 109-41 § 2, PHSA, § 924, 42 U.S.C. 299b–24(a)(2). 26 PSOs listed prior to the effective date of the final rule implementing the Patient Safety Act (January 19, 2009) had to apply for a new initial listing after the effective date of the final rule. 27 42 CFR 3.108(e). 28 P.L. No. 109-41 § 2, PHSA, § 926, 42 U.S.C. 299b-25. 29 42 CFR 3.304(b). 30 The Department of Financial and Professional Regulation v. Walgreen Co, 970 N.E.2d 552 (Ill. 2012). 31 Tibbs v. Bunnell, 448 S.W.3d 796 (Ky. 2014). 32 Charles v. Southern Baptist Hosp. of Florida, Inc., 209 So. 3d 1199 (Fla. Jan. 31, 2017). 33 Tibbs v. Bunnell, 136 S. Ct. 2504 (June 27, 2016). 34 S. Baptist Hosp. of Fla., Inc. v. Charles, 138 S. Ct. 129 (Oct. 2, 2017). 35 Baptist Health Richmond, Inc. v. Clouse, 497 S.W.3d 759 (Ky. Sept. 22, 2016). 36 81 Fed. Reg. 32656 (May 24, 2016). 37 OIG, Hospital Incidence Reports Do Not Capture Most Patient Harm (OEI-06-09-00091), January 2012. Accessed at https://oig.hhs.gov/oei/reports/oei-06-09-00091.pdf on July 24, 2019. 38 AHRQ, Patient Safety Primer. January 2019. Available online at https://psnet.ahrq.gov/primers/primer/5/Culture-of-Safety. 39 We define “recently” as those PSOs that completed continued listing within the year prior to receiving our survey in May 2018. 40 All but 2 of the 74 PSOs that responded to our survey recalled their experiences with initial listing. The remaining two respondents were unable to answer this question. Patient Safety Organizations: Hospital Participation, Value, and Challenges 38 OEI-01-17-00420 ABOUT THE OFFICE OF INSPECTOR GENERAL The mission of the Office of Inspector General (OIG), as mandated by Public Law 95-452, as amended, is to protect the integrity of the Department of Health and Human Services (HHS) programs, as well as the health and welfare of beneficiaries served by those programs. This statutory mission is carried out through a nationwide network of audits, investigations, and inspections conducted by the following operating components: Office of Audit The Office of Audit Services (OAS) provides auditing services for HHS, either by conducting audits with its own audit resources or by overseeing audit Services work done by others. Audits examine the performance of HHS programs and/or its grantees and contractors in carrying out their respective responsibilities and are intended to provide independent assessments of HHS programs and operations. These assessments help reduce waste, abuse, and mismanagement and promote economy and efficiency throughout HHS. Office of Evaluation The Office of Evaluation and Inspections (OEI) conducts national evaluations to provide HHS, Congress, and the public with timely, useful, and reliable and Inspections information on significant issues. These evaluations focus on preventing fraud, waste, or abuse and promoting economy, efficiency, and effectiveness of departmental programs. To promote impact, OEI reports also present practical recommendations for improving program operations. Office of The Office of Investigations (OI) conducts criminal, civil, and administrative investigations of fraud and misconduct related to HHS programs, Investigations operations, and beneficiaries. With investigators working in all 50 States and the District of Columbia, OI utilizes its resources by actively coordinating with the Department of Justice and other Federal, State, and local law enforcement authorities. The investigative efforts of OI often lead to criminal convictions, administrative sanctions, and/or civil monetary penalties. Office of Counsel to The Office of Counsel to the Inspector General (OCIG) provides general legal services to OIG, rendering advice and opinions on HHS programs and the Inspector operations and providing all legal support for OIG’s internal operations. General OCIG represents OIG in all civil and administrative fraud and abuse cases involving HHS programs, including False Claims Act, program exclusion, and civil monetary penalty cases. In connection with these cases, OCIG also negotiates and monitors corporate integrity agreements. OCIG renders advisory opinions, issues compliance program guidance, publishes fraud alerts, and provides other guidance to the health care industry concerning the anti-kickback statute and other OIG enforcement authorities.