Pennsylvania Patient Safety Advisory Using Administrative Data from Pennsylvania Hospitals to Monitor Patient Safety Since the Pennsylvania Patient Safety Authority was The PSIs that can be used at a state or regional level established, the most challenging question asked of its (referred to as the “area level” indicators) are as staff has been whether healthcare in Pennsylvania is follows: becoming safer. This question is not unique to Penn- ■ Accidental Puncture or Laceration sylvania, nor is it unique to the United States. Experts ■ Foreign Body Left during Procedure in patient safety are forced to admit that while prog- ress has been made since the 1999 publication of the ■ Iatrogenic Pneumothorax (i.e., collapsed lung) Institute of Medicine’s To Err Is Human, improving ■ Postoperative Hemorrhage or Hematoma (i.e., patient safety is a journey that is just beginning. bleeding) The ultimate measures of safety are the number of ■ Postoperative Wound Dehiscence (i.e., rupturing lives saved or the number of injuries prevented, but of the suture line following surgery) these measures are notoriously difficult to estimate ■ Selected Infections due to Medical Care (primarily reliably in a cost-effective way. The sources of data related to intravenous lines and catheters) typically reviewed for evidence of improvement are ■ Transfusion Reaction (due to blood incompatibility) all imperfect. Adverse event reports are subject to underreporting and variation in interpretation of These PSIs provide one window into the safety of reporting requirements. Survey data on structural Pennsylvania hospitals, and over time one hopes or process measures, as presented in the Authority’s to see these rates decline, suggesting that safety is 2008 annual report, is subject to response bias, the improving. Because of differences between the PSI selective memory of the respondent, and many other definitions and how reportable events are defined biases inherent in all survey research. Even retrospec- under Pennsylvania’s MCARE (Medical Care Avail- tive expert review of medical charts, often used as the ability and Reduction of Error) Act of 2002, direct gold standard in research on adverse events, is sub- comparisons with the reports submitted to the ject to the validity of the decision rules used by the Authority are not appropriate. What the PSIs pro- reviewers and the quality of the documentation in the vide is an independent source of information about patient records. patient safety. Use of multiple data sources can help ensure greater confidence in potential trends; changes While all these sources of data are imperfect, each observed in any single source of data are more likely can provide a unique perspective on the safety and to be genuine and meaningful when corroborated by resilience of the healthcare system. While each source changes observed in other independent sources. on its own is too flawed to rely on in isolation, when taken together they can paint a richer portrait of the Overall, the evidence for improvement in these PSIs problems faced in patient safety and whether there is over the past several years is mixed and uncertain. progress in resolving them. Some PSIs, such as Transfusion Reaction and Post- operative Wound Dehiscence, seem to have declined, Another source of information readily available to suggesting a move in the right direction. Yet others, all hospitals is uniform administrative data used in such as Selected Infections due to Medical Care and billing. Under contract to the Agency for Healthcare Postoperative Hemorrhage or Hematoma, seem to be Research and Quality (AHRQ), researchers from trending upward. However, all linear trend lines that Stanford University and the University of California were fit to these indicators failed tests for statistical developed the Patient Safety Indicators (PSIs) as a significance, leaving no convincing evidence that the tool to identify potentially preventable adverse events apparent trends in the data are due to anything other related to hospitalization. These indicators are based than chance.* The Figure presents the rates of these on records that hospitals complete on all inpatient complications from 2002 through 2007. discharges. While administrative systems were not Even if the apparent declines in some of these designed to identify adverse events, by screening complication rates were statistically significant, the patients’ diagnoses and what services they received, improvement would be only moderate (though the PSIs identify by inference patients who may have encouraging). Table 1 shows the PSIs with the percent suffered selected adverse events. change between 2002 and 2007 and with the num- As with other sources of patient safety information, ber of cases avoided or added based on the percent administrative data is subject to technical limitations. These include variations in coding practices at differ- ent institutions and by different individuals, errors * For each indicator, a linear trend line was fit to the data, and a Student t-test was performed on the slope of each trend line, in coding, and the quality of the underlying medical testing the hypothesis that the slope was different from 0 at the records on which the administrative data is based. = 0.05 level. Refer to the section “Technical Notes and Limita- tions” for further detail. (continued on page 124) Page 122 REPRINTED ARTICLE - ©2009 Pennsylvania Patient Safety Authority Vol. 6, No. 4—December 2009 Pennsylvania Patient Safety Advisory Figure. Patient Safety Indicators for Pennsylvania Hospitals, Rates per 100,000 Population (2002 to 2007) Accidental Puncture or Laceration Retained Foreign Body Iatrogenic Pneumothorax 62 2.0 11.0 10.92 60.27 1.88 10.86 60 1.9 10.81 1.68 1.79 1.76 10.8 10.70 1.8 1.71 58 57.08 57.36 1.7 10.61 10.6 56 1.6 1.53 53.67 53.98 1.5 10.4 54 53.16 1.4 10.22 1.3 10.2 52 1.2 10.0 50 1.1 48 1.0 9.8 2002 2003 2004 2005 2006 2007 2002 2003 2004 2005 2006 2007 2002 2003 2004 2005 2006 2007 Postoperative Hemorrhage or Hematoma Postoperative Wound Dehiscence Selected Infections 21 3.6 3.51 42 3.37 20 19.89 3.4 3.29 3.31 40 39.52 39.71 19.32 18.93 3.18 18.91 3.2 3.06 37.94 38.27 19 18.80 18.39 38 36.56 37.61 3.0 18 2.8 36 17 2.6 34 2.4 16 2.2 32 15 2.0 30 2002 2003 2004 2005 2006 2007 2002 2003 2004 2005 2006 2007 2002 2003 2004 2005 2006 2007 Transfusion Reaction 0.12 0.10 0.10 0.08 0.07 Notes: 0.08 0.07 Calculated using AHRQ Quality Indicators Software, Windows Ver. 3.2a (Rockville, MD). 0.06 0.05 Population estimates include the adult population of Pennsylvania, age 18 and older. 0.04 Rates calculated by the Pennsylvania Patient Safety Authority using data provided by the 0.02 Pennsylvania Health Care Cost Containment Council. See technical notes for details. 0.02 MS09513 0.00 2002 2003 2004 2005 2006 2007 Table 1. Change in Patient Safety Indicator Rates % CHANGE IN OBSERVED RATE LINEAR TREND NUMBER OF CASES PATIENT SAFETY INDICATOR (2002 TO 2007) SLOPE* AVOIDED/ADDED (2007) † Accidental Puncture or Laceration -4.8 -0.15 72 avoided Foreign Body Left during Procedure -9.3 -0.02 10 avoided Iatrogenic Pneumothorax -0.8 0.04 20 added Postoperative Hemorrhage or Hematoma +0.7 0.16 78 added Postoperative Wound Dehiscence -9.3 -0.05 22 avoided Selected Infections due to Medical Care +2.9 0.34 163 added Transfusion Reaction -75.6 -0.01 3 avoided * A Student t-test was performed on the observed slope for each indicator, all of which were found to be not statistically significant at the = 0.05 level. † Refers to the difference between the number of cases predicted for 2007 based on the linear trend lines shown in the Figure and the number that would have been predicted for 2007 if 2002 predicted rates had stayed constant (i.e., if linear trend lines were flat). Vol. 6, No. 4—December 2009 REPRINTED ARTICLE - ©2009 Pennsylvania Patient Safety Authority Page 123 Pennsylvania Patient Safety Advisory Table 2. Patient Safety Indicators, Comparison of Pennsylvania Observed Rates and National Estimated Rates PENNSYLVANIA OBSERVED NATIONAL ESTIMATED RATE PATIENT SAFETY INDICATOR RATE PER 100,000 (2007)* PER 100,000 (2006)† 1. Accidental Puncture or Laceration 57.36 48.08 2. Foreign Body Left during Procedure 1.53 1.53 3. Iatrogenic Pneumothorax 10.61 8.09 4. Postoperative Hemorrhage or Hematoma 18.93 16.11‡ 5. Postoperative Wound Dehiscence 3.18 2.48 6. Selected Infections due to Medical Care 37.61 29.82 7. Transfusion Reaction 0.02 0.06 * The Pennsylvania observed rate is the actual number of cases meeting the Patient Safety Indicator inclusion criteria divided by the Pennsylvania population as published in the U.S. Census; it is not risk-adjusted for differences between the Pennsylvania and U.S. populations. † National rates from: Agency for Healthcare Research and Quality (AHRQ). (1-3; 5-7) HCUPnet, Healthcare Cost and Utilization Project, QI summary tables [online]. [cited 2009 Aug 5]. Available from Internet: http://hcupnet.ahrq.gov; (4) PSI comparative data for area indicators, ver. 3.1 [online]. 2007 Mar 12 [cited 2009 Mar 30]. Available from Internet: http://www.qualityindicators.ahrq.gov/downloads/psi/psi_area_ comparative_v31.pdf. ‡ Based on 2004 data; 2006 data unavailable for this indicator. (continued from page 122) the Authority during the same time period. The statu- tory definition of events reportable to the Authority change over this period. Those with the greatest per- requires healthcare providers to assess whether cent change are not necessarily those in which adverse events were unanticipated, whether they the most improvement would have occurred. For require additional healthcare services, and whether example, a decline in the rate of Transfusion Reac- they compromise patient safety. tions per 100,000 population from 0.08 in 2002 to 0.02 in 2007 represents a 76% decline and 3 injuries These rates do not take into account the “Present on avoided. In comparison, cases of Accidental Puncture Admission” (or POA) indicator, which identifies in or Laceration, which occur more frequently, declined each patient’s discharge abstract the diagnosis codes about 5% from 2002 to 2007, but this equates to that were present when the patient was admitted to 72 cases avoided. the hospital. While hospitals were required to report this indicator starting in October 2008, it is not yet Data from Pennsylvania is on a par with the most included in publicly available discharge data. Of the recent national data available (see Table 2). While area-level indicators, the POA indicator is used only the observed rates in Pennsylvania for most PSIs are as an exclusion criterion for Selected Infections due slightly higher than national estimates, hospital dis- to Medical Care, and it would not affect calculations charge coding practices vary between hospitals and of the other indicators. between states. Therefore, tracking changes in the same set of institutions over time is more meaningful The Authority calculated the rates of the seven area- than making comparisons between hospitals or geo- level PSIs using data provided by the Pennsylvania graphic regions. Health Care Cost Containment Council (PHC4). The rates were calculated for the years 2002 through Technical Notes and Limitations 2007, the most recent full year for which data was The observed rates of complications presented here publicly available. Rates were calculated using are subject to the limitations inherent in all hospital AHRQ’s Quality Indicators software, Windows discharge data. The primary concern is with the accu- version 3.2a (Rockville, Maryland). For more infor- racy of discharge-based diagnosis coding. Errors made mation about the AHRQ Quality Indicators, visit in individual institutions’ discharge abstraction may http://www.qualityindicators.ahrq.gov. bias the rates calculated using those data sources. As PHC4 is an independent state agency responsible with any source of patient safety data, it is not pos- for addressing the problem of escalating health costs, sible to identify all relevant adverse events without ensuring the quality of healthcare, and increasing some false positives and false negatives. access to healthcare for all citizens regardless of ability It is not possible to distinguish, in this data, cases to pay. PHC4 has provided data to the Authority in that represent preventable adverse events from those an effort to further PHC4’s mission of educating the representing adverse events that are not preventable. public and containing healthcare costs in Pennsylva- Likewise, it is not possible to distinguish cases that nia. PHC4, including its agents and staff, has made represent medical errors from cases in which no error no representation, guarantee, or warranty, express occurred. For these reasons, it is not expected that the or implied, that the financial, patient, payer, and number of potential adverse events identified in the physician-specific data provided to the Authority is PSIs would equal the number of reports submitted to error-free, or that the use of the data will avoid Page 124 REPRINTED ARTICLE - ©2009 Pennsylvania Patient Safety Authority Vol. 6, No. 4—December 2009 Pennsylvania Patient Safety Advisory Using the PSIs in Your Hospital The Agency for Healthcare Research and Quality’s ■ Postoperative Hemorrhage or Hematoma (PSI 9) Quality Indicators software tool, which includes the ■ Postoperative Physiologic and Metabolic Patient Safety Indicators (PSIs), is distributed free Derangements (PSI 10) of charge. The software can be used to help hos- pitals identify potential adverse events that might ■ Postoperative Respiratory Failure (PSI 11) need further study. The software programs can be ■ Postoperative Pulmonary Embolism or Deep Vein applied to any hospital inpatient administrative Thrombosis (PSI 12) data. This data is readily available and relatively inexpensive to use. ■ Postoperative Sepsis (PSI 13) ■ Postoperative Wound Dehiscence in Abdomino- In addition to the seven area-level PSIs discussed pelvic Surgical Patients (PSI 14) in this article, additional measures valid for use at the level of individual institutions are available, ■ Accidental Puncture or Laceration (PSI 15) including: ■ Transfusion Reaction (PSI 16) ■ Complications of Anesthesia (PSI 1) ■ Birth Trauma—Injury to Neonate (PSI 17) ■ Death in Low-Mortality DRGs (PSI 2) ■ Obstetric Trauma—Vaginal Delivery with Instrument (PSI 18) ■ Decubitus Ulcer (PSI 3) ■ Obstetric Trauma—Vaginal Delivery without ■ Failure to Rescue (PSI 4) Instrument (PSI 19) ■ Foreign Body Left during Procedure (PSI 5) ■ Obstetric Trauma—Cesarean Delivery (PSI 20) ■ Iatrogenic Pneumothorax (PSI 6) The software is available in SAS® and Microsoft ■ Selected Infections due to Medical Care (PSI 7) Windows® formats. User guides and technical documentation are available. Visit http://www. ■ Postoperative Hip Fracture (PSI 8) qualityindicators.ahrq.gov/software.htm. differences of opinion or interpretation. This analysis staff, bears no responsibility or liability for the results was not prepared by PHC4. This analysis was done of the analysis, which are solely the opinion of the by the Authority. PHC4, including its agents and Authority. Vol. 6, No. 4—December 2009 REPRINTED ARTICLE - ©2009 Pennsylvania Patient Safety Authority Page 125 PENNSYLVANIA PATIENT SAFETY ADVISORY This article is reprinted from the Pennsylvania Patient Safety Advisory, Vol. 6, No. 4—December 2009. The Advisory is a publication of the Pennsylvania Patient Safety Authority, produced by ECRI Institute and ISMP under contract to the Authority. Copyright 2009 by the Pennsylvania Patient Safety Authority. This publication may be reprinted and distributed without restriction, provided it is printed or distributed in its entirety and without alteration. Individual articles may be reprinted in their entirety and without alteration provided the source is clearly attributed. This publication is disseminated via e-mail. To subscribe, go to https://www.papsrs.state.pa.us/ Workflow/MailingListAddition.aspx. To see other articles or issues of the Advisory, visit our Web site at http://www.patientsafetyauthority.org. Click on “Patient Safety Advisories” in the left-hand menu bar. THE PENNSYLVANIA PATIENT SAFETY AUTHORITY AND ITS CONTRACTORS The Pennsylvania Patient Safety Authority is an independent state agency created by Act 13 of 2002, the Medical Care Availability and Reduction of Error (“Mcare”) Act. Consistent with Act 13, ECRI Institute, as contractor for the Authority, is issuing this publication to advise medical facilities of immediate changes that can be instituted to reduce Serious Events and Incidents. For more information about the Pennsylvania Patient Safety Authority, see the Authority’s Web An Independent Agency of the Commonwealth of Pennsylvania site at http://www.patientsafetyauthority.org. ECRI Institute, a nonprofit organization, dedicates itself to bringing the discipline of applied scientific research in healthcare to uncover the best approaches to improving patient care. As pioneers in this science for nearly 40 years, ECRI Institute marries experience and independence with the objectivity of evidence-based research. More than 5,000 healthcare organizations worldwide rely on ECRI Institute’s expertise in patient safety improvement, risk and quality management, and healthcare processes, devices, procedures and drug technology. The Institute for Safe Medication Practices (ISMP) is an independent, nonprofit organization dedicated solely to medication error prevention and safe medication use. ISMP provides recommendations for the safe use of medications to the healthcare community including healthcare professionals, government agencies, accrediting organizations, and consumers. ISMP’s efforts are built on a nonpunitive approach and systems-based solutions.