Department of Health and Human Services OFFICE OF INSPECTOR GENERAL AS FUNDING FOR BPA RESEARCH INCREASED, NIEHS FOLLOWED ITS PEER REVIEW PROCESS WHILE ALSO EXERCISING ITS DISCRETION Daniel R. Levinson Inspector General August 2017 OEI-01-15-00150 Report in Brief August 2017 OEI-01-15-00150 As Funding for BPA Research Increased, NIEHS Why OIG Did This Review Key Takeaway Followed Its Peer Review Process While Also Bisphenol A (BPA), a chemical Exercising Its Discretion NIEHS funding for BPA research often used to produce food and drink packaging, has been linked to What OIG Found increased significantly as it adverse health conditions, Between FYs 2000 and 2015, NIEHS funding for prioritized its research to including cancer. The National BPA research increased significantly. NIEHS’s assess BPA’s link to adverse Institute of Environmental Health BPA grants were concentrated among few health conditions. Although Sciences (NIEHS) funds studies that institutions and researchers. NIEHS used NIEHS met the peer review test substances, including BPA, for targeted announcements to fund about one- process requirements for all carcinogenicity and other harmful fifth of BPA and other grants. NIEHS met basic grants, it used its discretion to effects. The Office of Inspector requirements of its peer review process for all fund applications with less General (OIG) received a grants. NIEHS used its discretion to fund 14 favorable scores than congressional request to review percent of BPA grants out of order as compared competing applications for the extent to which NIEHS funds to 4 percent of other grants from FYs 2010- 14 percent of BPA grants, research on the safety of BPA and 2015. Finally, FDA and CDC have limited roles versus 4 percent of other, non- the processes NIEHS uses in working with NIEHS on BPA research, although BPA grants. planning and funding that FDA contributed to the NIEHS-led Consortium research. Linking Academic and Regulatory Insights on BPA Toxicity. How OIG Did This Review Funding for BPA Research As A Percentage of Overall NIEHS Research To determine whether NIEHS (FYs 2000-2015) followed its peer review process, 12% we analyzed and compared 10% 101 BPA grants and 105 other, non-BPA grants awarded during 8% FYs 2010-2015. For both sets of 6% grants, we analyzed relevant 4% funding announcements, summary 2% statements, funding documents, 0% 2004 2000 2001 2002 2003 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 and justifications for funding, if applicable. Finally, we interviewed staff from NIEHS, the Food and What OIG Concludes Drug Administration (FDA), and NIEHS may prioritize its research, as it did with BPA. NIEHS’s peer review the Centers for Disease Control process is intended to ensure that applications submitted for funding are and Prevention (CDC) about their evaluated fairly, equitably, in a timely manner, and without bias, and NIEHS BPA research and processes. followed that process. NIEHS’s procedures also give it the discretion to fund applications with less favorable impact scores ahead of competing applications by justifying them in writing. Such discretion is allowed and enables NIEHS to be responsive to emerging threats to public health; however, applying it frequently or disproportionately in one research area may create an appearance of impropriety. Full report can be found at http://oig.hhs.gov/oei/reports/oei-01-15-00150.asp TABLE OF CONTENTS Objectives ....................................................................................................1 Background ..................................................................................................1 Methodology ................................................................................................7 Findings......................................................................................................10 Between FYs 2000 and 2015, NIEHS funding for BPA research increased significantly ...................................................................10 NIEHS’s BPA grants were concentrated among few institutions and researchers .............................................................................13 NIEHS used targeted announcements to fund about one-fifth of BPA and other grants......................................................................15 Although a few grant summary statements did not include all required information, NIEHS met basic requirements of its peer review process from FYs 2010-2015 ............................................15 NIEHS used its discretion to fund 14 percent of BPA grants out of order as compared to 4 percent of other grants from FYs 2010-2015...............................................................................17 FDA and CDC have limited roles working with NIEHS on BPA research, although FDA contributed to the CLARITY research program ..........................................................................................18 Conclusion .................................................................................................19 Appendices .................................................................................................20 A: Point Estimates and Confidence Intervals ...............................20 B: Results of Independent Group T-Tests for Grant Types ...........21 C: NIEHS Extramural Funding and BPA Research Funding by Fiscal Year ......................................................................................22 Acknowledgments......................................................................................23 OBJECTIVES 1. To determine the extent to which the National Institute of Environmental Health Sciences (NIEHS) funded extramural research on the safety of Bisphenol A (BPA) from fiscal years (FYs) 2000-2015. 2. To determine the extent to which NIEHS followed grant application processes related to peer review when awarding funds for BPA extramural research from FYs 2010-2015. 3. To determine the extent to which other Department of Health and Human Services (the Department) programs and agencies played a role in planning, funding, and conducting NIEHS’s BPA research. BACKGROUND BPA, a chemical often used to produce food and drink packaging, has been linked to a variety of adverse health conditions, including cancer.1 NIEHS funds studies that test substances, including BPA, for carcinogenicity and other harmful biological effects. The Office of Inspector General (OIG) received a congressional request to review the extent to which NIEHS funds research on the safety of BPA and the processes NIEHS used in planning and funding that research. BPA BPA is a widely produced chemical used to make polycarbonate plastics and epoxy resins. Polycarbonate plastics are found in some food and drink packaging, such as water and infant feeding bottles. Epoxy resins are used to coat metal products such as food cans, bottle tops, and water supply pipes. Other products such as cash register receipts and dental sealants may also include BPA.2 Because manufacturers use BPA in food and beverage packaging, the Food and Drug Administration (FDA) regulates its use.3 ____________________________________________________________ 1 Birnbaum, Linda S., et al., Consortium-Based Science: The NIEHS’s Prolonged, Collaborative Approach to Assessing the Health Effects of Bisphenol A, Environmental Health Perspectives, Volume 120, Issue 12, December 2012. 2 National Toxicology Program, Center for the Evaluation of Risks to the Human Reproductive System NTP-CERHR, Monograph on the Potential Human Reproductive and Developmental Effects of Bisphenol A, September 2008 NIH Publication 08-5994 pg. 1. 3 Federal Food, Drug, and Cosmetic Act § 409 (21 U.S.C. § 348) and 21 CFR §§ 174.5, 177.1595, and 177.1555. As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 1 Discretion (OEI-01-15-00150) BPA is found throughout the environment and in the human body. Humans may consume BPA that leaches from containers into food or drink, or absorb it through skin contact or inhalation. A 2009 Centers for Disease Control and Prevention (CDC) report found that 93 percent of a sample of adults and children had traces of BPA in their urine.4 BPA is classified as an endocrine disruptor, and some research has linked it to a variety of adverse health conditions, including infertility, weight gain, early onset puberty, diabetes, and cancer.5 Research has yet to determine safe levels of BPA exposure or the health effects of cumulative exposure.6 NIEHS The Public Health Service Act of 1944 authorizes the Secretary of the Department to conduct and financially support research and testing of substances for carcinogenicity, teratogenicity, mutagenicity, and other harmful biological effects.7 Within the National Institutes of Health (NIH), this authority is largely carried out by NIEHS. To fulfill its mission, NIEHS funds extramural studies through a grant- making process.8, 9 NIEHS’s grant funding strategy allows for targeted and nontargeted grants. For targeted grants, NIEHS issues announcements seeking principal investigators (researchers) to submit applications to fulfill specific research objectives.10 NIEHS also accepts nontargeted grant applications, which may be for any research topic that falls under NIEHS’s purview.11 ____________________________________________________________ 4 CDC, Fourth National Report on Human Exposure to Environmental Chemicals, 2009. 5 Birnbaum, Linda S., et al. 6 CDC Factsheet Bisphenol A (BPA), 2010. Accessed at https://www.cdc.gov/biomonitoring/pdf/BisphenolA_factSheet.pdf, on July 31, 2017. 7 Public Health Service Act § 301 (42 USC § 241). 8 NIEHS also conducts intramural research. 9 NIEHS has many different mechanisms for funding research. For the purposes of this report, “extramural grants” applies to research and cooperative grants. 10 NIH’s policy manual refers to these announcements as requests for applications, requests for proposals, or program announcements. See NIH Policy Manual, 54110 Program Announcements and Requests for Applications, 10/1/1994, p. 5. 11 NIEHS refers to such announcements as “unsolicited.” As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 2 Discretion (OEI-01-15-00150) NIEHS’s Division of Extramural Research and Training (DERT) provides administrative guidance to NIEHS regarding extramural grants. This division also acts as a liaison between NIEHS and the research community by guiding researchers through and overseeing the grant application process.12 NIEHS generally funds research grants for more than 1 year. After the first award year, NIEHS funds grants for additional years (renewals), as stated in the original grant applications. In some cases, researchers incur additional, unexpected costs while conducting the research. In these instances, NIEHS may provide supplemental funding should researchers apply for it. NIH Peer Review Process NIH requires all of its institutes, including NIEHS, to follow its peer review process.13 Research applications for grants undergo two levels of review: the initial peer review and review by an advisory council.14 This peer review process “is intended to ensure that applications for funding submitted to NIH are evaluated on the basis of a process that is fair, equitable, timely, and conducted in a manner that strives to eliminate bias.”15 First level of review. A scientific review group (SRG) conducts the initial review.16 An SRG comprises mostly non-Federal scientists with the necessary expertise to evaluate the scientific merit of the grant applications. To manage conflicts of interest, NIEHS has processes in place to ensure that individual reviewers do not review grant applications from the institution where they are employed.17 The SRG evaluates each application on the basis of the criteria included in the announcement. These criteria generally include scored elements, such as the significance ____________________________________________________________ 12 NIEHS, DERT overview. Accessed at https://www.niehs.nih.gov/research/supported/dert/ on August 1, 2017. 13 Although NIH requires all of its institutes to follow its peer review process, this report is focused on NIEHS. For the sake of clarity and readability, this report will refer to this process as a NIEHS requirement. 14 Public Health Service Act § 492 (42 USC § 289a) and 42 C.F.R. § 52h. 15 NIH Grant Policy Statement (10/1/2013) Part 1, Page I-51. Accessed at http://grants.nih.gov/grants/policy/nihgps_2013/ on August 1, 2017. 16 NIH’s Center for Scientific Research conducted the first level of review for 48 of 206 grant files in our review. 17 NIH Grant Policy Manual, 4204-204B–Peer Review Process, 9/17/2013, Section V.C.b.6. As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 3 Discretion (OEI-01-15-00150) of the research and soundness of the study design, as well as nonscored items, such as human subjects’ protections.18, 19 DERT staff use the SRG reviews to calculate an impact score and produce grant summary statements for each application. After DERT staff calculate the impact score, they rank the applications in slightly different ways depending on whether they were for a targeted or nontargeted funding opportunity announcement:  For applications to targeted funding opportunity announcements, staff rank the applications by the impact score assigned during the SRG.  For applications to nontargeted funding opportunity announcements, DERT staff rank the applications by percentile on the basis of their impact scores.20 DERT staff package and rank both the targeted and nontargeted applications together for the second level of peer review and approval. NIEHS officials then review the ranked applications and compare them to NIEHS’s programmatic needs. At this point, NIEHS’s peer review process allows for discretion. A designated NIEHS official may choose to move up a less favorably ranked application for funding consideration. For applications to targeted funding announcements, this is called “funding out of order.” For applications to nontargeted funding announcements, this is called “raising to the payline.” (Hereinafter, we will use the term “funding/funded out of order” when referring to either instance.) For both targeted and nontargeted announcements, a designated NIEHS official must write a justification for the decision to fund out of order. The applications that are approved for funding out of order then continue to the second level of review with the other applications (see Exhibit 1). ____________________________________________________________ 18 NIH Peer Review Process. Accessed at http://grants.nih.gov/grants/peer_review_process.htm, on August 1, 2017 19 In reviewing a grant application, one or more of the SRG reviewers may find a non- scored criterion to be unacceptable. In these instances, the reviewers determined that the grant applicant may not have adequately described how the criterion will be met, or a description may be missing from the application. To receive funding, the applicant must address concerns regarding the criterion. 20 Starting in FY 2013, NIEHS set a “payline” based on these percentages. The “payline” applies only to applications for certain nontargeted funding opportunity announcements. In this report, we refer to applications funded by raising to payline as funded “out of order.” As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 4 Discretion (OEI-01-15-00150) Exhibit 1: Example of a Grant With a Less Favorable Score Being Funded (funding out of order) Source: OIG analysis of NIH’s policies and procedures. Second level of review. The National Advisory Environmental Health Sciences Council (the advisory council), conducts the second level of peer review.21 The advisory council is chaired by the NIEHS Director and comprises 18 non-Federal scientists and public representatives, and several ex officio members.22, 23 The advisory council assesses the SRG review and provides concurrence that the review occurred according to NIH policy; that the roster of ____________________________________________________________ 21 Public Health Service Act § 492 (42 U.S.C. § 289a) and 42 CFR § 52h. 22 “Special Government Employee” is defined by 18 U.S.C. § 202(a). 23 Current members of the advisory council accessed at http://www.niehs.nih.gov/about/boards/naehsc/roster/ on August 1, 2017. As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 5 Discretion (OEI-01-15-00150) reviewers held the proper expertise; and that the narratives match the scores SRG members gave for each application. Generally, the advisory council receives and approves applications “en bloc,” but the Director has the final say on which research applications receive funding and may choose not to fund any individual application. The advisory council’s deliberations over the package of grants are private.24 See Exhibit 2 for a summary of the peer review process. Exhibit 2: Overview of the Peer Review Process Source: OIG analysis of NIH’s policies and procedures. National Toxicology Program The National Toxicology Program (NTP), led by NIEHS, is an interagency group that coordinates toxicity research and testing across the Department. NTP works to strengthen the science base in toxicology, develop and validate improved testing methods, and provide information about potentially toxic chemicals to health regulatory and research agencies, scientific and medical communities, and the public. As the lead agency, NIEHS is largely responsible for NTP’s activities. ____________________________________________________________ 24 Government in the Sunshine Act § 552b(c) (5 USC § 552b(c)). As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 6 Discretion (OEI-01-15-00150) NTP broadly solicits nominations of chemicals for evaluation from the public and private sectors. It selected BPA for an evaluation completed in 2007 because of the:  potential for human exposure from use and occurrence in the environment;  extent of public concern;  production volume; and  extensive database on animal reproductive and developmental toxicity studies.25 As a result of its evaluation, NTP called for more BPA research.26 In 2011, NIEHS and FDA jointly started the Consortium Linking Academic and Regulatory Insights on BPA Toxicity (CLARITY) study, which is a collaboration between NIEHS, FDA, and researchers to research BPA safety. NIEHS provided the funding for the CLARITY study. FDA and CDC are also members of the NTP. Their roles are to provide feedback on research proposals and to provide scientific expertise, when requested. Both agencies conduct BPA research as well. FDA conducts toxicological research on BPA in lab rats through an interagency agreement with NTP. CDC has conducted occupational research on workers exposed to high levels of BPA and other chemicals. METHODOLOGY Scope We analyzed NIEHS grants for extramural research on BPA safety, including NTP’s CLARITY-BPA cooperative grants, for FYs 2000-2015. Data Collection BPA Grants. We requested from NIH a list of all NIEHS extramural grants related to BPA research conducted from FYs 2000-2015. NIH ____________________________________________________________ 25 National Toxicology Program, Center for the Evaluation of Risks to the Human Reproductive System NTP-CERHR, Monograph on the Potential Human Reproductive and Developmental Effects of Bisphenol A, September 2008 NIH Publication 08-5994 pg. vii. 26 Ibid., pgs. 21, 25, 31, and 38. As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 7 Discretion (OEI-01-15-00150) identified these grants using its internal database.27 NIH identified 562 BPA-related grants during this period. For these grants, we also requested information about the institutions, researchers, and total funding amount. We requested additional information for a subset of these grants. For each of the 101 BPA grants NIEHS funded between FYs 2010-2015, we requested the related funding opportunity announcements, summary statements, and documents related to the Advisory Council review. For any grants that NIEHS funded out of order, we also requested the justification for doing so. All Other Grants. We selected a simple random sample of 105 grants from the population of 1,341 grants not related to BPA research that NIEHS funded from FYs 2010-2015. These grants included such topics as the role of pesticides in childhood cancer and the role of occupational exposures in cancer of the blood. Such a sample allowed us to make projections to the whole population of grants and make comparisons between BPA and all other grants funded from FYs 2010-2015. See Appendix A for point estimates, confidence intervals, and p-values. For each of these 105 grants, we requested the related funding opportunity announcements, summary statements, and documents related to the advisory council review. For any grants that NIEHS awarded out of order, we also requested the justification for doing so. Structured Interviews. We conducted structured interviews with staff from NIEHS about NIEHS’ BPA research, as well as its policies and procedures for awarding grants. We also conducted structured interviews with staff from FDA and CDC staff to determine the extent of their involvement in NIEHS’s BPA research. Data Analysis BPA Grants. For the 562 BPA grants funded from FYs 2000-2015, we analyzed the amount of funding to identify any trends, and the institutions and researchers to identify which were the most common. For the subset of 101 BPA grants funded from FYs 2010-2015, we analyzed the related documents to determine whether they included ____________________________________________________________ 27 The list of BPA grants included the following types of awards: NIEHS-funded, competitive and noncompetitive, Superfund, and American Recovery and Reinvestment Act. NIEHS awards “competitive” grants through the peer review process, as described. “Noncompetitive” grants are renewals after the first year of an awarded grant. The list of BPA grants excluded the following types of awards: duplications, withdrawn applications, loan repayment applications, and applications funded by other NIH institutes. As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 8 Discretion (OEI-01-15-00150) records of initial and second-level peer reviews, such as complete summary statements and justifications for funding grants out of order. All Other Grants. For the sample of 105 grants funded from FYs 2010-2015, we analyzed the related documents to determine whether they included records of initial and second-level peer reviews, such as complete summary statements and justifications for funding grants out of order. We then compared the BPA grants with the other grants to determine whether there is any statistical difference between the two groups with respect to their completeness and justifications. Limitations We did not distinguish the role that NIH’s Center for Scientific Research played in conducting the first level of peer review for 48 of the 206 grants applications within our sample for analysis. We did not independently confirm the reasons that NIEHS chose to award or not award any grants. We did not review the scientific rationales included in the justifications when NIEHS funded grants out of order. Examining the factors that led to grant funding concentrations among institutions and researchers was also outside the scope of our review. Standards We conducted this study in accordance with the Quality Standards for Inspection and Evaluation issued by the Council of the Inspectors General on Integrity and Efficiency. As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 9 Discretion (OEI-01-15-00150) FINDINGS Between FYs 2000 and 2015, NIEHS funding for BPA research increased significantly NIEHS funded $186 million in BPA research from FYs 2000-2015 (see Exhibit 3). Over 80 percent ($152 million) of BPA spending during this period occurred since FY 2009 when BPA safety became a public concern and NIEHS prioritized BPA research. Exhibit 3: NIEHS Funding For BPA Research, FYs 2000-2015 Source: OIG analysis of NIH’s internal database. NIEHS’s overall research budget ranged from $225 million to $342 million between 2000 and 2015.28 From FYs 2000-2005, NIEHS funding for BPA research remained steady at about $2 million per year, less than 1 percent of the institute’s extramural research budget (see Exhibits 3 and 4). By FY 2015, BPA research accounted for $30 million, or 9.5 percent, of NIEHS’s extramural budget.29 (See Appendix C for the details of NIEHS’s extramural budget and the percentage spent on BPA research by fiscal year.) ____________________________________________________________ 28 NIEHS increased BPA funding after it received $168 million of American Reinvestment and Recovery Act funds and $19 million for the Superfund Research Program over FYs 2009 and 2010. 29 NIEHS awarded a $9 million grant in FY 2015, skewing the data. As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 10 Discretion (OEI-01-15-00150) Exhibit 4: Percentage of NIEHS’s Extramural Research Funds Awarded for BPA Research FYs 2000-2015 Source: OIG analysis of NIH’s internal database. Most of NIEHS’s BPA grant activity consisted of renewing and supplementing existing grants rather than awarding new grants. However, in line with the increase in BPA funding in 2009, NIEHS funded 20 new grants in that year for about $12 million. NIEHS funded the most grants (both new awards and renewals/supplemental) in 2010 when it issued 20 and 58, respectively, for about $25 million (see Exhibits 5 and 6). As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 11 Discretion (OEI-01-15-00150) Exhibit 5: Number of New and Renewal/Supplemental NIEHS BPA Grants From FYs 2000-2015 Source: OIG analysis of NIH’s internal database. Exhibit 6: NIEHS’s Funding For New and Renewal/Supplemental BPA Grants From FYs 2000-2015 Source: OIG analysis of NIH’s internal database. * The difference in the sum of New and Renewal/Supplemental grants funding compared to total grants funding is due to rounding. As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 12 Discretion (OEI-01-15-00150) NIEHS’s BPA grants were concentrated among few institutions and researchers Of the 87 institutions that received NIEHS grants for BPA research during FYs 2000-2015, the highest 10 percent (9 institutions) in terms of dollars received accounted for nearly half of the total grant dollars, with about $92 million. The second-highest 10 percent of institutions received about one-third of that amount, with about $31 million (see Exhibit 7). Exhibit 7: Concentration of NIEHS BPA Funding Among Institutions, FYs 2000-2015 Source: OIG analysis of NIH’s internal database. The mean grant value for the highest 10 percent of institutions was about the same as for the second-highest 10 percent, at about $380,000. However, the highest 10 percent received about three times as many grants (see Exhibit 8). As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 13 Discretion (OEI-01-15-00150) Exhibit 8: Distribution of NIEHS BPA Grants Among Institutions, FYs 2000-2015 Total Mean Minimum Maximum Median Mean grant grant dollars dollars dollars dollars dollars Grant count per received received received received received count institution per grant per grant per grant per grant Top 10 percent (N =9) of institutions by BPA grant dollars $91.7 M 239 26.6 $7,878 $ 9.5 M $324,092 $383,878 Second 10 percent (N =9) of institutions by BPA grant dollars $31.2M 82 9.1 $1,000 $ 1.2 M $367,715 $380,839 Remaining 80 percent (N= 69) of institutions by BPA grant dollars $63.0 M 241 3.5 $53,088 $1.5 M $225,000 $261,022 Source: OIG analysis of NIH’s internal database. NIEHS’s BPA research dollars are concentrated among researchers as well. Of the 140 researchers that received NIEHS grants for BPA research during FYs 2000-2015, the top 10 percent (14 researchers) received more than 40 percent of the total grant dollars. Furthermore, when combined with the next 10 percent (14 researchers), the top 20 percent of researchers received about 60 percent of the total grant dollars. Funding concentration among institutions and researchers may be explained by several factors. For example, institutions might have specialized laboratories appropriate for BPA work. Similarly, researchers with expertise in the topic may be concentrated at certain institutions and skilled at BPA research and grant writing. As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 14 Discretion (OEI-01-15-00150) NIEHS used targeted announcements to fund about one-fifth of BPA and other grants NIEHS used targeted funding announcements for about 22 percent of BPA and all other grants (see Appendix A).30 Targeted funding announcements specify the research topic being funded. NIEHS ranks applications for targeted funding announcements against other applications with research proposals on the same topic. In contrast, nontargeted announcement are open to applications for any research topics that fall under NIEHS’s purview. NIEHS ranks applications to these announcements against other applications that may cover a range of research topics. Using targeted funding announcements ensures that NIEHS has flexibility to choose its research priorities and to direct its research budget. Although a few grant summary statements did not include all required information, NIEHS met basic requirements of its peer review process from FYs 2010-2015 One hundred percent of BPA grants and all other grants went through the SRG review and advisory council review processes.31 NIEHS requires all grant applications to go through the peer review process, which is intended to ensure that applications are evaluated fairly, equitably, in a timely manner, and without bias. NIEHS convened its council meetings each quarter during FYs 2010-2015. The council received the packages of grants, ranked by DERT on the basis of SRG’s first level of peer review. During this time, 139 SRGs reviewed the 206 grants in our review. Ninety-eight SRGs reviewed only one grant application NIEHS prepared summary statements reflective of their SRG analyses for 100 percent of the BPA and all other grants (see Appendix A). These summary statements included impact scores and SRG rosters, and reflected the criteria found within the funding announcements. ____________________________________________________________ 30 The percentage difference between BPA and other grants was not statistically significant at the 95 percent confidence interval. 31 Although all 105 other grants in our sample went through SRG review and advisory council review, we are not certain—because of sampling error—that 100 percent of the 1,341 other grants not related to BPA research that NIEHS funded from FY 2010–2015 went through SRG review and advisory council review. However, we are at least 95 percent confident that more than 94.3 percent of that population, and possibly 100 percent, went through SRG review and advisory council review. As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 15 Discretion (OEI-01-15-00150) Summary statements for 13 percent of the grant reviews included one or more nonscored criteria deemed to be unacceptable by SRG reviewers.32 Nonscored criteria include human subjects’ protections plans, biohazard plans, and vertebrate welfare. Reviewers may consider nonscored criteria, such as an unacceptable human subjects’ protection plan, when determining an overall impact score. NIEHS staff told us they have a process that requires grantees to address unacceptable criteria before their grants are funded.33 Although the majority of summary statements for all grants met the relevant requirements, some did not: 6 percent of summary statements did not include all required information.34 One NIEHS staff member told us that reviewers are instructed to provide the scores and narratives in their reviews, but sometimes they provide less than what is requested. Furthermore, in two BPA grants the summary statements included only two reviewers, although three are required. ____________________________________________________________ 32 The percentage difference between BPA and other grants was not statistically significant at the 95 percent confidence interval. 33 Verifying that grant applications with unacceptable, nonscored criteria were corrected by the applicants was beyond the scope of our review. 34 The percentage difference between BPA and other grants was not statistically significant at the 95 percent confidence interval. As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 16 Discretion (OEI-01-15-00150) NIEHS used its discretion to fund 14 percent of BPA grants out of order as compared to 4 percent of other grants from FYs 2010 to 2015 NIEHS was more likely to Excerpts From a NIEHS Official’s award BPA grants to Justification For Out-of-Order applications with less Funding For a BPA Grant favorable impact scores compared to their “This is [a] novel and important area competition than it was for that could lead to new endpoints for other grant types. NIEHS EPI [epidemiological] studies and new awarded 14 percent of the mechanistic insights and pathways for BPA grants, compared to the effects of BPA. No one else is studying these pathways and if not 4 percent for other grants, funded this design and expertise will be with less favorable scores lost.” than their competition (see Appendix A). Such “Reviewers noted [the] study was discretion is permitted if an significantly improved[,] including NIEHS official justifies the preliminary data showing BPA affects award by including both a [sic] on placental function. The central scientific and subjective concern that reduced [the] score argument for funding despite somewhat was that there is currently a less favorable SRG score. no data to show BPA affects [sic] on placental function.” Most of the justifications we reviewed included opinions “While we have a significant portfolio about how valuable the on BPA this proposal is novel and research was and in some addresses an endpoint not yet cases specifically stated that addressed….I would bring [this] world not funding the grant would renowned placental and stem cell be a missed opportunity. See expertise into the NIEHS portfolio.” excerpts from one of these justifications in the box to the right. In nine instances, the justifications disagreed with the conclusions of a specific member of the SRG or the SRG’s review in general. As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 17 Discretion (OEI-01-15-00150) FDA and CDC have limited roles working with NIEHS on BPA research, although FDA contributed to the CLARITY research program FDA and CDC officials told us the two agencies have no formal role in NIEHS’s planning and funding of BPA research. Both agencies conduct their own research on BPA, as they deem appropriate. For example, FDA has conducted BPA literature reviews and laboratory research that examined chronic exposure of BPA to a specific breed of rats. NIEHS, in conjunction with FDA, required grantees to use these rats for the CLARITY program. CDC conducts two studies annually on the chemical toxicity of BPA. As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 18 Discretion (OEI-01-15-00150) CONCLUSION When a widely used chemical such as BPA is linked to a variety of adverse health effects, it is within NIEHS’s mission to prioritize its research to determine, to the best of its ability, whether safe levels of exposure exist. From 2000 to 2015, NIEHS increased its funding for BPA research significantly, both in total dollars and as a percentage of its extramural research funding. NIEHS has significant discretion to choose research topics for directed funding through targeted funding announcements. NIEHS used targeted announcements for about one-fifth of BPA grants and non-BPA grants from FYs 2010-2015. NIEHS’s peer review process is intended to ensure that applications submitted for funding are evaluated fairly, equitably, in a timely manner, and without bias. In principle, all grant applications received for each announcement, whether for targeted or nontargeted funding announcements, are evaluated to ensure that NIEHS funds only the best research proposals. We found that NIEHS followed its peer review process. NIEHS’s procedures also give it the discretion to fund applications with less favorable impact scores ahead of competing applications by justifying them in writing. NIEHS used that discretion to fund 14 percent of BPA grants and 4 percent of other, non-BPA grants out of order. Such discretion is allowed and enables NIEHS to be responsive to emerging threats to public health; however, applying it frequently or disproportionately in one research area may create an appearance of impropriety. As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 19 Discretion (OEI-01-15-00150) APPENDIX A Point Estimates and Confidence Intervals (all estimates are for BPA and other grants funded FYs 2010-2015) 95-percent Estimate Sample Size Percent Confidence Description Interval Grants that used targeted funding 206 21.21.21.6 % 15.4% - 29.5% announcements. BPA grants and all other grants that went through the SRG review 206 100% 94.3% - 100% and advisory council review processes. NIEHS prepared summary statements reflective of their 206 100% 94.3% - 100% SRG analyses for BPA and other grants. Grant reviews that included one of more non- 206 12.6% 7.9% - 19.5% scored criteria deemed to be unacceptable. Grant reviews that did not include all 206 5.6% 2.7% - 11.2% required information. As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 20 Discretion (OEI-01-15-00150) APPENDIX B Results of Independent Group T-Tests for Grant Types Grant Description Estimate p-value Type Difference between NIEHS- awarded grants with less BPA 13.9% 0.0115 favorable scores Other 3.8% than their competition, by grant type. As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 21 Discretion (OEI-01-15-00150) APPENDIX C NIEHS Extramural Funding and BPA Research Funding by Fiscal Year Funding for all extramural Funding for research BPA BPA Research As a (in research (in Percentage of FY millions) millions) Overall Research 2000 $225.2 $1.6 0.7% 2001 $259.8 $1.7 0.7% 2002 $291.8 $2.2 0.8% 2003 $306.0 $2.7 0.9% 2004 $309.3 $2.3 0.7% 2005 $308.6 $2.6 0.8% 2006 $299.2 $6.3 2.1% 2007 $312.0 $6.9 2.2% 2008 $311.5 $8.1 2.6% 2009 $319.7 $23.7 7.4% 2010 $341.6 $25.5 7.5% 2011 $327.0 $15.3 4.7% 2012 $328.6 $17.1 5.2% 2013 $304.7 $19.8 6.5% 2014 $312.5 $20.1 6.4% 2015 $311.7 $29.7 9.5% Total $4,869.2 $186 3.8% Source: NIEHS’s internal database. As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 22 Discretion (OEI-01-15-00150) ACKNOWLEDGMENTS Chris Galvin served as team leader for this study, and Ivan Troy served as lead analyst. Office of Evaluation and Inspections staff who provided support include Althea Hosein, Sherri Weinstein, and Christine Moritz. We would also like to acknowledge the contributions of other Office of Inspector General staff, including Jessica Swanstrom. We would also like to acknowledge the contributions of Russell Hereford, former Deputy Regional Inspector General for Evaluation and Inspections in the Boston regional office, who has since retired. This report was prepared under the direction of Joyce Greenleaf, Regional Inspector General for Evaluation and Inspections in the Boston regional office, and Kenneth Price, Deputy Regional Inspector General. To obtain additional information concerning this report or to obtain copies, contact the Office of Public Affairs at Public.Affairs@oig.hhs.gov. As Funding for BPA Research Increased, NIEHS Followed Its Peer Review Process While Also Exercising Its 23 Discretion (OEI-01-15-00150) Office of Inspector General http://oig.hhs.gov The mission of the Office of Inspector General (OIG), as mandated by Public Law 95452, as amended, is to protect the integrity of the Department of Health and Human Services (HHS) programs, as well as the health and welfare of individuals served by those programs. This statutory mission is carried out through a nationwide network of audits, investigations, and inspections conducted by the following operating components: Office of Audit Services The Office of Audit Services (OAS) provides auditing services for HHS, either by conducting audits with its own audit resources or by overseeing audit work done by others. Audits examine the performance of HHS programs and/or its grantees and contractors in carrying out their respective responsibilities and are intended to provide independent assessments of HHS programs and operations. These assessments help reduce waste, abuse, and mismanagement and promote economy and efficiency throughout HHS. Office of Evaluation and Inspections The Office of Evaluation and Inspections (OEI) conducts national evaluations to provide HHS, Congress, and the public with timely, useful, and reliable information on significant issues. These evaluations focus on preventing fraud, waste, or abuse and promoting economy, efficiency, and effectiveness of departmental programs. To promote impact, OEI reports also present practical recommendations for improving program operations. Office of Investigations The Office of Investigations (OI) conducts criminal, civil, and administrative investigations of fraud and misconduct related to HHS programs, operations, and individuals. With investigators working in all 50 States and the District of Columbia, OI utilizes its resources by actively coordinating with the Department of Justice and other Federal, State, and local law enforcement authorities. The investigative efforts of OI often lead to criminal convictions, administrative sanctions, and/or civil monetary penalties. Office of Counsel to the Inspector General The Office of Counsel to the Inspector General (OCIG) provides general legal services to OIG, rendering advice and opinions on HHS programs and operations and providing all legal support for OIG’s internal operations. OCIG represents OIG in all civil and administrative fraud and abuse cases involving HHS programs, including False Claims Act, program exclusion, and civil monetary penalty cases. In connection with these cases, OCIG also negotiates and monitors corporate integrity agreements. OCIG renders advisory opinions, issues compliance program guidance, publishes fraud alerts, and provides other guidance to the health care industry concerning the anti-kickback statute and other OIG enforcement authorities.