United States Government Accountability Office Report to Congressional Committees VA HEALTH CARE April 2021 Efforts Needed to Ensure Effective Use and Appropriate Staffing of Suicide Prevention Teams GAO-21-326 April 2021 VA HEALTH CARE Efforts Needed to Ensure Effective Use and Appropriate Staffing of Suicide Prevention Teams Highlights of GAO-21-326, a report to congressional committees Why GAO Did This Study What GAO Found Compared to the general population, The Department of Veterans Affairs’ (VA) Veterans Health Administration (VHA) veterans suffer a disproportionately uses suicide prevention teams at local facilities to implement its Suicide higher rate of suicide. VA has stated Prevention Program, which was mandated by law in 2007. As VHA has added that suicide prevention is the agency’s new initiatives to the program since its inception—such as Risk ID, a top clinical priority. At its local medical standardized suicide risk identification process that it initiated in 2018—local facilities, VHA employs suicide teams’ activities have increased, in particular as more veterans at risk of suicide prevention teams, which include are identified. This has created challenges, according to the selected teams in coordinators—staff with a full-time GAO’s review. For example, team staff at one facility said they have experienced commitment to suicide prevention burnout and turnover due to new initiatives and a large caseload of veterans at activities—and case managers. high risk for suicide. Staff from a team at another facility said they typically The Support for Suicide Prevention manage a caseload of about 150-200 veterans identified as being high risk for Coordinators Act included a provision suicide. According to VHA guidance, teams are expected to engage in a number for GAO to review, among other of activities related to these veterans, including regular interactions. Team staff at issues, the responsibilities and one facility noted the importance of these interactions in building relationships workload of suicide prevention with veterans at risk of suicide, so veterans know they can reach out to their local coordinators. This report examines teams when they need help. how VHA (1) uses local suicide prevention teams, and (2) helps VHA’s Office of Mental Health and Suicide Prevention (OMHSP) determines determine facilities’ suicide prevention policy for and monitors the program. OMHSP officials said that they have made staffing needs. To do this work, GAO changes to their guidance and technical assistance, which may help address analyzed team staffing data, reviewed some of the challenges reported by teams. For example, in January 2021 VHA policies, and interviewed OMHSP OMHSP issued new guidance that consolidates information on teams’ activities, officials and team staff from five including those related to recent initiatives. In addition, in June 2020, OMHSP medical facilities. The facilities were initiated monthly “office hours” calls for teams to answer questions and provide selected for variation in composition of technical assistance beyond what is covered during other regular calls. suicide prevention teams, geographic location, and total number of patients, However, VHA has not conducted a comprehensive evaluation of local suicide among other factors. prevention teams, including an assessment of any challenges teams face in implementing VHA policies and the effects of program growth on workload. What GAO Recommends Without such an evaluation, VHA does not have a good understanding of how its GAO is making three various activities and initiatives are affecting teams, including any effects on the recommendations, including that VHA care teams provide veterans who may be at risk for suicide. Such an evaluation (1) conduct an evaluation of local would allow OMHSP to refine the guidance and other support it provides to local suicide prevention teams that includes teams to potentially reduce challenges they face. an identification of the effects of OMHSP uses a benchmark to help determine facilities’ suicide prevention program growth on workload, and (2) staffing needs. Although facilities are responsible for making their own staffing incorporate key practices for staffing model design into its determination of decisions, according to the benchmark, each facility should aim to have at least facilities’ suicide prevention staffing one coordinator or case manager for every 10,000 unique patients the facility needs. VA concurred with GAO’s serves. However, the benchmark may not accurately reflect facilities’ staffing recommendations and identified needs because it was not developed according to the key practices for staffing actions it is taking to address them. model design previously identified by GAO. For example, the benchmark does not account for the increasing workload of teams, such as the addition of activities related to new initiatives over time. Additionally, it does not account for risk factors, like suicide rates, that may vary among facilities. Because the benchmark was not developed following key practices, VHA may not be able to View GAO-21-326. For more information, contact Debra A. Draper at (202) 512-7114 or help facilities to appropriately determine their staffing needs. As a result, suicide draperd@gao.gov. prevention teams may be vulnerable to understaffing, which may leave facilities unable to meet veteran needs related to suicide prevention. United States Government Accountability Office Contents Letter 1 Background 4 VHA Uses Local Teams to Carry Out a Variety of Suicide Prevention Program Activities but Has Not Evaluated the Effect of Program Growth on Their Workload 9 VHA’s Benchmark for Suicide Prevention Staffing May Not Accurately Reflect Facilities’ Needs 14 Conclusions 20 Recommendations for Executive Action 20 Agency Comments 21 Appendix I Comments from the Department of Veterans Affairs 22 Appendix II GAO Contact and Staff Acknowledgments 25 Tables Table 1: Suicide Prevention Teams’ Responsibilities and Examples of Related Activities, as of January 2021 9 Abbreviations OMHSP Office of Mental Health and Suicide Prevention REACH VET Recovery Engagement and Coordination for Health – Veterans Enhanced Treatment VA Department of Veterans Affairs VHA Veterans Health Administration This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Page i GAO-21-326 VA Suicide Prevention Teams Letter 441 G St. N.W. Washington, DC 20548 April 5, 2021 The Honorable Jon Tester Chairman The Honorable Jerry Moran Ranking Member Committee on Veterans’ Affairs United States Senate The Honorable Mark Takano Chairman The Honorable Mike Bost Ranking Member Committee on Veterans’ Affairs House of Representatives Veterans are at higher risk for suicide than the general population, according to Department of Veterans Affairs’ (VA) data. In 2018, veterans accounted for 13.8 percent of all deaths by suicide among U.S. adults, despite constituting only 8.0 percent of the adult population. 1 VA also reported that an average of 18 veterans died by suicide each day in 2018, and about one-third of these veterans (37 percent) had recently used Veterans Health Administration (VHA) health care services. In response, VA has identified suicide prevention as its highest clinical priority in its strategic plan for fiscal years 2018 through 2024. 2 Veterans at high risk for suicide require clinical evaluation and care to identify and treat behavioral and medical conditions and to specifically address suicide risk. VHA has used suicide prevention coordinators at local facilities since 2007 to support implementation of its suicide prevention program, which 1SeeDepartment of Veterans Affairs, 2020 VA National Veteran Suicide Prevention Annual Report (November 2020). 2See Department of Veterans Affairs, Department of Veterans Affairs Fiscal Years 2018- 2024 Strategic Plan (Washington, D.C.: May 31, 2019). Page 1 GAO-21-326 VA Suicide Prevention Teams was mandated by law. 3 Facilities may also use suicide prevention case managers and others who share or support the duties of coordinators, which along with the coordinators collectively represent local suicide prevention teams. Although VHA’s approach to veteran suicide prevention involves commitment from across the agency—including clinical providers from a variety of specialties—local suicide prevention teams serve as a focal point for suicide prevention efforts at their respective facilities. The Support for Suicide Prevention Coordinators Act included a provision for us to review, among other issues, the responsibilities and workload of suicide prevention coordinators. 4 This report examines 1. how VHA uses local suicide prevention teams in its suicide prevention efforts; and 2. how VHA helps determine facilities’ suicide prevention staffing needs. To address both objectives, we reviewed information and interviewed suicide prevention teams and supervisors from five selected VA medical centers and health care systems. 5 These facilities were selected for 3The Joshua Omvig Veterans Suicide Prevention Act required VA to develop and carry out a comprehensive program designed to reduce the incidence of suicide among veterans, including designating a suicide prevention counselor at each VA medical facility other than a Vet Center. A Vet Center is a facility that is operated by VA for the provision of health care services but is situated apart from VA general health care facilities. Pub. L. No. 110-110, § 3, 121 Stat. 1031 (2007), codified at 38 U.S.C. § 1720F. VHA provides care through various types of medical facilities, including medical centers and outpatient clinics. In many areas of the country, several medical centers and clinics may work together to offer services to area veterans as a health care system in an effort to provide more efficient care. For the purpose of this report we refer to VA health care systems and medical centers as VHA facilities, unless otherwise noted. 4Pub. L. No. 116-96, § 2, 133 Stat. 3250 (2019). 5The five selected facilities are Cheyenne VA Medical Center (Cheyenne, WY); Oscar G. Johnson VA Medical Center (Iron Mountain, MI); Tennessee Valley Healthcare System (Nashville, TN); VA Black Hills Health Care System (Fort Meade, SD); and VA Maryland Health Care System (Baltimore, MD). At the time of our review, each of the five facilities included multiple outpatient clinics, and the three health care systems included multiple VA medical centers. The suicide prevention teams covered all the facilities. Team supervisors at most of the selected facilities supervised the individual team members and provided broader supervision of the implementation of local suicide prevention initiatives. Page 2 GAO-21-326 VA Suicide Prevention Teams variation in geographic location (i.e., from different Veterans Integrated Service Networks), facility complexity, the ratio of coordinators and case managers to unique patients served, and the composition of the suicide prevention team (e.g., coordinators only or coordinators and case managers). 6 We selected these facilities using information from VHA’s Office of Mental Health and Suicide Prevention (OMHSP), which maintains data on staffing levels for suicide prevention teams in facilities nationwide. We assessed the reliability of these data by comparing them to staffing level data reported to us by the five selected facilities, checking for missing values and obvious errors, and discussing them with VHA officials who were knowledgeable about the data. We determined that these data were sufficiently reliable for the purpose of selecting facilities for our review. Information obtained from the selected facilities is not generalizable. To examine how VHA uses local suicide prevention teams in its suicide prevention efforts, we reviewed VHA policies and procedures related to these efforts. We also reviewed documentation from and conducted interviews with officials from OMHSP regarding their support and expectations of local teams. Additionally, we interviewed suicide prevention teams from the five selected facilities about their work, including their experiences; their workload; the support they receive from OMHSP, their Veterans Integrated Service Network, and local leadership; and any challenges they face in doing their work. We determined the federal standards for internal control for risk assessment and monitoring (along with the underlying principles that management should analyze, identify, and respond to significant changes and conduct ongoing monitoring that is responsive to change) were relevant to this objective. 7 We assessed VHA’s procedures and practices regarding its use of suicide prevention teams in the context of these standards and the underlying principles. We also interviewed representatives from the following outside organizations to obtain contextual information: the American Legion—a veterans service organization—and Zero Suicide 6Veterans Integrated Service Networks are responsible for overseeing VHA facilities within a defined geographic area. VHA categorizes facilities according to complexity level, which is determined on the basis of the characteristics of the patient populations served, clinical services offered, educational and research missions, and administrative complexity. 7GAO, Standards for Internal Control in the Federal Government, GAO-14-704G (Washington, D.C.: Sept. 10, 2014). Internal control is a process effected by an entity’s management, oversight body, and other personnel that provides reasonable assurance that the objectives of an entity will be achieved. Page 3 GAO-21-326 VA Suicide Prevention Teams Institute, which provides training and consultation to health and behavioral health care systems and other organizations to promote safer suicide prevention care. To examine how VHA helps determine facilities’ suicide prevention staffing needs, we reviewed VHA policy and other documents related to suicide prevention staffing across its facilities. We also interviewed officials from OMHSP about their monitoring of suicide prevention staffing nationwide and their mental health hiring initiatives. We assessed OMHSP’s approach in helping to determine suicide prevention staffing by using the key practices for developing staffing models we previously identified. 8 We also reviewed OMHSP’s data on staffing of suicide prevention teams at facilities nationwide. We determined that these data were not sufficiently reliable for purposes of reporting suicide prevention staffing levels nationwide due to discrepancies we identified, among other issues, as discussed later in this report. We also interviewed suicide prevention teams and their supervisors from the five selected facilities about hiring and staffing for suicide prevention teams and any challenges they face with respect to suicide prevention staffing. We conducted this performance audit from March 2020 to April 2021 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. VHA operates one of the largest health care systems in the country, Background providing care to more than 9 million veterans at 1,240 medical facilities. These facilities include individual VA medical centers and local health 8GAO, Federal Protective Service: Enhancements to Performance Measures and Data Quality Processes Could Improve Human Capital Planning, GAO-16-384 (Washington, D.C.: Mar. 24, 2016). The report on the Federal Protective Service describes the key practices we identified for the design of staffing models. These key practices can be used to assess staffing models generally, including the benchmark for suicide prevention. Page 4 GAO-21-326 VA Suicide Prevention Teams care systems comprising more than one VA medical center working together, as well as community-based outpatient clinics. 9 In May 2017, VHA established OMHSP to organizationally consolidate mental health and suicide prevention under one office. 10 OMHSP is responsible for (1) monitoring and supporting the implementation of mental health and suicide prevention policies and the performance of mental health programs in VHA facilities, and (2) conducting ongoing evaluations of mental health services and policies. VA Suicide Prevention In 2007, the Joshua Omvig Veterans Suicide Prevention Act mandated that VA develop and implement a comprehensive program to reduce the incidence of suicide among veterans, which was to include mandatory training for staff working with veterans to recognize suicide risk factors and respond appropriately in crisis situations, among other things. 11 The act also required that each VHA facility have staff designated for suicide prevention, which, according to VHA, was the origin of the local suicide prevention team. VA reported in November 2020 that there were over 450 suicide prevention coordinators and team members at VHA facilities across the country. OMHSP manages VHA’s Suicide Prevention Program, which includes a variety of initiatives related to screening for suicide risk and increasing access to mental health care. The Suicide Prevention Program has grown over time, as the agency has undertaken several initiatives, including • Veterans Crisis Line. In 2007, VA established the Veterans Crisis Line. The crisis line provides confidential support 24 hours per day for veterans and their families and friends through phone, online chat, or 9Community-based outpatient clinics are stand-alone clinics that are geographically separate from VA medical centers and provide outpatient primary care, mental health care, and, in some cases, specialty care services. These outpatient clinics are administratively assigned to a VA medical center or health care system. 10Prior to May 2017, suicide prevention and mental health issues were organizationally located under different offices within VHA. 11Pub. L. No. 110-110, § 3, 121 Stat. 1031, 1031-32 (2007), codified at 38 U.S.C. § 1720F(a),(b). Page 5 GAO-21-326 VA Suicide Prevention Teams text message. 12 This support is available to veterans regardless of whether they receive health care through VA. • Recovery Engagement and Coordination for Health – Veterans Enhanced Treatment (REACH VET). In 2016, VA established REACH VET. This initiative uses predictive modeling to analyze existing data from veterans’ health records to identify veterans at increased risk for adverse outcomes, such as suicide, hospitalization, or illness. REACH VET coordinators staffed at VA medical centers are to notify the appropriate VHA mental health or primary care provider that a veteran has been identified as being at high risk for suicide, using a list of high-risk veterans generated monthly by REACH VET’s predictive model. After REACH VET identifies veterans, VA health care providers are to contact them to determine whether they need additional care or services. • Standardized suicide risk identification. In 2018, VHA began implementing a standardized suicide risk identification process (which VHA refers to as Risk ID) for all veterans receiving VHA health care. Risk ID began as a three component process that included 1. an annual initial mental health screening, which included a question about suicide risk; 2. a secondary screening for those veterans with positive initial screens and for those receiving mental health care, who may be at risk for suicide; and 3. a comprehensive screening for veterans with positive secondary screens or who have demonstrated suicidal behavior, who are at higher risk for suicide. VHA implemented the Risk ID process in three phases—one phase for each of the three components. According to VHA, it completed 12The Veterans Crisis Line can be reached (1) by calling a national toll-free number—1- 800-273-8255—and pressing “1” to be connected with a responder; (2) online at www.VeteransCrisisLine.net; or (3) by sending a text message to 838255. We have previously reported on the Veterans Crisis Line and made recommendations for improving the timeliness and quality of crisis line responses. VA concurred with our recommendations and, in July 2016, developed and implemented procedures to regularly test the crisis line system. In addition, in July 2017, VA updated its quality assurance plan to document measurable targets and time frames for key performance indicators needed to assess the crisis line’s performance. See GAO, Veterans Crisis Line: Additional Testing, Monitoring, and Information Needed to Ensure Better Quality Service, GAO-16-373 (Washington, D.C.: May 26, 2016). Page 6 GAO-21-326 VA Suicide Prevention Teams implementation of all three components in October 2019. In November 2020, VHA updated Risk ID to make it a two-component process; the tool previously used for the secondary screening has replaced the suicide risk question included in the annual initial mental health screening. According to VHA, this change was made to better align with accrediting body standards. • Mental health hiring initiative. OMHSP started this initiative in 2017 in response to a workforce study that identified staffing needs for mental health, including for suicide prevention, at facilities nationwide. The goal of the initiative was for local facilities to add 1,000 outpatient mental health staff, including suicide prevention coordinators and case managers. According to OMHSP, this hiring goal was reached in 2019. The initiative did not include funding for new staff but did encourage facilities to focus their resources on mental health hiring. Local Suicide Prevention According to OMHSP, suicide prevention teams typically operate within Teams the outpatient mental health department at the facility level, and teams generally work together to serve entire VA medical centers or health care systems. Team composition. VHA policy requires that every VA medical center and every “very large” community-based outpatient clinic—a clinic that serves more than 10,000 unique patients—have at least one full-time suicide prevention coordinator. 13 VHA requires coordinators to be funded as full-time, permanent staff positions with no collateral duties. Beyond meeting these basic requirements, individual facilities are responsible for staffing their local suicide prevention teams, including hiring and making decisions about how many coordinators their facility has and which qualifications their coordinators have, based on local needs. Coordinators may be social workers, registered nurses, or psychologists. Unlike the coordinator position, VHA does not have policies requiring a minimum number of case managers at a facility or governing the qualifications for this position. According to OMHSP, local facilities may 13See Department of Veterans Affairs, Veterans Health Administration, Uniform Mental Health Services in VA Medical Centers and Clinics, VHA Handbook 1160.01 (Washington, D.C.: Sept. 11, 2008, amended Nov. 16, 2015). In 2016, VHA issued VHA Directive 6330, which provided new definitions for national policy and guidance. Specifically, the directive eliminated handbooks as a national policy document option. It also stated that handbooks signed by the Under Secretary for Health that were certified before VHA Directive 6330 was published would continue to serve as national policy until rescinded, recertified, or August 31, 2021, whichever is later. Page 7 GAO-21-326 VA Suicide Prevention Teams decide how many case managers they need, if any. Coordinators and case managers have overlapping responsibilities and are sometimes viewed as interchangeable. Other suicide prevention team members may include various program support staff. For example, teams may have program support assistants who provide administrative support. Teams may also have peer support specialists—veterans who draw on past experiences to help bridge the gap between veterans and providers. Direction, guidance, and support for teams. OMHSP disseminates suicide prevention policies, which provide direction and guidance to facilities that are responsible for implementing them. In addition, to support facilities’ implementation of suicide prevention policies, among other things, OMHSP hosts regular calls to which all suicide prevention teams nationwide are invited. During these calls, OMHSP shares information with teams and invites them to ask questions. Some calls may highlight best practices from an individual suicide prevention team. In addition to these calls, OMHSP also provides technical assistance regarding implementation of suicide prevention efforts, including answering questions from individual suicide prevention teams. 14 The Veterans Integrated Service Networks may also provide technical assistance and support to suicide prevention teams from facilities within their network. Oversight of teams and individual team members. OMHSP does not have policies governing oversight of suicide prevention teams and individual team members, but reported monitoring the work of teams nationwide through review of data dashboards. These dashboards include data on the number of attempted suicides and deaths by suicide at each facility and the types of outreach and support team members provide. According to OMHSP, oversight and supervision of teams currently occur at the facility level—often within the mental health department—and vary by facility. Recently enacted legislation requires VA to conduct a study to determine the feasibility and advisability of the realignment and reorganization of suicide prevention coordinators within OMHSP. 15 14Inaddition, VA provides support to teams through SharePoint resources, data dashboards, and implementation tools. 15Pub. L. No. 116-171, § 506(b), 134 Stat. 778, 821 (2020). Page 8 GAO-21-326 VA Suicide Prevention Teams OMHSP uses local suicide prevention teams consisting of coordinators, VHA Uses Local case managers, and others to implement its Suicide Prevention Program Teams to Carry Out a at local facilities. However, OMHSP has not comprehensively evaluated the effect of program growth on teams’ workload, including any resulting Variety of Suicide challenges for these staff. Prevention Program Specifically, teams perform a variety of programmatic, administrative, and Activities but Has Not clinical activities in implementing VA’s Suicide Prevention Program. Evaluated the Effect These activities fall under eight core responsibilities defined in VHA policy. 16 (See table 1.) Although VHA policy documents may designate of Program Growth certain activities to coordinators specifically, other members of the suicide on Their Workload prevention team may also complete these activities. Teams in our review described a variety of ways they had divided the activities within their individual teams. For example, members from one team told us that their coordinator handled the programmatic activities related to suicide prevention (e.g., monitoring data), and the case managers worked more directly with veterans, including responding to referrals from the Veterans Crisis Line. Table 1: Suicide Prevention Teams’ Responsibilities and Examples of Related Activities, as of January 2021 Responsibilities Examples of related activities Track and report on veterans at high Track and report all suicide-related events, including preparatory behaviors, ideations, risk for suicide attempts, and deaths, in the facility catchment area. Submit all suicidal event information through national standardized templates in the electronic health record or to the national Suicide Prevention Application Network database. Complete required documentation after a confirmed veteran suicide as part of the Behavioral Health Autopsy Program—a VA initiative to collect information on veteran suicides nationwide that can be used to develop policy and procedures to prevent future deaths by suicide. Documentation includes standardized medical chart reviews for all veteran suicides known to facility staff. Respond to referrals from the Respond to all consult referrals sent to the team from the crisis line within 24 hours (1 Veterans Crisis Line business day). There should be a minimum of three attempts to reach the veteran in the form of three phone calls, followed by a letter if the calls were unsuccessful in reaching the veteran. When needed, assist the veterans with the process for enrollment for VA health care benefits. 16Veterans Health Administration, Uniform Mental Health Services, VHA Handbook 1160.01. Page 9 GAO-21-326 VA Suicide Prevention Teams Responsibilities Examples of related activities Train all VA staff who have contact Ensure all clinical and non-clinical facility staff are appropriately trained in suicide prevention with patients within 90 days of hiring. Specifically, provide training on how to • identify potentially suicidal veterans; • be aware of misinformation about suicide; • talk to veterans about their suicidal ideation, and ask appropriate questions; • effectively communicate that help is available; and • get veterans to a competent provider for assessment or treatment if deemed necessary. Collaborate with community Deliver five community outreach activities every month. organizations and partners Build relationships with local and state suicide prevention organizations, local crisis line organizations, and local veterans service organizations. Provide general consultation to Consult with clinical staff members who are assessing risk for suicide and educate them providers about risk stratification tools. Work with providers on high-risk Make personal contact with veterans who have been identified as surviving a suicide attempt, veteran patients or who are identified as being at high risk for suicide and placed on the facility’s high-risk list. Report on a monthly basis to VHA’s Complete monthly data entry in the Suicide Prevention Application Network. Office of Mental Health and Suicide Prevention Ensure providers follow up with high- Ensure that veterans identified as being at high risk for suicide receive follow up for any risk patients on any missed missed mental health and substance abuse appointments, and that follow-up efforts are appointments to confirm safety and documented in the veteran’s electronic medical record. initiate problem-solving about any issues with ongoing care Source: GAO summary of information from Department of Veterans Affairs (VA) | GAO-21-326 Note: Although some Veterans Health Administration (VHA) policy documents may designate certain responsibilities or activities to coordinators specifically, other members of suicide prevention teams may also perform these activities. In January 2021, VHA issued a new guide for suicide prevention teams, which provides a detailed summary of the expected activities for local coordinators and teams. 17 VHA noted that the guide is not a policy document and is not intended to be all-inclusive; the work of the coordinator and teams continues to evolve, as new initiatives and directives are implemented. In addition to the activities specifically required of local suicide prevention teams, other suicide prevention efforts may also be tasked to them by local leadership, according to OMHSP officials. OMHSP officials said that although teams’ core responsibilities have not changed since the coordinator role was established, their related activities 17See Department of Veterans Affairs, Veterans Health Administration, Suicide Prevention Program Guide (revised November 2020, issued January 2021). The guide specifically states that it is to be used by all Suicide Prevention Program staff, including coordinators, case managers, program support staff, and others. Page 10 GAO-21-326 VA Suicide Prevention Teams have increased, due to both the addition of new VHA suicide prevention initiatives over time and an increase in the volume of their veteran caseload. Officials specifically cited recent initiatives such as the REACH VET and the Risk ID programs as sources of additional activities for teams. 18 Officials also noted that the increase in teams’ caseload is due in part to an increase in the number of calls to the Veterans Crisis Line, which may result in referrals to local teams. Teams in our review cited several challenges they face in completing suicide prevention activities: • Increased workload volume. Staff from all five teams in our review described challenges related to increased workload volume, as VHA has added new initiatives or made changes to existing efforts. OMHSP officials said the increased workload was true for suicide prevention teams overall, and they said that the role of teams has become more complex as a result of additional activities that have been assigned to teams by VHA or local facility leadership. For example, one team said they have experienced burnout and turnover in the team due to the “constant deluge” of new initiatives, combined with their large caseload of high-risk veterans. Members of another team told us their team has a caseload of about 150-200 veterans identified as being high risk for suicide; teams are expected to engage in a number of activities related to these veterans, including regular interactions. See text box for an example of this team’s procedures for managing activities regarding veterans at high risk for suicide. 18VA officials said that although activities related to these initiatives do not necessarily fall to teams automatically, these activities may be assigned to the teams by local facility leadership, as permitted by national policy. Page 11 GAO-21-326 VA Suicide Prevention Teams Example of one suicide prevention team’s procedures for tracking and reporting on veterans at high risk for suicide • Receive and review consults from facility providers who have concerns that a veteran is at risk of self-harm. • Determine if placement of a high-risk flag in the veteran’s electronic health record is appropriate, based on criteria, such as a recent suicide attempt or suicidal ideation with a plan. • If appropriate, activate high-risk flag in the veteran’s electronic health record, which would alert providers that additional care should be taken by everyone interacting with the veteran to attend to the increased risk for suicide—such as considering frequent follow-up appointments, involving significant others in care planning, and limiting access to means of harming oneself when possible. • If transferring a veteran’s flag to another facility, facilitate transfer of care: coordinate with receiving site’s local suicide prevention team on transferring the flag, scheduling of initial mental health appointments, and confirming updated address in the electronic health record. • If receiving a veteran’s flag from another facility, coordinate with the transferring facility’s suicide prevention team to complete necessary steps and accept the flag transfer in the electronic health record. • Ensure that flags are reviewed for removal or continuance every 90 days by facility’s review team—comprised of at least four members from three different mental health disciplines such as social work and psychiatry. • Twice per month, participate with the facility review team to review flags due for 90-day review, including chart reviews and discussions with treating providers. Source: GAO summary of information from one Department of Veterans Affairs facility’s local suicide prevention team. | GAO-21-326 • Insufficient guidance for implementation of policies related to new programs. All five teams in our review described challenges related to implementing new policies or initiatives related to the Suicide Prevention Program and, in some cases, insufficient guidance from OMHSP on how to implement them locally. For example, members from one team said that sometimes OMHSP does not give local facilities guidance for how to carry out responsibilities related to new initiatives; instead, OMHSP gives local facilities the authority to determine how best to do it. One team in our review acknowledged that local implementation of new Suicide Prevention Program initiatives is not a one-size-fits-all endeavor, and another team said that it can be both beneficial and challenging to have such flexibility at the facility level. A third team told us about an instance when they overcame the lack of guidance from OMHSP by drawing on the knowledge of their peers at another facility to implement a change to their approach to managing patient record flags for those identified as being at high risk for suicide. • Responsibility for suicide prevention initiatives across their facilities. All five teams in our review described instances where they were asked to ensure implementation of suicide prevention initiatives across their facilities, including assisting departments outside of mental health. For example, members from one team in our review told us that to implement screenings in their facility’s emergency department, they must train emergency department staff. Members Page 12 GAO-21-326 VA Suicide Prevention Teams from that same team also described how the implementation of Risk ID—which was completed in October 2019—required them to quickly learn the new approach and provide education to all providers across their facility on how it would be implemented. Risk ID posed a significant change, as VA did not previously have a national standard for screening, assessment, and documentation of suicide risk. 19 In addition to the new guide for suicide prevention teams, OMHSP has made—and plans to make—other changes to its support and guidance for suicide prevention teams, which may help mitigate some of the challenges identified by teams in our review. For example, • OMHSP officials told us that they started hosting a monthly “office hours” call for teams in June 2020, in addition to their monthly national call. According to officials, this new call is intended to provide local teams more opportunities to ask questions of OMHSP and seek technical assistance. In contrast, they explained that the national call is more of a means for OMHSP to provide information and updates with less opportunity for discussion and questions. • OMHSP officials told us they are in the process of developing a new overarching policy for the Suicide Prevention Program, outlining the overall responsibilities for suicide prevention teams, along with responsibilities for VHA program offices, Veterans Integrated Service Networks, and facility leadership. Officials said that the new policy is an effort to formalize and consolidate expectations that have previously been documented across a variety of sources. At the time of our review, there were more than 20 policy documents issued since 2008 (including directives, memos, notices, and handbooks) related to the Suicide Prevention Program that include specific activities for local suicide prevention teams. Officials told us the new policy is undergoing internal review, and they expect it to be issued by March 2021. Although OMHSP is taking these actions to help teams, it has not comprehensively evaluated the effect of the growth of the Suicide Prevention Program on teams’ workload, or any related challenges. The lack of such an evaluation is inconsistent with federal internal control standards, which state that management should identify, analyze, and respond to significant changes that could impact the ability to effectively 19Prior to Risk ID, VA conducted annual mental health depression screenings and evaluations that included questions related to suicide risk. Page 13 GAO-21-326 VA Suicide Prevention Teams and efficiently meet program objectives—including OMHSP’s support of local teams. 20 Evaluations of the program that have been conducted have not included an assessment or recognition of the role of local suicide prevention teams in implementing the program. 21 Although the new guide for teams summarizes the expected activities in one place, it is a resource document that does not include an assessment of the time needed to complete the activities or their effect on workload. Similarly, OMHSP’s upcoming planned overarching policy does not call for such an assessment. Without a comprehensive understanding of local teams’ workload in the face of recent program growth, VHA may be overwhelming these teams with new activities and initiatives, which ultimately could reduce teams’ effectiveness in serving veterans at risk for suicide. Further, according to federal standards for internal control, management should establish and operate monitoring activities, which may include ongoing monitoring that is responsive to changes within the organization. Such monitoring could include a comprehensive evaluation as described above. 22 The information obtained through such an evaluation would better allow OMHSP to refine the support it provides to local suicide prevention teams—e.g., direction, guidance, and technical assistance—to potentially reduce challenges and workload burdens that teams may face. VHA’s OMHSP uses a staffing benchmark to help determine facilities’ VHA’s Benchmark for suicide prevention staffing needs—specifically, the number of Suicide Prevention Staffing May Not Accurately Reflect Facilities’ Needs 20See GAO-14-704G. 21Department of Veterans Affairs, Clay Hunt Suicide Prevention for American Veterans (SAV) Act 2018 Annual Report: VA Mental Health Program and Suicide Prevention Services Independent Evaluation (Washington, D.C.: October 2018). See also Department of Veterans Affairs, Clay Hunt Suicide Prevention for American Veterans (SAV) Act 2019 Annual Report: VA Mental Health Program and Suicide Prevention Services Independent Evaluation (Washington, D.C.: October 2019). 22See GAO-14-704G. Page 14 GAO-21-326 VA Suicide Prevention Teams coordinators and case managers each facility should aim to have. 23 According to OMHSP’s benchmark, each VHA facility should have at least one coordinator or case manager for every 10,000 unique patients the facility serves. 24 However, as stated previously, facilities are responsible for making their own staffing decisions. Accordingly, OMHSP officials said that the benchmark serves to alert facilities that they may need to make changes to their suicide prevention staffing. OMHSP officials said that the benchmark represents a rough proxy for coordinator and case manager workload. In fiscal year 2017, when the benchmark was developed, one coordinator or case manager per 10,000 unique patients the facility served was the median for suicide prevention teams in facilities nationwide, according to OMHSP officials. We found that this staffing benchmark may not help facilities accurately determine their staffing needs because OMHSP did not develop the benchmark according to key practices for the design of staffing models that we have identified in our prior work. 25 According to these key practices, staffing model design should (1) involve key stakeholders; (2) incorporate work activities, their frequency, and the time required to conduct them; (3) ensure the quality of data used in the model; and (4) incorporate risk factors. Key stakeholders. OMHSP officials told us that they did not formally involve key stakeholders, e.g., facility coordinators and case managers, in developing its benchmark for suicide prevention staffing. Formal involvement of key stakeholders may include forming working groups, soliciting stakeholder comments on draft plans, conducting surveys, and interviewing stakeholders. 26 Involving stakeholders and subject matter experts when designing a staffing benchmark can help 23OMHSP’s benchmark for suicide prevention staffing does not include program support or administrative staff who may also be part of some facilities’ suicide prevention teams. 24The suicide prevention staffing benchmark is one part of OMHSP’s broader staffing model for outpatient mental health across VHA, which was first developed in fiscal year 2012. That staffing model provides a goal for overall outpatient mental health staff members—7.72 per 1,000 treated patients—and benchmarks for specific types of outpatient mental health staff. The model includes the benchmark for coordinators and case managers, which was added in fiscal year 2017. 25See, for example, GAO-16-384. 26See, for example, GAO-16-384. Page 15 GAO-21-326 VA Suicide Prevention Teams an office ensure that its benchmark reflects operating conditions and meets the needs of those using it. 27 OMHSP officials told us that they discussed the benchmark informally with facility coordinators who were working on temporary assignments with OMHSP during the time the benchmark was developed. However, the office did not formally solicit the involvement of coordinators or case managers in the field. By not formally involving key stakeholders, OMHSP did not sufficiently include the perspectives of individuals conducting the day-to-day suicide prevention work in facilities in developing its staffing benchmark. Including local suicide prevention team members’ perspectives would allow OMHSP to better understand teams’ operating conditions and staffing needs and reflect those needs in the benchmark. Work activities. OMHSP officials told us that the staffing benchmark has not been updated since fiscal year 2017 when it was developed. As a result, it may not accurately reflect how many coordinators and case managers are required for teams’ current work activities, their frequency, and the time required to conduct them. As discussed previously, workloads for coordinators and case managers—as part of their larger teams—have increased over time, as VHA has implemented new suicide prevention initiatives and veteran caseloads have grown in recent years, according to OMHSP officials. As suicide prevention teams implement new initiatives locally, the team’s work activities, their frequency, and the time required to conduct them may change. Coordinators and a case manager from one of our five selected facilities told us that they had fallen behind on implementing some initiatives due to not having enough staff. Conversely, staff from another team told us that adding two new staff—tripling the size of their team—in the last several years had contributed to their ability to keep up with their workload. Because the benchmark has not been updated as new work activities for coordinators and case managers have been added, it may not accurately reflect the number of staff each facility needs to complete their current workload. For example, by using the benchmark, OMHSP may not accurately estimate how many coordinators and case managers facilities should aim to have in order to make regular personal contact with veterans identified as being high risk for suicide. OMHSP’s guide outlines this 27See, for example, GAO, DOJ Workforce Planning: Grant-making Components Should Enhance the Utility of Their Staffing Models, GAO-13-92 (Washington, D.C.: Dec. 14, 2012). Page 16 GAO-21-326 VA Suicide Prevention Teams as an expected activity of coordinators and case managers, and staff from one team told us this plays an important part in their work. The amount of time per contact and the complexity of each veteran’s needs may vary, contributing to a larger workload, according to the coordinator and case managers from one facility in our review. Specifically, case managers from this suicide prevention team told us that they may spend 20-40 minutes on a call with a veteran at high risk for suicide. The case managers explained that it is difficult to keep calls short because many veterans suffer from social isolation, and building relationships takes time. They also said it is crucial that staff take the time to build these relationships, so veterans know that they can reach out to their local teams when they need help. Without an understanding of how much time it takes for coordinators and case managers to complete their work activities, OMHSP cannot ensure that it is helping facilities to accurately determine their staffing needs. Quality of the data. The benchmark is based on data that may not be of sufficient quality for OMHSP to use to help facilities accurately determine coordinator and case manager staffing needs. Our prior work defined data quality as the use of relevant data from reliable internal or external sources based on the information requirements. 28 Specifically, we found discrepancies between the staffing data that OMHSP provided to us and the staffing levels that facilities in our review reported to us, which raises concerns about the quality of the data. Of the five facilities included in our review, two reported having more coordinators, one reported having more case managers, and one reported having fewer case managers than were reflected in the staffing data that OMHSP provided to us. These discrepancies, along with the self-reported nature of the data, led us to determine that these data may not be of sufficient quality to help determine facilities’ suicide prevention staffing levels. OMHSP officials told us that individual facilities are responsible for entering staffing data into the Patient Centered Management Module—an application that VA facilities use to assign individual staff to health care teams, like the suicide prevention team—and keeping their data up to date. 29 Officials told us that one of the challenges related to data quality is that individual facilities decide who, within the 28See GAO-16-384. 29See Department of Veterans Affairs, Veterans Health Administration, Patient Centered Management Module (PCMM) for Primary Care, VHA Directive 1406 (Washington, D.C.: June 20, 2017). Page 17 GAO-21-326 VA Suicide Prevention Teams facility, can access and update these data; often, the mental health department—where suicide prevention teams are typically housed— does not have access to the Patient Centered Management Module, which could contribute to discrepancies. In addition, the data from the Patient Centered Management Module are also of insufficient detail for OMHSP to ensure facilities adhere to VHA’s policy because the data are not always tracked at the level of each individual VA medical center and community-based outpatient clinic. As mentioned previously, VHA policy requires that every VA medical center and “very large” community-based outpatient clinic have at least one full-time suicide prevention coordinator. 30 OMHSP officials explained that they review local suicide prevention team staffing data for each quarter and follow up with facilities whose data show that they have less than one full-time coordinator. Although data from the module shows OMHSP the number of suicide prevention staff at each health care system, the data do not provide information on the number of staff at the level of individual VA medical centers within a health care system. 31 Similarly, they do not provide information on the number of staff at the level of individual community- based outpatient clinics. OMHSP officials told us that they do not have a data system that would allow them to track staffing data at more granular levels, although they have explored options for creating such a system. They explained that creating a new system would require time and resources that are currently focused on other data system initiatives. By relying on data that may not be of sufficient quality and that are not sufficiently detailed, OMHSP may incorrectly estimate facilities’ staffing needs using the benchmark. Risk factors. In developing its benchmark, VHA did not account for risk factors that could result in variation among facilities. Specifically, the staffing benchmark does not reflect risk factors that may affect coordinators’ and case managers’ work activities and the time it takes to conduct them, such as the geographic area covered by a suicide prevention team and variation in suicide risk based on location. OMHSP officials said that the benchmark was developed based on the median staffing ratio at the time. Accounting for risk factors in a 30Veterans Health Administration, Uniform Mental Health Services, VHA Handbook 1160.01. 31Staffing data are available at the VA medical center level for locations that are not part of a larger health care system. Page 18 GAO-21-326 VA Suicide Prevention Teams staffing benchmark can help better determine the number and type of staff needed to mitigate that risk. 32 For example, coordinators from one rural facility told us that it was challenging to conduct the required number of outreach activities— five events per month—due to the travel time necessary to reach more remote parts of their service area, which comprises more than 25,000 square miles. The coordinators told us that the facility’s addition of a second coordinator helped to alleviate some of the challenges associated with travel time to outreach events, ensuring that such travel does not detract from the team’s ability to perform other suicide prevention activities. Another risk factor that coordinators and case managers face is variation in suicide risk by location. For example, OMHSP officials explained that suicide rates are not uniform among states. According to VHA data, in 2018, the three states with the highest veteran suicide rates were Montana, Oregon, and West Virginia, and those with the lowest rates were Hawaii, New Jersey, and Massachusetts. 33 Officials explained that the number of deaths by suicide of a given facility may cause variation in caseloads. Without incorporating risk factors into the staffing benchmark for coordinators and case managers, OMHSP’s determination of the appropriate staffing levels in facilities nationwide may not be sufficient to mitigate associated risks. Without a staffing benchmark that reflects the key practices we identified, OMHSP does not have complete information to help facilities accurately determine their staffing needs. As a result, suicide prevention teams may not be appropriately staffed to meet facilities’ needs. 32See, for example, GAO, Homeland Security: Preliminary Observations on the Federal Protective Service’s Workforce Analysis and Planning Efforts, GAO-10-802R (Washington, D.C.: June 14, 2010). 33Among all 50 states, veteran suicide rates ranged from 16.5 to 60.9 per 100,000 veterans. Page 19 GAO-21-326 VA Suicide Prevention Teams VA has stated that preventing veteran suicide is its top clinical priority, Conclusions and the agency has introduced a number of initiatives as part of its Suicide Prevention Program since the program’s inception in an effort to address this problem. However, it does not have a good understanding of the effects of the growth of the program on its local suicide prevention teams. Without an evaluation that takes into account program growth and changes to workload, OMHSP does not know how best to support suicide prevention teams in meeting their responsibilities, putting those teams— and ultimately the care provided to the veterans they serve—at risk of falling short of the program’s goal to reduce the incidence of suicide among veterans. Further, OMHSP’s reliance on a staffing benchmark that was not developed according to key practices for the design of staffing models hampers its ability to help ensure facilities have appropriate staffing for suicide prevention, leaving teams vulnerable to understaffing. Without an accurate understanding of facilities’ needs for suicide prevention staff, VA cannot ensure that all of its facilities are prepared to care for veterans at risk for suicide. We are making the following three recommendations to VA: Recommendations for Executive Action • The Under Secretary for Health should ensure that OMHSP conducts a comprehensive evaluation of local suicide prevention teams (including suicide prevention coordinators and case managers and others). Such an evaluation should seek to (a) obtain a full understanding of how facilities are using these teams, (b) identify challenges teams may experience in implementing VHA policies, and (c) identify the effects of program growth on teams’ workload. (Recommendation 1) • The Under Secretary for Health should ensure that OMHSP uses information obtained through the comprehensive evaluation of local suicide prevention teams to inform the support it provides—e.g., direction, guidance, and technical assistance. (Recommendation 2) • The Under Secretary for Health should ensure that OMHSP incorporates key practices for staffing model design into its determination of facilities’ suicide prevention staffing needs: (a) involving key stakeholders; (b) incorporating work activities, their frequency, and the time required to complete them; (c) ensuring the quality of data used in the model; and (d) incorporating risk factors. (Recommendation 3) Page 20 GAO-21-326 VA Suicide Prevention Teams We provided a draft of this report to VA for review and comment. VA Agency Comments provided written comments, which are reprinted in appendix I. In its comments, VA concurred with all three of our recommendations and identified actions it is taking to address them. These actions include using information obtained through an evaluation required by law to address each of our recommendations. 34 VA also described OMHSP’s plans to design and pilot a site visit process to provide more in-depth monitoring of and communication with local suicide prevention teams. VA also noted it will conduct a separate review targeted to the key practices for staffing model design we previously identified to inform and evolve its staffing model. VA also provided technical comments, which we incorporated as appropriate. We are sending copies of this report to the appropriate congressional committees, the Secretary of Veterans Affairs, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-7114 or DraperD@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix II. Debra A. Draper Director, Health Care 34Section 506 of the Commander John Scott Hannon Veterans Mental Health Care Improvement Act of 2019 requires VA to complete a feasibility and advisability study of the realignment and reorganization of suicide prevention coordinators within OMHSP and of the creation of a suicide prevention coordinator program office. Pub. L. No. 116-171, § 506(b), 134 Stat. 778, 821 (2020). Page 21 GAO-21-326 VA Suicide Prevention Teams Appendix I: Comments from the Department Appendix I: Comments from the Department of Veterans Affairs of Veterans Affairs Page 22 GAO-21-326 VA Suicide Prevention Teams Appendix I: Comments from the Department of Veterans Affairs Page 23 GAO-21-326 VA Suicide Prevention Teams Appendix I: Comments from the Department of Veterans Affairs Page 24 GAO-21-326 VA Suicide Prevention Teams Appendix II: GAO Contact and Staff Appendix II: GAO Contact and Staff Acknowledgments Acknowledgments Debra A. Draper, (202) 512-7114 or draperd@gao.gov. GAO Contact In addition to the contact named above, Janina Austin, Assistant Director; Staff Julie T. Stewart, Analyst-in-Charge; and Miranda Richard made key Acknowledgments contributions to this report. Also contributing were Rob Dougherty, Jacquelyn Hamilton, and Caitlin Scoville. (104205) Page 25 GAO-21-326 VA Suicide Prevention Teams The Government Accountability Office, the audit, evaluation, and investigative GAO’s Mission arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no cost is Obtaining Copies of through our website. Each weekday afternoon, GAO posts on its website newly GAO Reports and released reports, testimony, and correspondence. You can also subscribe to GAO’s email updates to receive notification of newly posted products. Testimony Order by Phone The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, https://www.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO on Facebook, Flickr, Twitter, and YouTube. Connect with GAO Subscribe to our RSS Feeds or Email Updates. Listen to our Podcasts. Visit GAO on the web at https://www.gao.gov. Contact FraudNet: To Report Fraud, Website: https://www.gao.gov/fraudnet/fraudnet.htm Waste, and Abuse in Automated answering system: (800) 424-5454 or (202) 512-7700 Federal Programs Orice Williams Brown, Managing Director, WilliamsO@gao.gov, (202) 512-4400, Congressional U.S. Government Accountability Office, 441 G Street NW, Room 7125, Relations Washington, DC 20548 Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 Public Affairs U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548 Stephen J. Sanford, Acting Managing Director, spel@gao.gov, (202) 512-4707 Strategic Planning and U.S. Government Accountability Office, 441 G Street NW, Room 7814, External Liaison Washington, DC 20548 Please Print on Recycled Paper.