United States Government Accountability Office Report to Congressional Committees May 2022 VA VET CENTERS Opportunities Exist to Help Better Ensure Veterans' and servicemembers' Readjustment Counseling Needs Are Met GAO-22-105039 GAO Highlights Highlights of GAO-22-1050339, a report to congressional committees Why GAO Did This Study Some veterans and servicemembers experience challenges, such as mental illness, when readjusting to civilian life or to continued military service. This can be due to trauma experienced during military service. VHA's Vet Centers provide services to eligible veterans, servicemembers, and their families. The Commander John Scott Hannon Veterans Mental Health Care Improvement Act of 2019 included a provision for GAO to review VHA's RCS. This report examines (1) how VHA assesses whether Vet Centers are meeting clients' needs, (2) how Vet Centers assess the effectiveness of their outreach, and (3) barriers to Vet Center care VHA has identified and steps it has taken to address them. GAO reviewed VHA policies and interviewed officials from RCS and counselors and outreach staff from five Vet Centers. GAO selected Vet Centers for variation in geographic location and other factors. GAO also interviewed representatives from two veterans service organizations. What GAO Recommends GAO is making five recommendations, including that VHA (1) develop a process to assess the extent that Vet Centers are meeting the needs of clients collectively; (2) provide Vet Centers guidance to assess the effectiveness of their outreach; and (3) develop a process to assess the extent of barriers to obtaining Vet Center care. The Department of Veterans Affairs concurred with GAO's recommendations and identified actions VHA is taking to implement them. View GAO-22-105039. For more information, contact Sharon Silas at (202) 512-7114 or silass@gao.gov. VA VET CENTERS Opportunities Exist to Help Better Ensure Veterans' and Servicemembers' Readjustment Counseling Needs Are Met What GAO Found The Veterans Health Administration's (VHA) Readjustment Counseling Service (RCS) provides counseling (individual, group, couples, and family) through 300 Vet Centers to help eligible veterans, servicemembers, and their families readjust to civilian life or to continued military service. RCS and Vet Centers use psychosocial assessments and feedback surveys to assess individual client needs and whether those needs are being met throughout the course of counseling. However, RCS does not assess the extent to which Vet Centers are meeting the needs of clients collectively, including client subpopulations that may experience different readjustment challenges. For example, RCS has not analyzed information from psychosocial assessments or feedback surveys to assess what proportion of Vet Center clients is making progress in having their needs met. Periodically assessing the extent to which Vet Centers are meeting the needs of clients collectively would help RCS identify whether any actions are needed to help ensure clients achieve successful readjustment. Vet Centers also conduct outreach activities to inform eligible individuals and their family members about available services. Vet Centers use information they collect from outreach activities, such as the number of contacts made, to try to assess their effectiveness. However, there are limitations to using this information, according to officials from RCS and the Vet Centers in GAO's review. For example, these officials told GAO the number of outreach contacts may not be an appropriate way to assess the effectiveness of activities that do not generate a lot of contacts, such as those tailored to specific communities (e.g., Native Americans) with which it can take time to develop trust. Vet Center officials told GAO it would be helpful if RCS provided them with guidance that includes metrics and targets for assessing the effectiveness of their outreach activities. RCS officials told GAO they were in the early stages of developing such metrics. After their development, guidance for using these metrics and targets would help Vet Centers ensure their outreach is effective in increasing awareness among those who are eligible for services. Officials from RCS and the five Vet Centers in GAO's review identified barriers to Vet Center care. Officials also identified steps they have taken to address barriers, including outreach to local veterans service organizations to increase awareness of services and providing care during non-traditional hours (e.g., evenings) and via telehealth to increase access. However, GAO found that RCS does not know the extent to which barriers to Vet Center care remain, because RCS does not have processes for assessing their extent. For example, RCS does not know approximately how many veterans experience challenges getting to Vet Centers during their hours of operation. RCS officials told GAO they have begun to develop efforts to assess the extent to which barriers remain but are in the early stages. Periodically assessing this would improve RCS's understanding of the magnitude of barriers. Such assessments would also improve RCS's understanding of whether additional steps are needed to mitigate or reduce any remaining barriers. United States Government Accountability Office Contents Letter Background 7 VHA Assesses Whether Vet Centers Are Meeting the Readjustment Counseling Needs of Clients Individually but Not Collectively 11 Vet Centers Develop Outreach Plans but Lack Data to Better Tailor Outreach Activities and Guidance to Assess Their Effectiveness 16 VHA Has Identified and Taken Some Steps to Address Barriers to Vet Center Care, but Does Not Know the Extent to Which These Barriers Remain 22 Conclusions 29 Recommendations for Executive Action 29 Agency Comments and Our Evaluation 30 Appendix | Comments from the Department of Veterans Affairs 32 Appendix II GAO Contact and Staff Acknowledgments 41 Table Table 1: Selected Characteristics of VHA Vet Center Client Visits, Fiscal Years 2019 through 2021 26 Figures Figure 1: Number of VHA Vet Center Clients and Visits, Fiscal Years 2019 through 2021 8 Figure 2: Percentage of VHA Vet Center Clients by Selected Eligibility Categories, Fiscal Year 2021 10 Figure 3: Percentage of VHA Vet Center Clients by Selected Psychosocial Focus Areas, Fiscal Year 2021 13 Page i GAO-22-105039 VA Vet Centers Abbreviations RCS Readjustment Counseling Service VA Department of Veterans Affairs VHA Veterans Health Administration This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Page ii GAO-22-105039 VA Vet Centers GAO U.S. GOVERNMENT ACCOUNTABILITY OFFICE 441 G St. N.W. Washington, DC 20548 May 17, 2022 The Honorable Jon Tester Chairman The Honorable Jerry Moran Ranking Member Committee on Veterans' Affairs United States Senate The Honorable Mark Takano Chairman The Honorable Mike Bost Ranking Member Committee on Veterans' Affairs House of Representatives The Department of Veterans Affairs (VA) anticipates that each year about 200,000 servicemembers make the transition from the military to civilian life. Many servicemembers readjust to civilian life without major difficulty. However, some face challenges after leaving the military, such as mental illness, substance use, unemployment, and homelessness. Furthermore, according to VA, recent transition from military service to civilian life is a risk factor for suicide. 1 Although the Department of Defense has a role in assisting servicemembers with preparing for their transition, it is primarily VA's role to assist veterans after they separate from the military and begin readjusting to civilian life. Congress established Vet Centers as part of VA's Veterans Health Administration (VHA) in 1979 to provide readjustment counseling for veterans, recognizing that a significant number of Vietnam-era veterans were experiencing readjustment problems.2 Eligibility was originally limited to veterans who served on active duty during the Vietnam era but has subsequently been expanded several times. Currently, veterans and servicemembers who have served 1Department of Veterans Affairs, National Strategy for Preventing Veteran Suicide 2018- 2028 (Washington, D.C.: June 29, 2018). 2Pub. L. No. 96-22, tit. |, § 103(a), 93 Stat. 47, 48 (1979), codified, as amended, at 38 U.S.C. § 1712A. Page 1 GAO-22-105039 VA Vet Centers in any combat theater or area of hostility are eligible for Vet Center care, as are their family members.? The care Vet Centers provide includes readjustment counseling to help their clients successfully transition to civilian life or to continued military service. Readjustment counseling encompasses a range of services, such as counseling for post-traumatic stress disorder and military sexual trauma, and is provided through individual, group, couples, and family counseling visits.4 Vet Centers also conduct outreach to contact and engage local eligible individuals to inform them about their services. According to VHA, in fiscal year 2021, its 300 Vet Centers provided counseling services to 107,367 clients and conducted 32,415 outreach activities. Vet Centers operated at a cost of about $287 million in this fiscal year. The Commander John Scott Hannon Veterans Mental Health Care Improvement Act of 2019 included a provision for us to review various issues regarding Vet Centers.5 This report examines 1. how VHA assesses whether Vet Centers are meeting the needs of their clients, 2. how Vet Centers tailor outreach activities and assess their effectiveness, and 3. barriers to Vet Center care VHA has identified and steps it has taken to address them. To address all three objectives, we reviewed documentation and interviewed officials from VHA's Readjustment Counseling Service (RCS)-the office that oversees Vet Centers. We also interviewed 3In October 2020, Congress expanded eligibility for Vet Centers to include, among other groups, members of reserve components who served on active service in response to a national emergency or major disaster declared by the President. See 38 U.S.C. § 1712A(a)(1)(C) for currently eligible veterans and servicemembers and their families. 4Military sexual trauma refers to trauma resulting from sexual assault, battery, or harassment experienced during military service. 5Pub. L. No. 116-171, Tit. V, § 503, 134 Stat. 778, 818-19 (2020). Page 2 GAO-22-105039 VA Vet Centers officials from RCS's regional districts and five selected Vet Centers. We selected one Vet Center from each of the five districts. In addition, we selected each Vet Center for variation in geographic location-urban or rural status-and the presence of satellite locations." To select these Vet Centers, we used information from VA's Site Tracking system, which lists all VHA facilities and their characteristics. We assessed the reliability of these data by checking for missing values and obvious errors and reviewed documentation related to their purpose and structure. We determined that these data were sufficiently reliable for our purpose of selecting Vet Centers. From each selected Vet Center, we interviewed the director, counselors, and outreach staff. Information obtained from our interviews with these officials is not generalizable. Further, we interviewed representatives from two veterans service organizations-Vietnam Veterans of America and Iraq and Afghanistan Veterans of America- which we selected to ensure representation of different populations that are eligible for Vet Center services. To examine how VHA assesses whether Vet Centers are meeting the needs of their clients, we reviewed RCS procedures for its client feedback survey, as well as RCS procedures that Vet Center counselors follow for client intake and for documenting services provided. We interviewed officials from RCS, its districts, and the Vet Centers in our review about their efforts to understand whether Vet Center clients' needs were being met. We compared RCS's efforts to VHA's Directive 1500, which outlines the roles and responsibilities of Vet Centers, and to VHA's Health Equity Action Plan, which includes a goal to ensure the needs of different veteran subpopulations are being met.® We also compared these efforts 8VHA's 300 Vet Centers are organized into five regional districts, which are each led by a district director. The districts are responsible for overseeing the implementation of VA and VHA policies for RCS and supervising clinical and administrative staff at each of the Vet Centers within their region, among other things. Our review included the following five Vet Centers: (1) Huntington Vet Center (West Virginia); (2) Atlanta Vet Center (Georgia); (3) Chicago Vet Center (Illinois); (4) Alexandria Vet Center (Louisiana); and (5) Farmington Vet Center (New Mexico). 7Vet Centers also provide services at satellite locations, including outstations (typically in leased space), community access points (located in donated spaces, such as community centers or college campuses), and through Mobile Vet Centers (vehicles that Vet Centers operate to provide outreach and counseling in the community). 8Veterans Health Administration, Readjustment Counseling Service, VHA Directive 1500(2) (Washington, D.C.: Jan. 26, 2021, amended Dec. 30, 2021), and Department of Veterans Affairs, VHA Health Equity Action Plan FY 2021 Operational Plan (Washington, D.C.: Dec. 16, 2020). Page 3 GAO-22-105039 VA Vet Centers to federal standards for internal control for monitoring, information and communication, and control activities.2 We determined that the monitoring component of internal control was significant to this objective, along with the underlying principles that management should establish and operate monitoring activities to monitor the internal control system and evaluate the results, and remediate identified internal control deficiencies on a timely basis. In addition, we analyzed RCS data for fiscal years 2019 through 2021 on the number of clients who received counseling services, the focus areas of their treatment (e.g., post-traumatic stress disorder, marriage and family), and their demographic and eligibility characteristics (e.g., age or gender identity, conflicts they served in), as reported in RCSNet, RCS's client and visits database. We assessed the reliability of the data by checking for missing values and obvious errors and discussing them with RCS officials who were knowledgeable about the data. We found the data sufficiently reliable for our reporting purposes. To examine how Vet Centers tailor outreach activities and assess their effectiveness, we reviewed the most recent outreach plan available from each Vet Center in our review and RCS's policies concerning the development and implementation of the plans.1° We also interviewed Officials from RCS, its districts, and the Vet Centers in our review about their efforts to (1) ensure Vet Center outreach efforts are tailored to populations in need of services and (2) determine the effectiveness of Vet Center outreach activities. We compared RCS's and Vet Centers' efforts to VHA's Directive 1500, which outlines VHA's objectives for Vet Center outreach activities, and to key issues we previously identified for program 9GAO, Standards for Internal Control in the Federal Government, GAO-14-704G (Washington, D.C.: Sept. 10, 2014). Internal control is a process effected by an entity's oversight body, management, and other personnel that provides reasonable assurance that the objectives of an entity will be achieved. 10Vet Centers are required to develop annual community outreach plans to promote access and referral services that are tailored to the demographic distributions and sociocultural orientations of their assigned service areas. See Veterans Health Administration, VHA Directive 1500(2), amended Dec. 30, 2021. For three of the Vet Centers in our review, the most recent outreach plan was from fiscal year 2021; for the remaining two Vet Centers, the most recent outreach plan was from fiscal year 2020. Page 4 GAO-22-105039 VA Vet Centers evaluation.'1 We also compared these efforts to federal standards for internal control for information and communication, control activities, and monitoring. '2 We determined that the information and communication and monitoring components of internal control were significant to this objective, along with the underlying principles that management should use quality information to achieve the entity's objectives, establish and operate monitoring activities to monitor the internal control system and evaluate the results, and remediate identified internal control deficiencies on a timely basis. To examine what barriers to Vet Center care VHA has identified and steps it has taken to address them, we reviewed documents, such as RCS's reports to Congress on Vet Center activities for fiscal years 2019 through 2021, and procedures used to administer RCS's client feedback survey.13 We also interviewed officials from RCS, its districts, and the Vet Centers in our review about any barriers they had identified to Vet Center care, including those that may hinder the ability of veterans and servicemembers to obtain care and the ability of Vet Center counselors to provide care. We also asked about any steps they had taken to address those barriers. We also interviewed officials from VHA's Office of Mental Health and Suicide Prevention and the VHA medical facilities that are aligned with the Vet Centers in our review.14 We asked these officials about their coordination with RCS and Vet Centers and any related challenges they have experienced that may have affected the ability of veterans or servicemembers to obtain care or Vet Center counselors to provide care. In addition, we analyzed RCS data on Vet Center client visits for fiscal years 2019 through 2021, including the different modalities counselors 11Veterans Health Administration, VHA Directive 1500(2), amended Dec. 30, 2021, and GAO, Designing Evaluations: 2012 Revision, GAO-12-208G (Washington, D.C.: Jan. 31, 2012). 12GA0-14-704G. 13Department of Veterans Affairs, Department of Veterans Affairs Report on Activities of Readjustment Counseling Services in Fiscal Year 2019, (March 2020); Department of Veterans Affairs, Department of Veterans Affairs Report on Activities of Readjustment Counseling Services in Fiscal Year 2020, (March 2021); and Department of Veterans Affairs, Department of Veterans Affairs Report on Activities of Readjustment Counseling Services in Fiscal Year 2021, (March 2022). 14Per VHA policy, individual VA medical facilities provide clinical and administrative support to their aligned Vet Centers. See Veterans Health Administration, VHA Directive 1500(2), amended Dec. 30, 2021. Page 5 GAO-22-105039 VA Vet Centers used to provide services (e.g., the number of visits provided in person or by phone), when the visits occurred (e.g., non-traditional hours of operation, which includes visits before 8:00 a.m. or after 4:30 p.m., Monday through Friday or on weekends and holidays), and where the visits occurred (e.g., Vet Centers or their satellite locations). We assessed the reliability of these data by checking for missing values and obvious errors and discussing them with RCS officials who were knowledgeable about the data. We found the data sufficiently reliable for our reporting purposes. Barriers we report are those most commonly identified by officials, specifically officials from RCS and at least half of RCS's five districts and the five Vet Centers in our review and, as a result, may not be an exhaustive list of all potential barriers to Vet Center care. Further, we asked RCS officials about their efforts to understand the extent any identified barriers remain. We compared those efforts to VHA's Directive 1500 and VA's Fiscal Year 2018-2024 Strategic Plan, which includes a goal to make it easy for veterans to access benefits and services from VHA.''5 We also compared these efforts to federal standards for internal control for risk assessment, control activities, and monitoring. 1* We determined the risk assessment and monitoring components of internal control were significant to this objective, along with the underlying principles that management should identify, analyze, and respond to risks related to achieving the defined objectives, establish and operate monitoring activities to monitor the internal control system and evaluate the results, and remediate identified internal control deficiencies on a timely basis. We conducted this performance audit from February 2021 to May 2022 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. 15Veterans Health Administration, VHA Directive 1500(2), amended Dec. 30, 2021, and Department of Veterans Affairs, FY 2018-2024 Strategic Plan (February 2018, refreshed May 31, 2019). 16GAO-14-704G. Page 6 GAO-22-105039 VA Vet Centers Background The purpose of VHA's Vet Centers is to help eligible veterans and servicemembers and their families who experience challenges from deployment, combat, or other military-related trauma readjust to civilian life or to continued military service. Currently, veterans and servicemembers who have served in any combat theater or area of hostility are eligible for Vet Center services, as are their family members, at no cost to them.1" Vet Centers' services and structure are separate from health care provided at VHA medical facilities to better ensure privacy and confidentiality. Vet Centers have separate staff, medical records, and eligibility requirements. However, VHA medical facilities are required to align with nearby Vet Centers to provide clinical and administrative support.18 For example, VHA medical facilities provide clinical consultations for Vet Center counselors to discuss appropriate treatment for complex cases. Vet Center Locations, Services, and Clients In fiscal year 2021, there were 300 Vet Centers located in all 50 states, the District of Columbia, Puerto Rico, American Samoa, and Guam. Vet Centers augment their geographic reach in their local communities through several means. RCS maintains a fleet of 83 Mobile Vet Centers, which are vehicles that Vet Center staff operate to provide outreach and counseling in the community.19 Vet Center staff also provide services at satellite locations, including both outstations (typically in leased space) and community access points (located in donated spaces, such as community centers or college campuses).2° According to RCS, Vet Centers operated 18 outstations and 1,011 community access points in fiscal year 2021. Vet Centers provide readjustment counseling, which encompasses a range of services, including individual, group, family, and couples 1738 U.S.C. § 1712A(a)(1)(C) and Veterans Health Administration, VHA Directive 1500(2), amended Dec. 30, 2021. 18Veterans Health Administration, VHA Directive 1500(2), amended Dec. 30, 2021. 19One specific use of the Mobile Vet Centers is to provide outreach and readjustment counseling at active military, Reserve, and National Guard demobilization activities. In addition, in March 2020, RCS deployed Mobile Vet Centers to several cities in response to the COVID-19 pandemic as part of VA's Fourth Mission to respond to natural disasters and emergencies. 20Vet Center services are also augmented by the Vet Center Call Center, which is a 24- hour, confidential, national call center staffed by combat veterans. Page 7 GAO-22-105039 VA Vet Centers counseling, as well as counseling for post-traumatic stress disorder and military sexual trauma. Vet Centers can offer counseling through various modalities, including in person, by phone, or via VHA's video-capable telehealth platform. Group counseling visits may also take the form of recreational therapy activities, such as exercise classes and outdoor sports. In fiscal year 2021, RCS data show that Vet Centers provided about 1.5 million counseling visits to more than 107,000 clients. Based on our analyses of RCS's data, 28 percent of these clients had not received services from any Vet Center in the previous 2 years. RCS data show that the number of Vet Center counseling visits and clients decreased from fiscal year 2019-the most recent fiscal year prior to the COVID-19 pandemic-through fiscal years 2020 and 2021, during the COVID-19 pandemic. (See fig. 1.) RCS officials attributed the declines in visits and clients over this timeframe to the pandemic. Figure 1: Number of VHA Vet Center Clients and Visits, Fiscal Years 2019 through 2021 Number of Vet Center clients (in thousands) Number of Vet Center visits (in millions) Number of Vet Center visits (by type of visit) 140 2.00 130.7 120 117.0 1.75 1.657 107.4 1.560 4460 49,395 4.50 . Family 100 75,386 4.25 Couples 80 1.00 60 ye 432,580 Group 40 0.50 20 0.26 932,900 Individual 0 0.00 2019 2020 2021 2019 2020 2021 Fiscal year Fiscal year Source: GAO analysis of Veterans Health Administration's (VHA) Readjustment Counseling Service (RCS) data. | GAO-22-105039 Note: Fiscal year 2019 is the most recent fiscal year prior to the COVID-19 pandemic, and RCS officials attributed the declines in Vet Center clients and visits from fiscal years 2019 through 2021 to the pandemic. Page 8 GAO-22-105039 VA Vet Centers Vet Center clients represented various demographic and eligibility characteristics in fiscal year 2021, as noted below.21 e Gender identity. Eighty-four percent of clients identified as male, 16 percent as female, and less than 1 percent identified as transgender female, transgender male, non-binary, or other. Less than 1 percent did not have their gender documented in their client records. 22 e Age. Fifty-five percent of clients were between the ages of 30 and 59, 39 percent were aged 60 or older, and 5 percent of clients were aged 29 or younger. Less than 1 percent did not have their age documented in their client records. « Race/ethnicity.23 Fifty-nine percent of clients identified as White, 20 percent as African American, 12 percent as Hispanic or Latino, 2 percent as Asian American, 2 percent as Pacific Ilslander/Hawaiian, 1 percent as Alaskan Native or American Indian, and 1 percent as Other. Three percent did not have their race/ethnicity documented in their client records. e Eligibility. Some Vet Center clients were eligible for services because of their prior combat experience-e.g., 20 percent of them served in Iraq post-9/11. Others were eligible for reasons other than combat experience, such as those who experienced military sexual trauma. (See fig. 2.) 21Our analyses of client characteristics are based on data in RCSNet-RCS's client and visits database. Our reporting of these characteristics is based on how RCS categorizes its data. According to RCS, demographic characteristics are self-reported by clients, and some clients choose not to provide this information. According to VHA policy, client eligibility is to be confirmed by Vet Center staff. See Veterans Health Administration, VHA Directive 1500(2), amended Dec. 30, 2021. 22Totals do not sum to 100 due to rounding. 23According to RCS officials, some Vet Center clients report having more than one race and ethnicity. For reporting purposes, these clients were included in the counts for each race and ethnicity category. As a result, adding together the percentage of clients from the different categories will not result in 100. Page 9 GAO-22-105039 VA Vet Centers Figure 2: Percentage of VHA Vet Center Clients by Selected Eligibility Categories, Fiscal Year 2021 Served in Afghanistan (post- 9/11) Served in Iraq (post- 9/11) Served in Operation Desert Storm/ Desert Shield Served in " Vietnam War PASI Served in World War Il Experienced military sexual trauma 0 5 10 15 20 25 30 Percentage of Vet Center clients Source: GAO analysis of Veterans Health Administration's (VHA) Readjustment Counseling Service (RCS) data. | GAO-22-105039 Note: According to RCS data, Vet Centers provided readjustment counseling to 107,367 clients in fiscal year 2021. These selected eligibility categories are reported based on how RCS categorizes its data, which includes 24 categories. According to RCS officials, some Vet Center clients were eligible for services based on more than one eligibility category, such as an individual who experienced combat and experienced military sexual trauma (trauma resulting from sexual assault, battery, or harassment experienced during military service). This would be included in the counts for each category. As a result, adding together the percentage of clients from the different eligibility categories will not result in 100. Vet Centers also conduct outreach to contact and engage local eligible individuals and bring them into Vet Centers for services. Outreach efforts may include contacting eligible individuals and family members, contacting local service providers and civic leaders, and building referral networks with community providers. Vet Centers conducted 35,347 outreach activities in fiscal year 2019, 28,624 activities in fiscal year 2020, and 32,415 activities in fiscal year 2021, according to data from RCS. RCS and Vet Center Organizational Structure RCS is the office within VHA that oversees Vet Centers. It is led by a Chief Officer who is responsible for strategic planning for Vet Centers and for ensuring coordination of readjustment counseling services with other VA program offices, such as VHA's Office of Mental Health and Suicide Prevention, which provides policy support for mental health services. RCS's 300 Vet Centers are organized into five regional districts, which are each led by a district director. Each Vet Center is managed by a Vet Center director, who is responsible for the day-to-day oversight of the Vet Center's staff. According to RCS, Page 10 GAO-22-105039 VA Vet Centers VHA Assesses Whether Vet Centers Are Meeting the Readjustment Counseling Needs of Clients Individually but Not Collectively Vet Centers have an average of six to seven total staff members, consisting of at least one counselor, an outreach staff person, and a program support assistant. Vet Center counselors are multi-disciplinary and have various professional licensures. The counselors include psychologists, social workers, licensed professional counselors, or marriage and family therapists. Vet Center outreach staff work in the community to create connections with eligible veterans and servicemembers and their families and to engage with community leaders and service providers to provide information about available Vet Center services. VHA Assesses the Readjustment Counseling Needs of Individual Clients and Whether Those Needs Are Being Met VHA''s RCS and Vet Centers use psychosocial assessments and feedback surveys to assess individual client readjustment counseling needs and whether these clients' needs are being met throughout the course of counseling. e Psychosocial assessments. According to VHA policy, Vet Center counselors must identify and document each individual client's counseling needs at intake and assess their progress towards meeting the client's needs following each visit.24 Counselors are to identify client needs by administering a psychosocial assessment during intake. The assessments help the counselors understand the 24Veterans Health Administration, VHA Directive 1500(2), amended Dec. 30, 2021. Page 11 GAO-22-105039 VA Vet Centers client's psychological, familial, social, and other needs.25 Using the results of the assessment, counselors are to identify psychosocial focus areas for care, such as post-traumatic stress disorder, marital and family issues, or bereavement. Counselors are to then develop a service plan for how they will address the client's counseling needs. They are also to document their progress towards meeting those needs in the client's record following each visit. Officials from the five Vet Centers in our review described using the results from the psychosocial assessments to help understand their clients' needs. They also told us they use these results to monitor treatment progress, such as by periodically reviewing their clients' treatment plans and making revisions, as needed. According to RCS data, Vet Center counselors provided care related to a variety of psychosocial focus areas in fiscal year 2021. (See fig. 3.) For example, according to RCS data, 73 percent of Vet Center clients in fiscal year 2021 received care for symptoms associated with post-traumatic stress disorder. 25According to VHA policy, RCS districts are to conduct annual clinical and administrative site visits to each of the Vet Centers to ensure staff compliance with RCS policies and procedures for the administration and provision of readjustment counseling. According to RCS, the districts are to review whether counselors are routinely meeting the requirements for the psychosocial assessments, such as by reviewing whether the Vet Center has at least 90 percent of their active cases having a psychosocial assessment completed by the fifth client visit. See Veterans Health Administration, VHA Directive 1500(2), amended Dec. 30, 2021. VA's Office of Inspector General reported in 2021 that it could not evaluate whether any of the 20 Vet Centers in its reviews had met this threshold for completing the psychosocial assessments because documentation of completion dates were lacking. Therefore, completion of the psychosocial assessments by the fifth visit could not be evaluated. In response to this finding, district officials said they would, among other efforts, provide additional training to Vet Center directors and counselors on completion of the intake portion of the psychosocial assessments. See Department of Veterans Affairs, Office of Inspector General, Vet Center Inspection of Continental District 4 Zone 2 and Selected Vet Centers, Report #20-04051-287 (Sept. 30, 2021); Vet Center Inspection of Pacific District 5 Zone 1 and Selected Vet Centers, Report #21-01805-286 (Sept. 30, 2021); Vet Center Inspection of Southeast District 2 Zone 2 and Selected Vet Centers, Report #20- 02014-270 (Sept. 30, 2021); Vet Center Inspection of Continental District 4 Zone 1 and Selected Vet Centers, Report #20-04050-37 (Dec. 2, 2021); and Vet Center Inspection of Pacific District 5 Zone 2 and Selected Vet Centers, Report #21-01804-56 (Dec. 20, 2021). Page 12 GAO-22-105039 VA Vet Centers Figure 3: Percentage of VHA Vet Center Clients by Selected Psychosocial Focus Areas, Fiscal Year 2021 Post-traumatic stress disorder Marital and family issues Military sexual trauma?® Bereavement? Drugs and alcohol abuse 0 10 20 30 40 50 60 70 80 Percentage of Vet Center clients Source: GAO analysis of Veterans Health Administration's (VHA) Readjustment Counseling Service (RCS) data. | GAO-22-105039 Note: According to RCS data, Vet Centers provided readjustment counseling to 107,367 clients in fiscal year 2021. Psychosocial focus areas describe the areas of care provided by Vet Center counselors. Examples of other psychosocial focus areas not included in this figure include crisis, homelessness, and benefits. According to RCS officials, some clients receive care related to more than one psychosocial focus area. For reporting purposes, these clients were included in the counts for each psychosocial focus area. As a result, adding together the percentage of clients receiving care for the different psychosocial focus areas will not result in 100. *Military sexual trauma refers to trauma resulting from sexual assault, battery, or harassment experienced during military service. >Bereavement refers to counseling services that are specifically for immediate family members, including parents, of servicemembers killed in the line of duty. e Client feedback survey responses. Officials from RCS, all of RCS's districts, and the five Vet Centers in our review told us they use responses to RCS's Vet Center client feedback survey to help them understand whether they are meeting individual client needs. RCS officials told us they implemented a new client feedback survey in July 2021. They told us this survey seeks to obtain information from Vet Center clients regarding their satisfaction with services at various stages of the treatment process, including prior to their intake and throughout the course of their treatment.26 For example, the survey asks clients who are engaged in treatment the extent to which the Vet Center provided or connected them to services or resources, helped them to meet their goals, or helped them to improve their quality of life. Some of these officials told us they can use the responses to these surveys to identify when an individual client's needs were not 26Prior to the implementation of its new client feedback survey in July 2021, RCS used a different client feedback survey that was distributed to clients who had been disengaged from Vet Center services for more than 90 days. Page 13 GAO-22-105039 VA Vet Centers met. In such cases, they told us they follow up with the Vet Center staff to reach out to the client for a resolution. For example, officials from one district told us that when reviewing the feedback survey results, they notify the associated Vet Center director of any negative comments and ask the director to develop and provide the district with a response for how they plan to address the identified concerns. VHA Does Not Assess the Extent to Which Vet Centers Are Meeting the Readjustment Counseling Needs of Clients Collectively VHA's RCS does not assess the extent to which Vet Centers are meeting the readjustment counseling needs of clients collectively, including client subpopulations that may experience different challenges with readjustment to civilian life or to continued military service. RCS and Vet Centers assess individual client readjustment counseling needs and whether these clients' needs are being met throughout the course of counseling using psychosocial assessments and feedback surveys. However, RCS has not used this information to assess the extent to which Vet Centers are meeting the needs of their clients collectively. For example, RCS has not analyzed information from the psychosocial assessments or feedback surveys to assess what proportion of Vet Center clients is making progress in having their needs met. Such assessments would improve RCS's understanding of how effective Vet Centers are in meeting the needs of their clients. Similarly, we found that RCS has not assessed the extent to which there are differences in the progress being made among different client subpopulations. According to VHA, different subpopulations of veterans may experience different challenges with readjusting to civilian life. Examples of such subpopulations include veterans who experienced a traumatic brain injury or military sexual trauma during military service.27 Further, the Advisory Committee on the Readjustment of Veterans reported that the readjustment counseling needs of veterans of recent conflicts (e.g., Afghanistan and Iraq post-9/11) are different than those of 27Department of Veteran Affairs (VA), Health Services Research and Development Service, "Spotlight on Post-Deployment Health" (Washington, D.C.: January 2022), accessed Jan. 19, 2022, https://Awww.hsrd.research.va.gov/news/feature/post-deployment- health.cfm. Traumatic brain injury can result from a blow to the head by an object or an impact to the head by a nearby blast or explosion. The injury can range from mild to severe, and it may increase risk for mental health conditions such as anxiety and depression, as well as for sleeping problems. Page 14 GAO-22-105039 VA Vet Centers veterans of the Vietnam War.28 RCS data show that 28 percent of Vet Center clients in fiscal year 2021 served in the Vietnam War, 20 percent served in lraq (post-9/11), and 11 percent served in Afghanistan (post- 9/11). RCS officials told us they do not currently conduct assessments of the extent to which Vet Centers are meeting the needs of their clients collectively, including client subpopulations because they do not currently have a way to do so. In February 2022, officials described efforts they have underway that could result in a process for conducting such assessments. Specifically, officials told us about a workgroup with VHA's Health Services Research & Development Service that is developing quality measures that RCS may be able to use to assess the extent to which Vet Centers are meeting the needs of their clients collectively. However, according to RCS officials, this effort is in the early stages. RCS officials also told us they plan to analyze responses to their new client feedback survey to determine the extent to which Vet Centers are meeting the needs of their clients collectively, including client subpopulations. In responding to a draft of this report, RCS noted that it will take time for it to receive a sufficient number of responses to be able to conduct analyses of demographic subpopulations. In addition, the documentation officials provided to us did not clearly specify how these efforts would lead to a process for assessing the extent to which Vet Centers are meeting the needs of their clients collectively. Per VHA's Directive 1500, RCS is responsible for ensuring Vet Centers are achieving their intended purpose to help clients overcome challenges to achieve successful readjustment to civilian life or to continued military service.29 In addition, VHA's Health Equity Action Plan includes a goal to ensure the needs of different veteran subpopulations are being met.%° Further, according to federal standards for internal control for monitoring, management should establish and operate monitoring activities to assess the quality of its performance over time and evaluate the results. These 28The Advisory Committee on the Readjustment of Veterans was established pursuant to 38 U.S.C. § 545 and advises the Secretary of VA on the provision of benefits and services to assist veterans in the readjustment to civilian life. It is responsible for providing a Congressionally-mandated report to the Secretary of VA each year. 28Veterans Health Administration, VHA Directive 1500(2), amended Dec. 30, 2021. 30Department of Veterans Affairs, VHA Health Equity Action Plan FY 2021 Operational Plan (Washington, D.C. Dec. 16, 2020). Page 15 GAO-22-105039 VA Vet Centers Vet Centers Develop Outreach Plans but Lack Data to Better Tailor Outreach Activities and Guidance to Assess Their Effectiveness standards specify that monitoring occur on an ongoing basis and these evaluations are used periodically. Agencies can use these evaluations to determine the effectiveness of its programs or the need for any corrective actions to achieve their objectives-such as helping veterans and servicemembers achieve successful readjustment.>' Taking steps to implement a process to periodically assess the extent to which Vet Centers are meeting the needs of clients collectively, including various subpopulations, would improve RCS's understanding of how effective Vet Centers are in meeting the needs of their clients. Such a process may include or be informed by the efforts RCS has underway or by other efforts RCS may develop to understand whether client needs are being met. Such a process would also help RCS identify and determine whether any actions, such as providing a new type of service, are needed to help ensure Vet Centers are able to help their clients achieve successful readjustment to civilian life or to continued military service. Vet Centers Are Required to Develop Annual Outreach Plans Tailored to Their Service Area Populations Vet Center outreach staff are responsible for developing an annual outreach plan that identifies populations that may be in need of Vet Center services to help better focus the staffs outreach activities throughout the community. According to VHA policy, the outreach plans are required to be tailored to the demographic distributions and sociocultural makeup of the Vet Center's service area.32 This includes tailoring outreach to maximize contacts with local eligible individuals of all genders, ethnicities, ages, and combat experiences. As part of these plans, Vet Center outreach staff are also required to identify outreach 31GAO0-14-704G. 32Veterans Health Administration, VHA Directive 1500(2), amended Dec. 30, 2021. Page 16 GAO-22-105039 VA Vet Centers activities to conduct and community stakeholders to work with, such as local veterans service organizations. According to officials from RCS, RCS's five districts, and the five Vet Centers in our review, Vet Center outreach staff engage with community stakeholders or use data available to them on the veterans in their service area, such as from the U.S. Census, to identify populations for their outreach activities.33 Specifically, they said that they do the following. e Engage community stakeholders. According to officials from RCS's districts and the five Vet Centers in our review, Vet Center outreach staff work with various community stakeholders, such as local veterans, VHA medical center staff, and representatives of veterans service organizations and local community organizations and agencies to identify populations for Vet Center outreach activities. Further, we found that all five of the Vet Centers identified in their outreach plans specific community stakeholders to engage with as part of their outreach to specific populations. For example, one Vet Center's outreach plan stated that the outreach staff would partner with staff from local colleges and universities to inform them and their student veteran populations about Vet Center services in an effort to increase services to those who served in more recent conflicts. e Use available data. According to officials from RCS, two of RCS's districts, and three Vet Centers in our review, outreach staff from some Vet Centers use U.S. Census data to inform their outreach. For example, one Vet Center's outreach plan included a county-level map of its service area and data from the U.S. Census on the total number of veterans in each county. Officials from this Vet Center told us outreach staff use these data to help determine the rural areas to focus outreach. Vet Centers Lack Some Data on Their Service Area Population to Better Tailor Outreach Activities Vet Centers do not have some data on the veterans eligible for services in their service area that could help them better tailor their outreach activities. Vet Centers may obtain some data on the veterans in their service area, such as the total number of veterans at the county-level from the U.S. Census. However, district and Vet Center officials told us that these data are not sufficient to allow them to tailor their outreach. They told us this is because the data do not include information on those 33According to VHA policy, although outreach staff have primary responsibility for Vet Center outreach activities, all Vet Center staff, including directors, maintain some responsibility for outreach. See Veterans Health Administration, VHA Directive 1500(2), amended Dec. 30, 2021. Page 17 GAO-22-105039 VA Vet Centers veterans that are eligible for Vet Center services, such as how many recently transitioned back to civilian life. According to officials from three of RCS's districts and all five of the Vet Centers in our review, Vet Centers' efforts to tailor their outreach would be improved if they had better data on the veterans in their service area. Specifically, these officials told us it would be helpful if Vet Centers had data on the veterans in their service area who are eligible for Vet Center services, including how many of them recently transitioned back to civilian life or experienced combat, and their demographic characteristics (e.g., age, gender, race, and ethnicity). These officials told us having such data would help them better determine where to focus their outreach activities, including community stakeholders they could work with as part of their outreach (e.g., schools and colleges for younger veterans). It is important for Vet Center outreach staff to be able to make contact with certain veteran populations, such as those who recently returned from conflict and are younger, because, according to VA, such populations are at higher-risk of suicide. *4 RCS officials agreed that it is important for Vet Centers to have these data. Officials told us they are currently working with staff from VHA's Office of Enrollment and Forecasting on a projection model that will result in data that they intend to use to identify areas where there are gaps in Vet Center services, and that Vet Centers can use for outreach planning. For example, officials told us they have received some projection model data that will help them estimate the future demand for Vet Center visits at the zip code level. Officials also told us they have received some data from the model that they can use to calculate the number of those eligible for Vet Center services. Having better data on those eligible for Vet Center services in their service areas could help Vet Centers with their efforts to meet RCS's primary objective for Vet Center outreach-to contact, inform, engage, and bring local eligible individuals into Vet Centers for needed services- as outlined in VHA's Directive 1500.35 Further, federal standards for internal control for information and communication state that agencies 34According to VA, veterans between the ages of 18 and 29 have the highest rate of suicide and recent transition from military service to civilian life is a risk factor for suicide. See Department of Veterans Affairs, National Strategy for Preventing Veteran Suicide 2018-2028 (Washington, D.C.: June 29, 2018). 35Veterans Health Administration, VHA Directive 1500(2), amended Dec. 30, 2021. Page 18 GAO-22-105039 VA Vet Centers should use quality information to achieve their objectives.36 These standards specify that management identifies information requirements in an iterative and ongoing process and obtains relevant data from reliable internal and external sources in a timely manner. With improved data on their service area populations, Vet Centers-as well as RCS and its districts-could better ensure that Vet Center outreach is appropriately tailored to those who may be in need of services. Vet Centers Lack Guidance to Assess the Effectiveness of Their Outreach Activities Vet Centers do not have guidance to follow for assessing the effectiveness of their outreach activities. Specifically, RCS has not provided Vet Centers with guidance that includes metrics for their outreach activities or targets against which to assess those metrics to determine effectiveness. Prior GAO work has identified key practices for evaluating public outreach campaigns that include establishing metrics and targets, which allows agencies to track their progress towards specific goals. 37 According to officials from RCS, its five districts, and the five Vet Centers in our review, Vet Center staff use information they collect from their outreach activities to try to assess the activities' effectiveness but experience challenges when doing so. According to RCS, Vet Center staff collect and document certain information about each outreach activity, including the location and audience type (e.g., veterans, servicemembers, families), certain information about each contact made during an outreach activity (e.g., the individual's first and last name, birth year, and last four digits of the Social Security number), and the total number of outreach contacts made. However, RCS officials told us it may not be appropriate to use this information to assess effectiveness, because it often takes time-sometimes years-for individuals to come into a Vet Center for services following an outreach activity, and it is unclear whether to credit the earlier outreach activity with an individual coming in for services. Officials from RCS, all five of RCS's districts, and the five Vet Centers in our review also told us that, in some cases, the number of outreach contacts made may not be an appropriate metric for assessing an outreach activity's effectiveness. For example, officials from one district told us it may be reasonable for Vet Centers to maintain a strategy where 36GAO-14-704G. 37GAO, Digital Television Transition: Increased Federal Planning and Risk Management Could Further Facilitate the DTV Transition, GAO-08-43, (Washington, D.C.: Nov. 19, 2007). Page 19 GAO-22-105039 VA Vet Centers certain activities do not generate a lot of contacts, such as when developing a relationship with a specific community (e.g., Native Americans) with which it can take time to develop trust. Further, it can be challenging for Vet Centers to accurately count outreach contacts because, according to officials from RCS, three RCS districts, and four Vet Centers in our review, it can be difficult for Vet Centers to verify the identity of some contacts.38 Without the information necessary to verify a contact, it can be challenging for Vet Centers to accurately determine whether that contact is a "new" contact (i.e. someone who has not been at another outreach activity for that Vet Center) or link that contact to a particular outreach activity if the person comes in for services. Officials from three districts and all five of the Vet Centers in our review told us it would be helpful for RCS to provide them with guidance on how to assess the effectiveness of Vet Center outreach activities. Officials told us such information would also help Vet Centers in their outreach planning. For example, officials from two Vet Centers told us that having guidance to determine which activities are more or less effective will help them make decisions about which types of activities to conduct and when and how often to conduct them. Officials from three of the districts and three of the Vet Centers in our review also told us any guidance RCS provides should include metrics and goals or targets against which to assess those metrics to determine effectiveness. However, RCS officials told us they are unable to provide such guidance because they have not developed metrics for Vet Center outreach activities or targets against which to assess those metrics to determine effectiveness. Officials told us identifying appropriate metrics and targets for outreach has been difficult and said they have been working towards developing them. Specifically, in February 2022, officials told us a VHA workgroup is working to develop potential metrics that could be used to assess the effectiveness of Vet Center outreach.°9 However, according to RCS officials, these efforts are in the early stages. In addition, the documentation officials provided to us did not clearly specify how this 38According to RCS, Vet Center staff use the last name, first name, and last four digits of the Social Security number to verify an outreach contact's identity. RCS officials told us outreach contacts are not required to provide any of this information and, given the sensitive nature of this information, some choose not to provide it. 39RCS officials told us this workgroup is the same workgroup that-as previously described-is in the early stages of developing potential quality measures that RCS may be able to use to assess the extent to which Vet Centers are meeting the needs of their clients collectively. Page 20 GAO-22-105039 VA Vet Centers effort will result in guidance for Vet Centers for assessing the effectiveness of their outreach. Providing Vet Centers with guidance that includes metrics and targets to assess the effectiveness of their outreach would help Vet Centers develop and implement their outreach plans more effectively, such as by helping them determine which types of activities to conduct and when and how often to conduct them. Federal standards for internal control for monitoring require agencies to assess the quality of their performance by evaluating the results of activities.4° Agencies can then use these evaluations to determine the effectiveness of their programs or need for any corrective actions. In addition, we have outlined in prior work key issues in planning evaluations of federal programs to best meet the needs of decision makers, including the development of appropriate performance goals and measures and the development of quantifiable, numerical targets for them.41 Such guidance for Vet Centers would help them develop and implement their outreach plans more effectively. It would also help Vet Centers, as well as RCS and its districts, ensure Vet Center outreach is effective, thereby ensuring a greater awareness of Vet Center services among those who are eligible for services and need help achieving successful readjustment. 40GAO-14-704G. 41GAO-12-208G. Page 21 GAO-22-105039 VA Vet Centers VHA Has Identified and Taken Some Steps to Address Barriers to Vet Center Care, but Does Not Know the Extent to Which These Barriers Remain VHA Has Identified and Taken Some Steps to Address Barriers to Vet Center Care VHA-Identified Barriers Officials from VHA's RCS, its districts, and the five Vet Centers in our review identified several barriers to Vet Center care. The officials most commonly identified two barriers that hinder the ability of some veterans and servicemembers to obtain Vet Center services: (1) a lack of awareness about Vet Center services and (2) challenges accessing those services. They also identified a third barrier, (3) challenges to counselors receiving clinical consultations for clients, that hinders the ability of Vet Center staff to provide services.42 1. Lack of awareness about Vet Centers. Some veterans, servicemembers, or VHA medical facility staff may be unaware of Vet Centers, according to officials from RCS, three districts, and all five Vet Centers in our review. For example, some of these officials told us that some veterans and servicemembers may be unaware of what services Vet Centers provide or their eligibility for services. In addition, 42These barriers were identified by officials from RCS and at least half of RCS's five districts and the five Vet Centers in our review. Others have also identified similar barriers to VHA mental health care, including awareness of eligibility for services and access to services. For example, see National Academies of Sciences, Engineering, and Medicine 2018, Evaluation of the Department of Veterans Affairs Mental Health Services (Washington, DC: The National Academies Press). A smaller set of officials also identified other barriers. For example, officials from one district and three Vet Centers identified barriers to care related to the COVID-19 pandemic. Page 22 GAO-22-105039 VA Vet Centers some VHA medical facility staff did not understand the services Vet Centers provide. As a result, these staff would not have been able to inform the veterans with whom they interact about their potential eligibility for Vet Center services or refer veterans to services that could help them readjust to civilian life. 2. Challenges accessing Vet Center services. Some veterans and servicemembers have had difficulty accessing Vet Center services, according to officials from RCS, its districts, and all five Vet Centers in our review. For example, officials told us that some clients had difficulty getting to a Vet Center because they lack transportation or live many miles away from a Vet Center or one of its service locations (an outstation, community access point, or Mobile Vet Center). For example, officials from one Vet Center told us it takes as long as 2 hours for some veterans in rural areas to get to the Vet Center for an appointment. They also described challenges some veterans face accessing services in urban areas, such as challenges with the availability of public transportation or parking and concerns over their safety when they must travel through possibly unsafe neighborhoods to reach the Vet Center. In addition, some officials told us that some veterans cannot get to the Vet Center during its traditional hours of operation (8:00 a.m. to 4:30 p.m., Monday through Friday). For example, officials from one Vet Center told us some veterans' work and school schedules overlap with the Vet Center's hours of operation, making it difficult for these veterans to come to the Vet Center for needed care during these hours. 3. Challenges to counselors receiving clinical consultations. Some Vet Center counselors have experienced challenges receiving Clinical consultations to discuss care for their complex cases, according to officials from RCS, three districts, and three Vet Centers in our review. According to VHA policy, Vet Center counselors are to work with an external consultant for at least 4 hours each month to discuss care or coordinate support needed for their complex cases.*3 This policy 43According to VHA policy, counselors at each Vet Center are required to receive at least 4 hours of clinical consultations from an external consultant each month to discuss appropriate treatment for complex cases, such as for clients at intermediate or high risk for self-harm or harm to others, clients who present with significant traumatic brain injury, or those suspected or identified as having a serious mental health condition not appropriate for Vet Center services. The consultations must occur onsite at the Vet Center or via telehealth. These consultations are in addition to those that individual Vet Center counselors may have with VA medical facility colleagues to coordinate care and referrals for individual veterans. See Veterans Health Administration, VHA Directive 1500(2), amended Dec. 30, 2021. Page 23 GAO-22-105039 VA Vet Centers requires that the external consultant be a qualified mental health professional assigned by the VHA medical facility aligned with the Vet Center.*4 In cases where this facility is unable to provide a consultant, the Vet Center may contract for the services from the private sector. However, officials from RCS, three districts, and three Vet Centers told us that, in some cases, Vet Center counselors experienced delays in receiving, or did not receive at all, the required hours of consultations each month. For example, officials told us this has occurred because the facility's staff were unavailable to provide the consultation hours, or because the process for obtaining qualified professionals from the private sector was difficult and time- consuming. Officials from the VHA medical facility aligned with one of the Vet Centers in our review described some delays in providing Vet Center counselors with these consultations because of challenges with staff availability, such as during staff transitions or medical leaves. Some of the district and Vet Center officials told us that these consultations have helped Vet Center counselors provide more effective care, and that the lack of them can hinder the ability of counselors to determine appropriate services for complex cases. Example of a Vet Center's Clinical Consultation for Complex Cases Per VHA policy, each Vet Center is required to receive at least 4 hours each month of regularly scheduled clinical consultations with an external, licensed mental health professional. These consultations provide Vet Center counselors the opportunity to discuss care or coordinate support needed for their complex cases. An official from one Vet Center in our review described their most recent clinical consultation, during which counselors presented six complex cases that included clients who were experiencing challenges with readjustment due to post-traumatic stress resulting from combat, grief and loss, and other issues. Counselors presented to the consultant each client's psychosocial histories, care plan, and treatment challenges. In addition to the consultant providing expertise, the counselors also received feedback and support from one another. The consultant helped each counselor evaluate each client's care and develop a plan to improve that care. The Vet Center official said that the counselors find these consultations to be very valuable because they result in more effective client care. Source: Veterans Health Administration (VHA) policy and an official from one Vet Center. | GAOQ-22-105039 44According to VHA policy, an external consultant must be independently licensed and have completed the VA credentialing process. See Veterans Health Administration, VHA Directive 1500(2), amended Dec. 30, 2021. Page 24 GAO-22-105039 VA Vet Centers VHA Steps to Address Barriers VHA's RCS, its districts, and the five Vet Centers in our review have taken several steps to address barriers to Vet Center care, according to officials we interviewed. 1. Addressing lack of awareness about Vet Center services. Vet Centers conduct outreach to veterans and a variety of stakeholders to inform them of Vet Center services. RCS officials told us they are coordinating with the Department of Defense, including the National Guard Bureau, to create awareness about Vet Center services among those newly eligible for their services. For example, as part of a 2019 memorandum of understanding between RCS and the National Guard Bureau, Vet Center staff are allowed to conduct outreach activities for National Guard members during drill weekends to address the challenges faced by transitioning servicemembers. In addition, officials from three districts and five Vet Centers told us that Vet Center staff meet with local VHA medical facility staff to increase awareness about their services, such as during local mental health care coordination meetings or by reaching out to individual facility staff.45 Further, RCS began a rebranding strategy for Vet Centers in fiscal year 2020 that, according to RCS officials, will include outreach materials tailored for specific groups, such as promotional videos and public service announcements intended to reach younger veterans. Addressing challenges accessing Vet Center services. For those who experience challenges getting to Vet Centers, RCS has implemented options to increase access to Vet Center services. Specifically, Vet Centers provide services (1) through satellite locations (outstations, community access points, and Mobile Vet Centers), (2) during non-traditional hours (including visits before 8:00 a.m. or after 4:30 p.m., Monday through Friday, or on weekends and holidays), and (3) via phone or telehealth. RCS data show that the number of Vet Center visits provided by phone or telehealth significantly increased from fiscal year 2019-the most recent fiscal year prior to the COVID-19 pandemic-through fiscal year 2021. The number of visits provided at satellite locations and during non- traditional hours of operation decreased during the same time period. (See table 1.) RCS officials told us Vet Centers' use of telehealth to provide care has helped address different access barriers. For example, the number of younger and working clients using virtual services has increased because telehealth allows clients to schedule 45Officials we spoke with from the VHA medical facilities aligned with the five Vet Centers in our review also described efforts to educate their staff about Vet Center services, such as inviting Vet Centers to make presentations at the facility's new-employee orientations. Page 25 GAO-22-105039 VA Vet Centers appointments when it is convenient for them, such as during a lunch break at work, according to RCS. RCS officials also told us that telehealth has helped Vet Centers provide services to clients in rural areas with transportation challenges and enabled clients to access services during the COVID-19 pandemic.4® Table 1: Selected Characteristics of VHA Vet Center Client Visits, Fiscal Years 2019 through 2021 Number (percentage) of visits Client visit characteristics Fiscal year 2019 Fiscal year 2020 Fiscal year 2021 Modality: In-person 1,472,083 (89) 790,159 (51) 376,662 (25) Through phone or telehealth# 184,806 (11) 769,075 (49) 1,113,552 (75) Mixed 40 (0) 1,012 (0) 47 (0) Location: Vet Centers 1,553,626 (94) 1,465,913 (94) 1,437,550 (96) Satellite Locations® 103,303 (6) 94,333 (6) 52,711 (4) Hours of operation: Traditional hours 1,342,453 (81) 1,318,323 (84) 1,280,060 (86) Non-traditional hours® 314,476 (19) 241,923 (16) 210,201 (14) Source: GAO analysis of Veterans Health Administration's (VHA) Readjustment Counseling Service (RCS) data. | GAO-22-105039 Note: According to RCS data, in fiscal year 2019-the most recent fiscal year prior to the COVID-19 pandemic-Vet Centers provided 1,656,929 counseling visits. In fiscal years 2020 and 2021, they provided 1,560,246 and 1,490,261 counseling visits, respectively. RCS officials attributed the decline in Vet Center counseling visits over this timeframe to the pandemic. *Telehealth visits are provided virtually through a video-capable device using VHA's telehealth platform, According to RCS officials, the large increase in the amount of visits provided through phone or telehealth is, in part, due to the COVID-19 pandemic requiring Vet Centers to limit in-person visits. >Fewer than 0.01 percent of visits were mixed visits, which are group counseling visits where one or more clients attend in person and one or more clients attend through phone or telehealth. "Vet Center satellite locations include outstations (typically in leased space), community access points (located in donated spaces, such as community centers or college campuses), and Mobile Vet Centers (vehicles that Vet Center staff operate to provide counseling in the community). 'Visits that occurred during non-traditional hours of operation are those that occurred before 8:00 a.m. or after 4:30 p.m., Monday through Friday, or on weekends or holidays. 3. Addressing challenges to counselors receiving clinical consultations. RCS officials told us that challenges to Vet Center 46RCS officials told us that that all Vet Center counselors have been trained to use telehealth to provide services. Officials from RCS, one district, and one Vet Center we spoke with also told us that they have provided training and support to help clients use VHA's telehealth platform. Page 26 GAO-22-105039 VA Vet Centers counselors receiving clinical consultations are addressed by district staff on a case-by-case basis. These officials told us that district staff generally identify such challenges during their annual Vet Center site visits, which is when they review, among other matters, whether the Vet Center has an external consultant and has been receiving at least 4 hours each month of clinical consultations.*" Officials from RCS and three districts told us they have taken actions to address any identified challenges. For example, officials from one district told us that, in some situations when medical facility staff have not been available to provide these consultations to Vet Center counselors, qualified staff from the district have provided the consultations. VHA Does Not Know the Extent to Which Barriers to Vet Center Care Remain Although VHA's RCS, its districts, and the Vet Centers in our review have taken some steps to address the barriers to Vet Center care they have identified, we found that RCS does not know the extent to which these barriers remain. Specifically, RCS does not know approximately how many veterans and servicemembers, eligible for Vet Center services, are not aware of, or are experiencing challenges accessing, those services. Similarly, RCS does not know how many counselors across all of the Vet Centers are unable to provide appropriate services in a timely manner because, for example, they experience challenges receiving clinical consultations for their complex cases. RCS does not know the extent to which the identified barriers to Vet Center care remain because it does not have processes for assessing their extent. However, RCS officials told us they have begun to develop efforts to assess the extent to which these barriers remain. For example, RCS officials told us they plan to analyze responses from RCS's newly implemented client feedback survey to assess how many eligible 47According to VHA policy, districts are to conduct annual clinical and administrative site visits to each of the Vet Centers to ensure staff compliance with RCS policies and procedures for the administration and provision of readjustment counseling. See Veterans Health Administration, VHA Directive 1500(2), amended Dec. 30, 2021. VA's Office of Inspector General reported in 2021 that none of the 20 Vet Centers in its reviews had received the required number of consult hours each month. See Department of Veterans Affairs, Office of Inspector General, Vet Center Inspection of Continental District 4 Zone 2 and Selected Vet Centers, Report #20-04051-287 (Sept. 30, 2021); Vet Center Inspection of Pacific District 5 Zone 1 and Selected Vet Centers, Report #21- 01805-286 (Sept. 30, 2021); Vet Center Inspection of Southeast District 2 Zone 2 and Selected Vet Centers, Report #20-02014-270 (Sept. 30, 2021); Vet Center Inspection of Continental District 4 Zone 1 and Selected Vet Centers, Report #20-04050-37 (Dec. 2, 2021); and Vet Center Inspection of Pacific District 5 Zone 2 and Selected Vet Centers, Report #21-01804-56 (Dec. 20, 2021). Page 27 GAO-22-105039 VA Vet Centers veterans or servicemembers may lack awareness about, or have challenges accessing Vet Center services. However, in February 2022, RCS officials told us they were in the early stages of this effort. In addition, the documentation officials provided to us did not clearly specify how this effort would result in or contribute to a process to assess the extent of identified awareness and access barriers. In addition, RCS officials told us that RCS districts identify challenges that counselors have to receiving clinical consultations as part of the districts' annual site visits to individual Vet Centers. However, based on our review of the site visit protocols and interviews with RCS officials, we found this effort does not assess the extent to which this and other such challenges are present across all Vet Centers. In February 2022, RCS officials told us they have efforts underway that may allow them to conduct such assessments, including updates to the system the districts use to collect information during their annual site visits. However, according to RCS Officials, these efforts are in the early stages. In addition, the documentation RCS provided to us did not clearly specify how its efforts would lead to a process for assessing the extent of barriers Vet Center staff may encounter to providing services, including challenges to receiving clinical consultations. Per VHA's Directive 1500, RCS's mission is to provide counseling to help clients readjust to civilian life or to continued military service, and VA's Fiscal Year 2018-2024 Strategic Plan includes a goal to make it easy for veterans to access benefits and services from VHA.48 Furthermore, according to federal standards for internal control for risk assessment and monitoring, management should routinely identify, analyze, and respond to risks to meeting its objectives-such as barriers to obtaining or providing Vet Center care. Management should also establish and operate ongoing monitoring activities to assess the quality of its performance over time and evaluate the results on a periodic basis. Agencies can use these evaluations to determine the effectiveness of its programs or the need for any corrective actions to achieve their objectives-such as helping veterans and servicemembers achieve successful readjustment.42 48Veterans Health Administration, VHA Directive 1500(2), amended Dec. 30, 2021, and Department of Veterans Affairs, FY 2018-2024 Strategic Plan (refreshed May 31, 2019). 49GAO-14-704G. Page 28 GAO-22-105039 VA Vet Centers Conclusions Recommendations for Executive Action Taking steps to develop processes to periodically assess the extent of barriers to Vet Center care would enable RCS to better understand the magnitude of the barriers it has identified, the significance of any risks they may pose to RCS's and VA's objectives, and the extent they may remain following any steps it, its districts, and Vet Centers have taken to address them. Such processes may include or be informed by the early efforts RCS has underway or by other efforts RCS may develop to understand the barriers to Vet Center care. Such processes would help RCS ensure it is responding appropriately to any risks posed by the barriers, such as by helping RCS determine whether any additional steps are needed to ensure that eligible veterans, servicemembers, and their families are able to obtain the services they need and to ensure that Vet Center counselors are able to provide such services in an appropriate and timely manner. Vet Centers play a pivotal role in helping veterans and servicemembers readjust to civilian life or to continued military service. Although many of them readjust without major difficulties, others experience challenges, such as mental illness and substance abuse, which increase their risk of suicide. Vet Centers provide services ranging from counseling for post- traumatic stress disorder and military sexual trauma to outreach activities aimed at reaching eligible individuals in need of this assistance. However, VHA lacks some processes that would help it better assess Vet Center activities, including the extent to which Vet Centers are providing services that meet the needs of their clients collectively and whether barriers to Vet Center care are minimized. Further, Vet Centers lack data that would help them better tailor their outreach activities to those in need of services and guidance for assessing whether their outreach is effective. Developing such processes and providing Vet Centers with these data and guidance would help VHA better ensure that veterans and servicemembers are receiving the help they need meeting the serious challenges many face readjusting to civilian life or to continued military service. We are making the following five recommendations to VHA: The RCS Chief Officer should develop and implement a process to periodically assess the extent to which Vet Centers are meeting the needs of their clients collectively, including subpopulations of clients that may experience different challenges readjusting to civilian life or to continued military service. Such client subpopulations could include those who experienced trauma during military service and those who served in different conflicts. (Recommendation 1) Page 29 GAO-22-105039 VA Vet Centers Agency Comments and Our Evaluation The RCS Chief Officer should ensure that Vet Centers have data on eligible individuals in their service area that they can use to tailor their outreach activities. These data could include information on veterans who have recently transitioned back from military service and veterans' demographic characteristics (e.g., age, gender, race, and ethnicity). (Recommendation 2) The RCS Chief Officer should provide Vet Centers with guidance for assessing the effectiveness of their outreach activities. This guidance should include metrics for the outreach activities and targets against which to assess those metrics to determine effectiveness. (Recommendation 3) The RCS Chief Officer should develop and implement a process to periodically assess the extent of identified barriers eligible veterans and servicemembers may experience to obtaining services, including a lack of awareness about Vet Centers and challenges accessing Vet Center services. (Recommendation 4) The RCS Chief Officer should develop and implement a process to periodically assess the extent of identified barriers Vet Center staff may encounter to providing services, including challenges obtaining clinical consultations for complex cases. (Recommendation 5) We provided VA with a draft of this report for review and comment. VA provided written comments, which are reprinted in appendix |. In its written comments, VA concurred with all five of the report's recommendations and identified actions VHA's RCS is taking to implement them. In response to our first recommendation to develop and implement a process to periodically assess the extent to which Vet Centers are meeting the needs of their clients collectively, including subpopulations of clients, VA stated that RCS has fully implemented a real-time customer feedback survey and requested that we close the recommendation as implemented. We acknowledge in our report VA's efforts with this survey. However, neither during our review nor as part of its comments on this report has VA indicated how it will use the survey responses to periodically assess the extent to which Vet Centers are meeting the needs of their clients collectively. We plan to follow up with VA as part of our process to close agency recommendations. Page 30 GAO-22-105039 VA Vet Centers In addition, in its general comments, VA stated that RCS is not in the "early stages" of implementing efforts regarding measuring customer experience. VA noted that RCS is actively referring to data from its customer feedback survey to evaluate how client needs are being met. VA also noted that it will take time to receive a sufficient number of responses for analyzing demographic subpopulations. We revised our report statement to clarify this. VA also provided technical comments, which we incorporated as appropriate. We are sending copies of this report to the appropriate congressional committees, the Secretary of Veterans Affairs, and other interested parties. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-7114 or SilasS@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff wno made key contributions to this report are listed in appendix II. Si Sharon M. Silas Director, Health Care Page 31 GAO-22-105039 VA Vet Centers Appendix |: Comments from the Department of Veterans Affairs DEPARTMENT OF VETERANS AFFAIRS WASHINGTON April 28, 2022 Ms. Sharon M. Silas Director Health Care U.S. Government Accountability Office 441 G Street, NW Washington, DC 20548 Dear Ms. Silas: The Department of Veterans Affairs (VA) reviewed the Government Accountability Office (GAO) draft report: VA VET CENTERS: Opportunities Exist to Help Better Ensure Veterans' and Servicemembers' Readjustment Counseling Needs Are Met (GAO-22-105039). The enclosure contains technical and general comments and the action plan to address the draft report recommendations. VA appreciates the opportunity to comment on your draft report. Sincerely, Vr S Dedeln Tanya J. Bradsher Chief of Staff Enclosure Page 32 GAO-22-105039 VA Vet Centers Appendix |: Comments from the Department of Veterans Affairs Enclosure Department of Veterans Affairs (VA) Response to the Government Accountability Office (GAO) Draft Report VA VET CENTERS: Opportunities Exist to Help Better Ensure Veterans' and Servicemembers' Readjustment Counseling Needs Are Met (GAO-22-105039) Recommendation 1: The RCS Chief Officer should develop and implement a process to periodically assess the extent to which Vet Centers are meeting the needs of their clients collectively, including subpopulations of clients that may experience different challenges readjusting to civilian life or to continued military service. Such client subpopulations could include those who experienced trauma during military service and those who served in different conflicts. VA Response: Concur. Readjustment Counseling Service (RCS) agrees that a process to assess how Vet Centers are meeting client needs is an important part of ensuring RCS is providing the highest levels of services to eligible individuals. RCS began work on Customer Service Modernization efforts in January 2020 in partnership with VA's Veterans Experience Office (VEO). RCS has fully implemented this process using RCS Veterans Signals (VSignals), a real-time customer feedback survey. Please reference the RCS Customer Experience Modernization Timeline embedded below. a) RCS Customer Experience Moderni As discussed in interviews with GAO auditors, RCS implemented the VSignals survey process in July 2021. All questions and data received through the VSignals Platform can be utilized to analyze how Vet Center services are meeting the needs of the individuals served. RCS VSignals Dashboard is a powerful tool which allows for all data points to be filtered for specific analysis of multiple demographic categories including Rurality, Service Focus Areas (i.e., Post-Traumatic Stress Disorder, Military Sexual Trauma, Legal, Marital/Family, Bereavement, Drug/Alcohol, etc.), Eligibility, Branch of Service, Service Component, Race, Gender and Age. The Dashboard also allows data points to be filtered by organizational level including Overall RCS, District, Zone and Vet Center; Counseling Location (Vet Center, Community Access Point, Outstation, Mobile Vet Center or Out of Center); Counseling Type (Couples, Family, Group or Individual); and Modality (In Person, Phone, or Telehealth). VSignals surveys capture customer feedback across the spectrum of RCS services by measuring customer experience at five separate and specific stages of engagement, including: (1) Initial Engagement, (2) Intake, (3) Ongoing Services, (4) Separation from Services and (5) Re-engagement Following Separation. These specific survey points allow RCS to assess the entire readjustment counseling process and related customer experience. Page 1 of 9 Page 33 GAO-22-105039 VA Vet Centers Appendix |: Comments from the Department of Veterans Affairs Enclosure Department of Veterans Affairs (VA) Response to the Government Accountability Office (GAO) Draft Report VA VET CENTERS: Opportunities Exist to Help Better Ensure Veterans' and Servicemembers' Readjustment Counseling Needs Are Met (GAO-22-105039) Please see embedded screen shots below for each of the RCS VSignals Surveys: 1) Initial Engagement (2 documents), 2) Client Intake (2 documents), 3) Client Counselor Relationship (1 document), 4) Unengaged Clients (2 documents), 5) Reengaged Clients (2 documents) and 6) Demographic Questions (1 document). Vet Center Survey1 Vet CenterSurvey1 VetCenterSurvey2 VetCenterSurvey2 Vet Center Survey 3 - Initial Engagemen_ - Initial Engagemeni_ -Client Intake NotA -Client Intake Not A - Ongoing Client Re Vet Center Survey4 Vet CenterSurvey4 VetCenterSurvey5 Vet Center Survey 5 Vet Center Survey - Unegaged Suspen -Unegaged Suspenc -Reengaged Clients -Reengaged Clients Demographics.pdf As of March 31, 2022, 152,456 VSignals surveys have been administered with 24,324 responses received for a 16% response rate. The survey system collected 12,645 comments made during this period, of which 79.8% were compliments, 7.8% were concerns and 12.4% were recommendations. For this same period, RCS has an aggregate trust score of 92.5%. RCS has reached a survey response threshold that allows for macro level data analysis and has integrated quarterly reporting of this data to the RCS Governance Board for oversight. The first iteration of this reporting was conducted during the February 2022 RCS Governance Board meeting (meeting minutes embedded below). As a result of quarterly reporting, the RCS Governance Board will direct action to appropriate RCS Governance Councils, Districts or Program Offices to conduct further analysis for areas identified as opportunities or concerns. Furthermore, as survey response numbers continue to grow, validity of data in the respective subsets for demographic and geographic data discussed above will allow for more extensive analysis. RCS has completed work on this recommendation and requests GAO close it as implemented. February 2022 Governance Board h Completion Date: March 2022 Page 2 of 9 Page 34 GAO-22-105039 VA Vet Centers Appendix |: Comments from the Department of Veterans Affairs Enclosure Department of Veterans Affairs (VA) Response to the Government Accountability Office (GAO) Draft Report VA VET CENTERS: Opportunities Exist to Help Better Ensure Veterans' and Servicemembers' Readjustment Counseling Needs Are Met (GAO-22-105039) Recommendation 2: The RCS Chief Officer should ensure that Vet Centers have data on eligible individuals in their service area that they can use to tailor their outreach activities. These data could include information on veterans who have recently transitioned back from military service and veterans' demographic characteristics (e.g., age, gender, race, and ethnicity). VA Response: Concur. RCS agrees that a process to identify eligible individuals in respective Vet Center service areas, including information related to demographic characteristics would enhance outreach efforts. RCS is in the process of elaborating on the requirements for the Department of Defense (DoD) Military Service information it needs to be able to identify those Veterans who have recently transitioned from active military service. RCS' intent is to utilize the Veterans Affairs/Department of Defense Identity Repository (VADIR) as the authoritative data source for this effort. In partnership with VA's Office of Information Technology, additional research is planned to select a constituent management software solution that can effectively distribute the military service data across RCS' National Network, and to consider alternatives for integrating this solution with VADIR. RCS confirmed that the demographic data fields referenced in this recommendation and several others, including information related to the individuals record of military service, marital status and contact information, are available to RCS through VADIR's data set. The RCS team anticipates development, testing, deployment to production and staff training will take up to a year to complete. However, once the target application is in production, data will be distributed by zip code to the associated district, zone and Vet Center for integration into local outreach processes. RCS also obtains information that helps to tailor outreach services through partnerships external to the organization. RCS participates in the DoD Intervention, Prevention and Outreach (DoD-IPO) forum. In fiscal year (FY) 2020, RCS joined the IPO forum to establish a formal, collaborative platform to improve behavioral health, through access to non-medical mental health care and overall well-being for the Reserve Component, geographically dispersed Veterans, Service members and their families. The committee includes numerous government agencies to focus on developing and implementing strategic, operational and tactical level solutions. The IPO also led to new partnerships with additional organizations such as Building Healthy Military Communities (BHMC), a DoD pilot program that assists geographically dispersed Service members and their families with resources that may impact their readiness, resiliency and well-being. BHMC shared a data tool, the Community Readiness-Open Data Dashboard (CR-ODD), to identify Veteran population by county. CR-ODD will help Vet Center leaders and outreach specialists make more informed decisions about the health and well-being of their Veterans, Service member and family populations and communities. Page 3 of 9 Page 35 GAO-22-105039 VA Vet Centers Appendix |: Comments from the Department of Veterans Affairs Enclosure Department of Veterans Affairs (VA) Response to the Government Accountability Office (GAO) Draft Report VA VET CENTERS: Opportunities Exist to Help Better Ensure Veterans' and Servicemembers' Readjustment Counseling Needs Are Met (GAO-22-105039) RCS will develop a communications plan to inform district offices and Vet Center teams of this resource. Additionally, in December 2021, RCS completed work with the Veterans Health Administration's (VHA) Office of Enrollment and Forecasting to expand an existing VHA contract for RCS application in projection of demand for readjustment counseling services. Planning and analysis and using data from this Demand Forecasting model will enable RCS to forecast future demand more accurately; and identify areas of unmet need that can be applied to specific Vet Center service areas. These projections will be used to prioritize outreach efforts to areas indicating a high demand for service and where geographical gaps in RCS services have been identified. RCS has already integrated the use of this data into service planning to support allocation of resources into areas where need was identified. Target Completion Date: May 2023 Recommendation 3: The RCS Chief Officer should provide Vet Centers with guidance for assessing the effectiveness of their outreach activities. This guidance should include metrics for the outreach activities and targets against which to assess those metrics to determine effectiveness. VA Response: Concur. RCS agrees that it is important to understand the effectiveness of outreach efforts and that establishing metrics to assess this effectiveness is beneficial to developing the best practices for reaching eligible Veterans, Service members and their families and expanding access to Vet Center services. RCS partnered with a group of consultants to work with RCS leadership and staff, which comprise a working unit of the organization referred to as the RCS Analytic Unit. Based on a review of capabilities and current RCS assets, the Analytic Unit began planning work in two main areas at the start of FY 2020. The two areas selected by RCS leadership for the Analytic Unit were data dashboards and counseling quality. Counseling quality is inclusive of services that are both counseling and non-counseling (e.g., outreach and case management) services delivered at a Vet Center. A key component of this work is the focus on defining and operationalizing counseling quality through the RCS Counseling Quality Workgroup with a target result of measuring Outreach Yield. At the December 2021 meeting, the RCS Counseling Quality Workgroup presented the RCS Counseling Quality Report 2021 to the RCS Governance System. The report outlined the group's efforts and plans for deliverables and was approved unanimously by the voting membership. Page 4 of 9 Page 36 GAO-22-105039 VA Vet Centers Appendix |: Comments from the Department of Veterans Affairs Enclosure Department of Veterans Affairs (VA) Response to the Government Accountability Office (GAO) Draft Report VA VET CENTERS: Opportunities Exist to Help Better Ensure Veterans' and Servicemembers' Readjustment Counseling Needs Are Met (GAO-22-105039) The RCS Governance Board has charged the RCS Stakeholder Relations Council to define effective outreach for RCS. The council will assess current RCS practices for outreach and will also consider practices from external organizations that have demonstrated success in this process. Additionally, the council will work with the RCS Counseling Quality Workgroup to provide feedback related to this process for inclusion in the development of metrics and refinement of data collection processes to measure outreach effectiveness. Recommendations from the council and workgroup will be brought back to the RCS Governance System for approval and direction for implementation. Once this process is finalized and approved by the RCS Governance System, the data related to outreach effectiveness will be incorporated into the RCS Data Dashboard to allow Vet Centers to assess and monitor outreach activities. Additionally, RCS will direct outreach specialists to improve preparation efforts before an outreach event, improve documentation of outreach efforts and improve reporting of post event results. RCS will add additional criteria to outreach event documentation. With improved data, Vet Center teams will be able to make more informed decisions on the effectiveness of their events. Target Completion Date: May 2023 Recommendation 4: The RCS Chief Officer should develop and implement a process to periodically assess the extent of identified barriers eligible veterans and servicemembers may experience to obtaining care, including a lack of awareness about Vet Centers and challenges accessing Vet Center services. VA Response: Concur. RCS agrees that it is important to understand barriers eligible individuals experience when accessing Vet Center services and to assess awareness of these services. Implementing VSignals real-time customer feedback surveys allows RCS to assess ease of access and knowledge of services through multiple questions, including screener questions, that collect information regarding potential barriers. This information is obtained through the following survey questions: e The Vet Center offers remote and/or in-person services that are accessible to me. e | was aware of the intake process to become a Vet Center client. e The Vet Center programs | am interested in are offered at times when | can attend. e The Vet Center scheduled my appointment within a reasonable amount of time. e Screener questions related to: o Hownew clients learned about Vet Center Services; o Why new clients chose the Vet Center for counseling and other services; Page 5 of 9 Page 37 GAO-22-105039 VA Vet Centers Appendix |: Comments from the Department of Veterans Affairs Enclosure Department of Veterans Affairs (VA) Response to the Government Accountability Office (GAO) Draft Report VA VET CENTERS: Opportunities Exist to Help Better Ensure Veterans' and Servicemembers' Readjustment Counseling Needs Are Met (GAO-22-105039) o When clients realized they could benefit from engaging in Vet Center services; and o Why clients stopped receiving services from the Vet Center. In addition to these structured questions, VSignals offers opportunities for clients to enter open text comments related to concerns they may have with Vet Center services. Operational guidance was provided to all RCS District Offices in the "RCS Veterans Signals Communication and Service Recovery Plan" (embedded document below) for conducting data analysis and service recovery including review of the open text comments. RCS VSignals SR Comms Plan 070221 Currently, there is not a resource to gauge awareness of Vet Center services and barriers to access for eligible individuals who are not already engaged with Vet Center services. RCS will explore partnerships with other VA offices and external stakeholders to develop and implement a process that can assess the level of awareness that eligible individuals have regarding Vet Centers and any barriers Veterans and Service members may have for seeking care. Target Completion Date: September 2023 Recommendation 5: The RCS Chief Officer should develop and implement a process to periodically assess the extent of identified barriers Vet Center staff may encounter to providing services, including challenges obtaining clinical consultations for complex cases. VA Response: Concur. RCS agrees that a method of collecting feedback from staff directly related to barriers in providing services is an important component to evaluate the overall effectiveness of Vet Center services. RCS currently attains some of this information from existing survey processes including training needs assessments, VSignals real-time customer feedback data and the annual All Employee Survey. Additionally, per VHA Directive 1500(2), Readjustment Counseling Service, paragraph 16.b., all Vet Centers are subject to a clinical oversight visit, either onsite or remote, within the fiscal year which includes assessment of the provision for external clinical consultation. Site visits are conducted by the district who routinely meet with assigned Vet Center external clinical consultants at the VA medical facility to discuss any relevant quality oversight issues under review. The Vet Center Director's supervision, as Page 6 of 9 Page 38 GAO-22-105039 VA Vet Centers Appendix |: Comments from the Department of Veterans Affairs Enclosure Department of Veterans Affairs (VA) Response to the Government Accountability Office (GAO) Draft Report VA VET CENTERS: Opportunities Exist to Help Better Ensure Veterans' and Servicemembers' Readjustment Counseling Needs Are Met (GAO-22-105039) required by paragraph 16.a. of VHA Directive 1500(2), also detects cases where consultation with a VA mental health professional was a challenge. RCS plans to evaluate the potential for including more specific questions related to barriers staff face in providing services while using existing survey processes. RCS also has plans to explore options, if needed, to create a specific feedback mechanism to obtain this information if it is not able to incorporate the process into current methodology. Target Completion Date: December 2022 Page 7 of 9 Page 39 GAO-22-105039 VA Vet Centers Appendix |: Comments from the Department of Veterans Affairs Enclosure Department of Veterans Affairs (VA) Response to the Government Accountability Office (GAO) Draft Report VA VET CENTERS: Opportunities Exist to Help Better Ensure Veterans' and Servicemembers' Readjustment Counseling Needs Are Met (GAO-22-105039) General Comments: Page 15, first paragraph, lines 7-11 "RCS officials also told us they plan to analyze responses to their new client feedback survey to determine the extent to which Vet Center are meeting the needs of their clients collectively, including subpopulations. However, according to RCS officials, these efforts are in the early stages." VA Comment: RCS disagrees with the statement that it is in the "early stages" of implementing efforts regarding measuring customer experience. RCS requests the deletion of the second sentence: "However, according to RCS officials, these efforts are in the early stages." RCS initiated work on Customer Service Modernization efforts in January 2020 in partnership with VA's Veterans Experience Office (VEO). RCS has fully implemented this process using RCS Veterans Signals (VSignals), real-time customer feedback survey, as of July 2021. RCS is actively referring to data from this process to evaluate how client needs are being met and to engage in proactive service recovery when challenges or concerns are identified. While the process is in place to evaluate client experience, it will take time to receive sufficient data at subpopulation demographic levels to have data validity to identify potential trends. As of March 31, 2022, 152,456 VSignals surveys have been administered with 24,324 responses received for a 16% response rate. The survey system collected 12,645 comments made during this period, of which 79.8% were compliments, 7.8% were concerns and 12.4% were recommendations. For this same period, RCS has an aggregate trust score of 92.5%. Review of data to this point shows high agreement scores from respondents and indicates high levels of client satisfaction with Vet Center services. RCS feels it is important to emphasize that they are not lacking a process to evaluate client experience and how needs are being met, but rather have a robust system that was fully developed through the Human Centered Design process utilizing expertise from our partners in VEO. Please reference the RCS Customer Experience Modernization Timeline embedded below that outlines this development process. Pa RCS Customer Experience Moderni Page 8 of 9 Page 40 GAO-22-105039 VA Vet Centers Appendix II: GAO Contact and Staff GAO Contact Sharon M. Silas, (202) 512-7114 or SilasS@gao.gov Staff In addition to the contact named above, Janina Austin (Assistant Director), Matthew Nattinger (Analyst-in-Charge), Kye Briesath, Hannah Acknowledgments Grow, and Laurie Pachter made key contributions to this report. Also contributing were Jacquelyn Hamilton and Cynthia Khan. (105039) Page 41 GAO-22-105039 VA Vet Centers GAO's Mission Obtaining Copies of GAO Reports and Testimony The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no cost is through our website. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. You can also subscribe to GAO's email updates to receive notification of newly posted products. Order by Phone Connect with GAO To Report Fraud, Waste, and Abuse in Federal Programs Congressional Relations Public Affairs Strategic Planning and External Liaison The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's website, https:/Avww.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO on Facebook, Flickr, Twitter, and YouTube. Subscribe to our RSS Feeds or Email Updates. Listen to our Podcasts. Visit GAO on the web at https:/Awww.gao.gov. Contact FraudNet: Website: https:/Avww.gao.gov/about/what-gao-does/fraudnet Automated answering system: (800) 424-5454 or (202) 512-7700 A. Nicole Clowers, Managing Director, ClowersA@gao.gov, (202) 512-4400, U.S. Government Accountability Office, 441 G Street NW, Room 7125, Washington, DC 20548 Chuck Young, Managing Director, youngce1 @gao.gov, (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548 Stephen J. Sanford, Managing Director, spel@gao.gov, (202) 512-4707 U.S. Government Accountability Office, 441 G Street NW, Room 7814, Washington, DC 20548 oy ue Please Print on Recycled Paper.