United States Government Accountability Office Report to Congressional Committees VETERANS HEALTH January 2023 CARE VA Actions Needed to Ensure Timely Scheduling of Specialty Care Appointments GAO-23-105617 January 2023 VETERANS HEALTH CARE VA Actions Needed to Ensure Timely Scheduling of Specialty Care Appointments Highlights of GAO-23-105617, a report to congressional committees Why GAO Did This Study What GAO Found VHA operates the largest health care The Veterans Health Administration (VHA) updated its process for scheduling delivery system in the United States, specialty care appointments in 2020. This included both appointments at VHA providing health care to 6.4 million facilities and with providers in the community. Under VHA's updated process, veterans in fiscal year 2021. In the last referral coordination teams at each VA medical center review referrals for decade, Congress has taken steps to specialty care, such as mental health treatment or cardiology, and discuss care expand the ability for eligible veterans options with veterans. to receive care from community providers, such as when they cannot According to the timeliness standards that VHA has established for specialty care access care in a timely manner from appointment scheduling, appointments at VHA facilities must be scheduled within VHA directly. GAO and others have 3 business days from the date a VHA provider enters a referral and community previously identified challenges VHA has had in scheduling appointments in care appointments must be scheduled within 7 days. VHA data from the third a timely manner. The Johnny Isakson quarter of fiscal year 2022 indicate that VA medical centers performed better and David P. Roe, M.D. Veterans scheduling timely referrals for VHA facility appointments than community care. Health Care and Benefits Improvement VHA officials said they are considering reexamining the community care Act of 2020 included requirements for standard. VHA's limited analysis on scheduling timeliness that VHA officials VHA related to appointment scheduling conducted in 2021 lacked information for the current standard. Conducting a and related oversight and training. It comprehensive analysis of scheduling timeliness data from all VA medical also includes a provision for GAO to centers could help VHA determine whether the community care standard is review VHA's efforts. achievable or adjustments to the standard or scheduling process are needed. This report, among other objectives, describes VHA's updated specialty Percentage of Referrals Meeting Timeliness Standards by Veterans Affairs Medical care scheduling process; examines Center, Third Quarter Fiscal Year 2022 VHA's scheduling timeliness standards; and examines VHA's efforts to provide training on the updated scheduling process. GAO reviewed VHA documentation on the scheduling process and scheduling timeliness data for the third quarter of fiscal year 2022, the most recent data available. GAO also interviewed VHA officials and officials at six VA medical centers that were randomly selected based on facility complexity and geography. What GAO Recommends GAO is making three recommendations, including that VHA conduct a comprehensive analysis of scheduling timeliness data from all VA medical centers and require training. VA concurred with the recommendations and identified steps it would take to implement them. In May 2022, VHA developed training materials on the updated scheduling process for both the providers who enter referrals and those who review them for View GAO-23-105617. For more information, the referral coordination team. However, VHA does not require this training. contact Sharon M. Silas at (202) 512-7114 or Doing so would help ensure that providers have the information needed to SilasS@gao.gov. perform their duties and help ensure veterans' timely access to care. United States Government Accountability Office Contents Letter 1 Background 5 VHA's Updated Appointment Scheduling Process Relies on Designated Teams to Review Specialty Care Referrals 9 VHA Has Scheduling Timeliness Standards, but Not for When Appointments Should Occur; Most VAMCs Did Not Meet VHA's Community Care Standard 14 VHA Has Implemented New Tools to Monitor Scheduling Timeliness 20 VHA Provides Training to Schedulers, but Does Not Require Providers to Take Training on VHA's Updated Scheduling Process 22 Conclusions 26 Recommendations for Executive Action 27 Agency Comments 27 Appendix I Objectives, Scope, and Methodology 29 Appendix II Appointment Scheduling Scenarios Other Than for Specialty Care Referrals 35 Appendix III Veterans Health Administration (VHA) Data on VHA Facility and Community Care Scheduling Timeliness 37 Appendix IV Comments from the Department of Veterans Affairs 39 Appendix V GAO Contact and Staff Acknowledgments 41 Related GAO Products 42 Page i GAO-23-105617 Veterans Health Care Tables Table 1: Veterans Health Administration (VHA) Referral Coordination Initiative Models as of July 2022 13 Table 2: Veterans Health Administration (VHA) Field Support Engagement Levels 21 Table 3: Veterans Integrated Service Networks (VISN) Grouped by U.S. Census Regions 34 Table 4: Department of Veterans Affairs Medical Centers (VAMC) Randomly Selected from Site Selection Groups 34 Table 5: Percentage of VHA Facility and Community Care Referrals That Met Appointment Scheduling Timeliness Standards and Average Number of Days to Schedule, Third Quarter Fiscal Year 2022, by VISN 37 Figures Figure 1: VHA Specialty Care Appointment Scheduling Process 10 Figure 2: Timeliness Standards for Scheduling VHA Facility and Community Care Referrals, as of September 2022 15 Figure 3: Percent of Referrals that Met the 3-business day VHA Facility and 7-day Community Care Timeliness Standards in the Third Quarter of Fiscal Year 2022, by VAMC 18 Abbreviations Isakson-Roe Act Johnny Isakson and David P. Roe, M.D. Veterans Health Care and Benefits Improvement Act of 2020 RCT Referral Coordination Team VA Department of Veterans Affairs VAMC Department of Veterans Affairs medical center VHA Veterans Health Administration VISN Veterans Integrated Service Network This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Page ii GAO-23-105617 Veterans Health Care Letter 441 G St. N.W. Washington, DC 20548 January 4, 2023 Chair Ranking Member Committee on Veterans' Affairs United States Senate Chair Ranking Member Committee on Veterans' Affairs House of Representatives The Veterans Health Administration (VHA) within the Department of Veterans Affairs (VA) operates the largest health care delivery system in the United States, providing health care to approximately 6.4 million veterans in fiscal year 2021. 1 In the last decade, Congress has taken steps to expand the ability for eligible veterans to receive care from community providers when they face challenges accessing care at VHA medical facilities; these steps include establishing the Veterans Community Care Program in 2019. 2 While most veterans still receive the majority of their care at VHA facilities, including 170 VA medical centers (VAMC) and over 1,000 outpatient facilities, approximately 2 million 1This includes veterans treated by VHA or whose treatment is paid for by VHA. 2In August 2014, after several well-publicized events highlighted serious and long- standing issues with veterans' access to care, Congress enacted the Veterans Access, Choice, and Accountability Act of 2014. Among other things, the law established a temporary program-called the Veterans Choice Program-and provided up to $10 billion in funding for veterans to obtain health care services from community providers when they faced long wait times, lengthy travel distances, or other challenges accessing care at VHA facilities. Pub. L. No. 113-146, tit. I, §§ 101, 802(d), 128 Stat. 1754, 1755-1765, 1802- 1803 (2014). In 2019, the Veterans Community Care Program replaced the Veterans Choice Program and consolidated other existing community care programs. The VA MISSION Act of 2018 (VA MISSION Act) broadened veterans' eligibility to receive care outside of the VA health care system under this program. Pub. L. No. 115-182, tit. I, §101, 132 Stat. 1393, 1395 (2018). Page 1 GAO-23-105617 Veterans Health Care veterans received care from non-VHA providers in the community in fiscal year 2021, according to VA. 3 In recent years, we and others have identified challenges that VHA has had ensuring that both VHA facility and community care appointments are scheduled in a timely manner and overseeing its scheduling process. 4 For example, in 2018, we found that VHA's appointment scheduling process for care from community providers was structured in a way that could make it difficult for VHA to meet the statutorily required time frames for veterans to receive care. This required time frame specified the number of days it should take for a veteran to receive care under the Veterans Choice Program-the precursor to the current community care program. 5 We recommended that VHA establish an achievable wait-time goal for the new community care program to monitor whether wait times for veterans to receive care in the community are comparable with those at VHA 3For the purposes of this report, we use "VHA facility" to refer to care provided by VHA providers at VAMCs and other VHA facilities, and we refer to care provided by community providers through the Veterans Community Care Program as "community care". 4See the full list of related GAO products at the end of this report. The VA Office of Inspector General has also reported on VHA's appointment scheduling challenges. For example, see VA Office of the Inspector General, Veterans Health Administration, Audit of Community Care Consults During COVID-19, Report No. 21-00497-46 (Washington, D.C.: Jan. 19, 2022) and Veterans Health Administration, Improvements Are Needed in the Community Care Consult Process at VISN 8 Facilities, Report No. 18-05121-36 (Washington, D.C.: Jan. 16, 2020). 5Under the Veterans Choice Program, the required timeframe provided that eligible veterans receive Veterans Choice Program care no more than 30 days from the date an appointment was deemed clinically appropriate by a VHA clinician (referred to as the clinically indicated date), or if no such determination had been made, 30 days from the date the veteran preferred to receive care. See GAO, Veterans Choice Program: Improvements Needed to Address Access-Related Challenges as VA Plans Consolidation of its Community Care Programs, GAO-18-281 (Washington, D.C.: June 4, 2018). Page 2 GAO-23-105617 Veterans Health Care facilities. 6 Due to this concern with wait times and other issues, VHA health care continues to be on our High Risk List. 7 One reason veterans may be eligible for community care is that services they need are not available at a VHA facility or in a timely manner, including appointments for specialty care. 8 In 2020, VHA updated its process for scheduling specialty care appointments at VHA facilities and in the community. 9 To further improve VHA's process to schedule veterans' appointments for care at VAMCs and in the community, the Johnny Isakson and David P. Roe, M.D. Veterans Health Care and Benefits Improvement Act of 2020 (Isakson-Roe Act) was enacted on January 5, 2021. 10 The Act included requirements for VA related to its appointment scheduling process for VHA facilities and community care and oversight of and training on that process. It also included a provision for us to review VHA's efforts to address those requirements. 11 This report 1. describes VHA's updated appointment scheduling process for specialty care provided at VHA facilities and through community providers; 2. examines VHA's appointment scheduling timeliness standards for specialty care; 6In2020, we also suggested that Congress take action to address this issue by requiring VA to establish an overall wait-time performance measure for community care. See GAO, Veterans Community Care Program: Improvements Needed to Help Ensure Timely Access to Care, GAO-20-643 (Washington, D.C.: Sept. 28, 2020). 7GAO maintains a high-risk program to focus attention on government operations that it identifies as high risk due to their greater vulnerabilities to fraud, waste, abuse, and mismanagement or the need for transformation to address economy, efficiency, or effectiveness challenges. We added VA health care to the High Risk List in 2015. See GAO, High-Risk Series: Dedicated Leadership Needed to Address Limited Progress in Most High-Risk Areas, GAO-21-119SP (Washington, D.C.: Mar. 2, 2021). 8While primary care addresses patients' routine health needs, specialty care involves specific services such as cardiology or physical therapy, among others. 9Specifically,VHA updated its process for scheduling primary care, mental health, and specialty care referrals. A referral is a request for clinical services on behalf of a veteran that is entered into VHA's electronic health record system by a VHA provider. According to VHA officials, referrals for primary care account for only a small portion of referrals. For the purposes of this report, we refer to the updated scheduling process as the specialty care appointment scheduling process, which also encompasses mental health. 10Pub. L. No. 116-315, 134 Stat. 4932 (2021). 11Pub. L. No. 116-315, tit. III, § 3101(d), 134 Stat. at 4999-5001 (2021). Page 3 GAO-23-105617 Veterans Health Care 3. examines how VHA oversees VAMCs to help ensure adherence to these scheduling timeliness standards; and 4. examines VHA's efforts to provide training on the updated scheduling process. For all four objectives, we reviewed relevant VHA documentation detailing requirements, roles, and responsibilities for the VHA facility and community care appointment scheduling process, and VHA's oversight of and training on the process. For the purposes of this report, we focused on the appointment scheduling process for VHA facility and community care specialty care referrals because the scheduling process has been updated recently, while the scheduling process for other types of care, such as follow-up appointments, has remained largely unchanged. We also analyzed VHA data on each VAMC's performance on established appointment scheduling timeliness standards for the third quarter of fiscal year 2022, which were the most recent data available at the time of our analysis. We interviewed officials from VHA's Office of Integrated Veteran Care-responsible for establishing scheduling policy and monitoring veterans' access to care at VHA and in the community-about these data and reviewed documentation that detailed how the data are measured and collected. We determined the data were sufficiently reliable for the purpose of providing information on VAMCs' performance on these standards. Further, we conducted virtual site visits from April through June 2022 with six VAMCs that were selected randomly on the basis of VAMC complexity level and geographic location. 12 We interviewed staff and leadership about their experiences with the appointment scheduling process, oversight of the process, and training. The information we obtained from the virtual site visits is not generalizable to other VAMCs. The six VAMCs we selected were in Montgomery, Alabama; Anchorage, Alaska; Mather (Sacramento), California; Hines, Illinois; East Orange, New Jersey; and San Juan, Puerto Rico. See appendix I for additional details on our objectives, scope, and methodology, including the VAMC site selection process. 12The complexity level VHA assigns to a VAMC is determined on the basis of the characteristics of the patient population, clinical services offered, educational and research missions, and administrative complexity of the VAMC. VHA assigns each VAMC one of five complexity levels (ranked from most to least complex): 1a, 1b, 1c, 2, and 3. Page 4 GAO-23-105617 Veterans Health Care We compared VHA's efforts related to specialty care scheduling timeliness standards for VHA facility and community care appointments against Office of Management and Budget guidance and key practices we have identified in our past work. 13 We also evaluated VHA's efforts to provide training on the scheduling process in the context of leading practices for training implementation, which includes the appropriate identification of employees to receive training. 14 We conducted this performance audit from December 2021 to December 2022 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background VHA Appointment There are three primary ways to initiate a request for a veteran's health Scheduling care, once enrolled in VHA: (1) a veteran-initiated appointment request; (2) a provider request for a follow-up appointment; and (3) a provider referral to specialty care. 15 To receive specialty care, a VHA provider must initiate a request on behalf of the veteran by submitting a referral, which VHA calls a consult. 16 Clinical staff, including providers and nurses, and schedulers at the VAMC review the referral. Then depending on whether the veteran is eligible for community care and that veteran's 13See Office of Management and Budget, Preparation, Submission and Execution of the Budget, Circular No. A-11, part 6, § 240.3 (Revised August 2022); GAO, Managing for Results: Practices for Effective Agency Strategic Reviews, GAO-15-602 (Washington, D.C.: July 29, 2015); and GAO, Agency Performance Plans: Examples of Practices That Can Improve Usefulness to Decisionmakers, GAO/GGD/AIMD-99-69 (Washington, D.C.: Feb. 26, 1999). 14GAO, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, GAO-04-546G (Washington, D.C.: Mar. 1, 2004). 15For the purposes of this report, we are focusing the scheduling process for specialty care referrals. 16VHA policy uses the terms 'consult' and 'referral' when describing requests placed by VHA providers. For the purposes of this report, we will use the term referral. For a limited number of outpatient specialty services, veterans can schedule an initial or follow-up appointment at VHA facilities without a referral from a provider. Veterans can utilize this option for audiology, optometry, and podiatry, among other services. Page 5 GAO-23-105617 Veterans Health Care scheduling preferences, VAMC staff will schedule an appointment either with a VHA provider at a VHA facility or with a non-VHA provider in the community. 17 Community Care Through the Veterans Community Care Program, which VHA implemented on June 6, 2019 in response to the VA MISSION Act, veterans may choose to obtain health care services from community providers rather than from VHA providers if they qualify. 18 There are six conditions that can qualify a veteran to receive care under the Veterans Community Care Program. 19 For example, veterans may be eligible for community care when the required services are not available at any VHA facility or if VHA cannot provide care within its designated access standards. VHA's designated access standards specify that a veteran may be eligible for community care if their average drive time to a VHA provider is more than 30 minutes for primary care or more than 60 minutes for specialty care; or the next available appointment with a VHA provider is not available within 20 days for primary care or 28 days for specialty care based on the date of the request for care unless a later date has been agreed upon. 20 Requirements for VHA, its regional networks (Veterans Integrated Service Networks Appointment Scheduling (VISN)), and VAMCs all play a role in VHA facility and community care appointment scheduling to help ensure VHA is managing referrals and scheduling appointments in a timely manner. 21 VHA's scheduling policy establishes the procedures for scheduling medical appointments with VHA providers at VHA facilities and in the community with community 17In addition to VAMC scheduling on behalf of veterans, as of September 2022, 24 VAMCs were receiving contractor support in scheduling community care appointments. In some cases, veterans may also directly schedule appointments with providers. 18Pub. L. No. 115-182, tit. I, § 101, 132 Stat. at 1395 (2018) (codified as amended at 38 U.S.C. § 1703). 1938 C.F.R. §§ 17.4000 - 17.4040 (2022). 20In addition to the six conditions, veterans must either be enrolled in VA health care or eligible for VA care without needing to enroll, and in most circumstances, veterans must receive approval from VHA prior to obtaining care from a community provider. See Pub. L. No. 115-182, tit. I, § 101, 132 Stat. at 1395-1404 (2018) (codified at 38 U.S.C. § 1703(d), (e), and implementing regulations at 38 C.F.R. §§ 17.4000 - 17.4040). 21VHA's health care system is divided into 18 areas called VISNs. Each VISN is responsible for managing and overseeing the VAMCs and other VHA facilities within a defined geographic area. Page 6 GAO-23-105617 Veterans Health Care providers, and sets the requirements for staff involved in the scheduling process. 22 Until spring 2022, at the national level, VHA's Office of Veterans Access to Care developed policy and provided oversight of appointment scheduling processes and requirements at VHA facilities, and the Office of Community Care had the same responsibilities for appointments in the community. In spring 2022, VHA merged the two offices to create a new consolidated entity called the Office of Integrated Veteran Care. 23 According to VHA, this new office has assumed responsibility for the activities previously performed by the Office of Veterans Access to Care and the Office of Community Care, and VHA expects that this merger will allow them to better coordinate care and to streamline and simplify their access efforts. Each VISN is responsible for overseeing policy implementation and the performance of the facilities within its designated region. VAMC directors are responsible for oversight of the facility scheduling and referrals policy, processes, outcomes and regular monitoring of performance on those outcomes, and allocating sufficient resources to manage scheduling, including referrals, to ensure the timely delivery of care. At the VAMC, both VHA clinics-including primary care, mental health, and specialty care clinics-and community care have scheduling staff and supervisors who are responsible on a day-to-day basis for managing the referrals that are entered by VHA providers and coordinating and scheduling veteran care. Referring providers, who are VHA primary care providers, are responsible for entering referrals for specialty care. 22See for example, Department of Veterans Affairs, Outpatient Scheduling Management, VHA Directive 1230 (Washington, D.C.: June 1, 2022); and Consult Processes and Procedures, VHA Directive 1232(4) (Washington, D.C.: Aug. 24, 2016, amended Dec. 14, 2021). 23VHA began realigning staff in the Office of Veterans Access to Care and the Office of Community Care to create the Office of Integrated Veteran Care in April 2022 and completed the realignment in July 2022. Page 7 GAO-23-105617 Veterans Health Care Isakson-Roe Act The Isakson-Roe Act-enacted on January 5, 2021-included requirements related to the scheduling of veterans' health care appointments that required VHA to take several actions: 24 • Establish a scheduling process and requirements. Within 60 days of enactment, the Secretary of VA was required to establish a process and requirements for scheduling health care appointments with VHA providers and community care providers. The Isakson-Roe Act further specified that this process was to account for VHA's designated access standards under community care and specify the maximum number of days allowed to complete each step of the scheduling process. 25 • Monitor VAMCs' compliance with the scheduling process. Within 180 days of enactment, VHA was to develop or maintain a method or tool that would enable it to monitor and ensure compliance with the established appointment scheduling process and requirements, including requirements related to the maximum number of days to complete each step of the process. • Train staff on the process and requirements. VHA was to ensure that each individual involved in scheduling VHA facility and community care appointments certified their understanding of the scheduling process and requirements within a year of enactment. Staff required to certify their understanding included schedulers, clinical coordinators, and supervisors. The Isakson-Roe Act further specified that any new staff involved in the scheduling of appointments are required to undergo training on the scheduling process and 24In addition to the requirements outlined in section 3101, the Isakson-Roe Act had other provisions related to appointment scheduling. Section 3102 required VA to conduct facility-level audits of the scheduling of appointments and management of referrals for VHA facility and community care appointments, and section 3103 focused on the administration of community care. See Pub. L. No. 116-315, tit. III, §§ 3101- 3103, 134 Stat. 4932, 4999-5005 (2021). 25The VA MISSION Act required VA to establish access standards for the purposes of eligibility determinations to receive care in the community through the Veterans Community Care Program. Pub. L. No. 115-182, tit. I, § 104, 132 Stat. 1393, 1409-1412 (2018). VA issued implementing regulations defining VHA's designated access standards at 38 CFR § 17.4010. VHA's designated access standards include when the veteran's average drive time to a VHA provider is more than 30 minutes for primary care or more than 60 minutes for specialty care, or the next available appointment with a VHA provider is not within 20 days for primary care or 28 days for specialty care of the date of request of care unless a later date has been agreed upon. If these standards cannot be met, then the veteran is eligible to receive care in the community. Page 8 GAO-23-105617 Veterans Health Care requirements and certify that they understand the process and requirements. In 2020, VHA updated its process for scheduling appointments internally VHA's Updated with VHA providers and with community care providers by establishing a Appointment new procedure for reviewing referrals for specialty care, called the Referral Coordination Initiative. 26 According to VHA documentation, the Scheduling Process Referral Coordination Initiative transitioned specialty care referral review Relies on Designated responsibilities for VHA facility and community care appointments from multiple clinical employees to designated referral coordination teams Teams to Review (RCT) at each VAMC. Specialty Care According to VHA documentation, the Referral Coordination Initiative Referrals aims to provide veterans with information on available care options-such as wait times for obtaining care from a VHA provider or from a community care provider, to enable veterans to make informed decisions about their care. The Referral Coordination Initiative also aims to improve scheduling timeliness and reduce administrative burden for referring providers by incorporating steps to ensure that referrals are complete and include all pertinent clinical information before referrals are scheduled within VHA or through community providers. Under the Referral Coordination Initiative, VHA's updated specialty care scheduling process involves a referring provider-a primary care provider-RCT staff, and scheduling staff, according to VHA documentation. The referral goes through several steps during the scheduling process, including the referral being entered, reviewed, and scheduled (see fig. 1). 26The process for scheduling appointments for other types of care, including the process for new patients and follow-up appointments, remains largely unchanged. See Appendix II for the new patient and follow-up appointment processes. Page 9 GAO-23-105617 Veterans Health Care Figure 1: VHA Specialty Care Appointment Scheduling Process a This step includes determining whether the veteran is eligible for community care. If the RCT contacts the veteran to gather missing clinical information, they are to discuss care options with the veteran at that time. • VHA provider referral. When a referring provider determines a veteran needs specialty care, such as an appointment with a cardiologist, the VHA provider enters a referral into the veteran's electronic health record. Page 10 GAO-23-105617 Veterans Health Care • RCT review. The RCT comprises both clinical and administrative staff who review referrals and schedule appointments. 27 First, the RCT completes a clinical review to determine the urgency and appropriateness of the referral and determines potential care options for the veteran, including whether the veteran is eligible for community care. 28 Next, the RCT contacts the veteran to discuss care options and preferred dates and times that the veteran would like to schedule an appointment. If the veteran is eligible for community care, the RCT discusses care options at the VAMC and in the community with the veteran, documents the veteran's choice for care, and either schedules the appointment or forwards it to be scheduled, as appropriate. 29 • Scheduling. Referrals are either scheduled by the RCT, specialty care schedulers at the VAMC, or by community care schedulers at the VAMC. 30 If a veteran is not eligible for community care, or is eligible for community care and chooses to receive care at the VAMC, the appointment is scheduled internally by the RCT or specialty care schedulers. If a veteran is eligible and chooses to get community 27According to VHA's Referral Coordination Initiative Guidebook, the ideal RCT clinical staff member is a registered nurse but RCT clinical staff may consist of providers, physician assistants, social workers, or advanced practice registered nurses or nurse practitioners. RCT administrative staff may consist of advanced medical support assistants, medical support assistants, or other clerical administrative roles such as licensed practical nurses, or health care technicians. See Department of Veterans Affairs VHA Office of Integrated Veteran Care, Referral Coordination Initiative Guidebook (May 2022). 28Some referrals are excluded from the required RCT clinical review. For example, referrals for services not offered at the VAMC can be sent directly to community care schedulers. 29The RCT also discusses potential appointment modalities with veterans, such as in- person or video appointments. 30Some VAMCs use contractors to schedule at least a portion of community care appointments associated with referrals. In addition, certain veterans are able to schedule their own appointments with community providers. Page 11 GAO-23-105617 Veterans Health Care care, the referral is sent to VAMC staff, who are responsible for contacting community providers to schedule appointments. 31 Prior to the updated Referral Coordination Initiative process for reviewing specialty care referrals, if a veteran met the eligibility criteria for community care, the referral could be sent directly to the community care scheduler before discussing care options with the veteran. However, it is now a requirement that the RCT review all referrals and discuss care options with each veteran. During the RCT's discussion with the veteran, the RCT communicates all care options and gives eligible veterans the option to receive care at a VAMC or with a community provider. For example, officials at one VAMC we spoke with told us that the wait time for neurology appointments at the VAMC is 35-50 days, resulting in veterans meeting eligibility criteria for community care due to the wait time exceeding 28 days. However, the wait time for neurology appointments in the community is 130-180 days meaning that veterans could actually receive care more quickly at the VAMC. VHA provided VAMCs with guidance on the different ways they could implement the Referral Coordination Initiative, giving them flexibility. VAMCs have the option of implementing either a centralized, decentralized, or hybrid RCT model, according to VHA officials (see table 1). 31Per the VHA minimum scheduling effort policy, schedulers must make a minimum of two contact attempts with veterans using different contact methods; the first can be via telephone, secure message, or email, and the second attempt can also be by letter. After 14 days from the second point of contact, if the veteran cannot be reached, schedulers can disposition the referral, meaning the scheduler discontinues contact attempts and cancels the referral in the system. Some specialties, such as mental health, require additional contact attempts. See Department of Veterans Affairs, Minimum Scheduling Effort for Outpatient Appointments, Standard Operating Procedure (January 2022). Page 12 GAO-23-105617 Veterans Health Care Table 1: Veterans Health Administration (VHA) Referral Coordination Initiative Models as of July 2022 Referral Coordination Initiative model Description Number of VAMCs Centralized One referral coordination team (RCT) at the Department of Veterans Affairs 30 medical center (VAMC) composed of clinical and administrative staff who support multiple specialty care departments, such as cardiology, dermatology, and neurology. Under this model, clinical RCT staff members perform clinical reviews and administrative RCT staff members schedule appointments for VHA facility care. RCT staff forward all community care referrals to the VAMC community care office for scheduling Decentralized Clinical and administrative RCT staff are embedded within each specialty and 79 each RCT staff member only clinically reviews referrals and schedules appointments for that individual specialty. Hybrid Use of elements from both a decentralized and centralized model. For example, 30 a hybrid model could include one RCT with centralized clinical reviewers and decentralized schedulers, according to VHA officials. Source: GAO analysis of VHA scheduling policy documentation and VHA officials. | GAO-23-105617 According to VHA guidance, there are several factors, such as availability of resources, that VAMCs should consider when deciding which Referral Coordination Initiative model to implement. 32 For example, an official from one VAMC we spoke with told us they chose a decentralized model because the VAMC had nursing and scheduling staff shortages. At the VAMC, providers and nurses from clinical specialties conduct clinical reviews for the RCT and schedulers within each specialty schedule VHA facility appointments. Another VAMC we spoke with hired both clinical nursing staff and schedulers to perform RCT responsibilities, and yet another VAMC is following a hybrid model with the intention of transitioning into a fully centralized Referral Coordination Initiative model. Officials from this VAMC told us that they chose to transition to a centralized model because it is easier to monitor compliance of a centralized team rather than decentralized teams reporting to multiple supervisors. 32At least one VAMC was already following a referral review process similar to the Referral Coordination Initiative prior to its implementation, according to VAMC officials. Page 13 GAO-23-105617 Veterans Health Care VHA Has Scheduling Timeliness Standards, but Not for When Appointments Should Occur; Most VAMCs Did Not Meet VHA's Community Care Standard Timeliness Standards Our review of VHA documentation shows that VHA has established Exist for Scheduling standards that facilities need to meet to ensure the timeliness of scheduling specialty care appointments for VHA facility and community Specialty Care care. These timeliness standards differ for VHA facility appointments and Appointments but Not for community care appointments. 33 According to VHA policy: the Maximum Amount of Time Veterans Should • specialty care appointments at VHA facilities must be scheduled Wait for Care within 3 business days from the date a provider enters a referral into the veteran's electronic health record-known as the file entry date and; • community care specialty care appointments must be scheduled within 7 days of the file entry date. • In addition to the overall scheduling timeliness standards, VHA has also established timeliness standards for how long it should take to review referrals, for both VHA facility and community care appointments, as shown in figure 2 below. However, VHA has not established a timeliness standard specifying the maximum number of days a veteran should have to wait before his or her appointment, for both VHA facility and community care appointments (see fig. 2). 33VHA outlines timeliness standards for contractors that are separate from the VHA facility and community care timeliness standards. For example, contractors have two business days to accept or reject a referral and appointments must be scheduled within five business days of the date the contractor accepted the referral. Page 14 GAO-23-105617 Veterans Health Care Figure 2: Timeliness Standards for Scheduling VHA Facility and Community Care Referrals, as of September 2022 Note: The timeliness standards are not counted cumulatively. For example, internal appointments need to be scheduled within three business days, but the referral needs to be marked as active within two of those business days. a The file entry date to forwarded standard only applies to referrals that need to be forwarded to community care, according to VHA officials. b VHA measures referrals timeliness from the file entry date to when the appointment is first scheduled. VA's strategic plan includes a goal to provide veterans with timely access to care and states that a key component of access is the time it takes to Page 15 GAO-23-105617 Veterans Health Care receive care at a VHA facility and in the community. 34 Our previous work on effective management practices has shown the benefit of connecting measurable goals-such as VHA's goal to provide timely access to care-with measurable outcomes. 35 Performance goals communicate the target the agency seeks to achieve within a certain timeframe and performance indicators are measures of the progress the agency is making towards a goal or target. 36 However, according to VHA officials, there is no standard for the number of days in which a veteran's appointment should occur for VHA facility appointments because the department no longer sets goals for how long it should take a veteran to receive care. 37 Instead, officials told us that VHA wants staff to focus on the appropriateness of clinical care and the needs of individual veterans. In 2018 we recommended that VHA establish time frames applicable to community care within which veterans' (1) referrals must be processed, (2) appointments must be scheduled, and (3) appointments must occur, to help ensure VHA has the ability to monitor whether veterans are receiving timely access to care. 38 Although VHA agreed with the recommendation, and implemented the first two components of our recommendation, the agency has not established time frames within which veterans' appointments must occur. As a result, our recommendation has not yet been fully implemented. In the past, VHA officials have stated concerns with establishing a wait time goal for community care as VHA cannot control how quickly veterans are seen by community care providers. However, in July 2022, VHA officials reported to us that they are working on identifying accurate, appropriate, and achievable measures for the community care program in response to this 34Specifically, VA's strategic goal is that the department will deliver timely, accessible and high-quality benefits, care and services to meet the unique needs of veterans and all those that VA serves. See Department of Veterans Affairs, Fiscal Years 2022-2028 Strategic Plan (Washington, D.C.: 2022). 35GAO-15-602. 36Office of Management and Budget, Cir. No. A-11, at § 240.3. 37Congress had previously set a wait time goal for community care appointments under the Veterans Access, Choice, and Accountability Act of 2014. Veterans were expected to receive care through the Veterans Choice Program, which was VHA's previous community care program, no more than 30 days from the date an appointment was deemed clinically appropriate or from the date the veteran preferred to receive care. 38GAO-18-281. Page 16 GAO-23-105617 Veterans Health Care recommendation. We are reiterating the importance of this recommendation. However, even if VHA fully implemented our 2018 recommendation, VHA would still be limited in its ability to assess progress toward its strategic goal of ensuring timely access to care. This is because VHA also lacks a standard specifying the maximum number of days within which veterans' appointments with VHA facility providers should occur. If VHA had a timeliness standard for the number of days in which a veteran's appointment should occur at VHA facilities and through community care, it would allow VHA to measure VAMCs' performance against the standard to determine the extent to which veterans are receiving timely access to specialty care. Furthermore, collecting data on VAMCs' ability to meet this standard would help VHA determine where additional resources or actions may be needed, such as expanding capacity for care at VHA facilities or expanding the community care networks. Most VAMCs Did Not Meet Our review of VHA appointment scheduling data from the third quarter of the Timeliness Standard fiscal year 2022 indicates that VAMCs performed better scheduling VHA facility appointments in a timely manner in comparison to community care for Scheduling Community appointments. For VHA facility appointments, VHA's data show that over Care Appointments, and 90 percent of VAMCs scheduled more than half of their VHA facility VHA's Analysis Used to appointments within 3 business days during the third quarter of fiscal year Create the Standard Was 2022. However, most VAMCs were unable to meet the standard VHA has Limited established for scheduling community care appointments. Specifically, less than 40 percent of VAMCs scheduled more than half of their community care appointments within the 7-calendar-day standard and fewer than 10 percent of VAMCs scheduled more than 75 percent of appointments within the standard (see fig. 3). 39 39See Appendix III for data on the performance on scheduling timeliness standards by VISN. Page 17 GAO-23-105617 Veterans Health Care Figure 3: Percent of Referrals that Met the 3-business day VHA Facility and 7-day Community Care Timeliness Standards in the Third Quarter of Fiscal Year 2022, by VAMC Notes: According to VHA officials, these data are primarily for specialty care referrals but include a small portion of primary care referrals. This figure includes data for 137 VAMCs for VHA facility referrals and 136 VAMCs for community care referrals. The data for VHA facility referrals has 137 VAMCs because VHA only captures data for the VHA clinic in the Manila, Philippines in the VHA facility data. In September 2022, VHA officials said they recognize that many VAMCs have struggled to meet the 7 day timeliness standard for scheduling community care appointments that VHA implemented by March 2022. VHA officials also told us that because the current standard is based on data analysis that is a few years old-having been conducted between 2020 and 2021-the department is considering conducting another review of the community care timeliness standard. Officials did not have additional details on their plans for an updated analysis. Page 18 GAO-23-105617 Veterans Health Care We found that VHA's 7-day timeliness standard was developed and implemented based on data from a limited number of VAMCs. According to documentation from VHA and interviews with VHA officials, VHA's prior analysis reviewed community care scheduling timeliness data from only 21 VAMCs. Moreover, between December 2020 and February 2021, those 21 VAMCs were among the worst performers-taking longer than 30 days to schedule community care appointments. Officials told us they used results from the review and held internal discussions to develop a phased approach that reduced the number of days VAMCs have to schedule a community care appointment from 21 days of the file entry date starting in the fourth quarter of fiscal year 2021 to 7 days of the file entry date starting at the end of the second quarter of fiscal year 2022. 40 However, VHA did not have a clear rationale for setting the standard at 7 days and officials were unable to provide documentation that explained the basis for the standard. 41 VHA's use of a limited data set to inform its appointment scheduling timeliness standard is inconsistent with best practices related to performance management. Our previous work on performance management has shown that using baseline and trend data on past performance can provide an agency with a context for drawing conclusions about whether performance goals-such as VHA's goal to schedule community care appointments within 7-days-are reasonable and appropriate. 42 If VHA conducted a comprehensive analysis of its available baseline and trend data on past performance for community care scheduling timeliness across all VAMCs, it could have a sounder basis for determining whether the standard is achievable or whether revisions to its standard for community care appointments or other adjustments to its scheduling process may be necessary. 40This approach also included an interim goal to have community care appointments scheduled within 14 days by the end of the first quarter of fiscal year 2022. VHA also included a goal to schedule community care appointments within three days to eventually align the community care standards with the VHA facility timeliness measures. The timeliness standard of 3 business days for scheduling VHA facility appointments remained unchanged over this time. Prior to developing the phased approach, VHA was monitoring whether 90 percent of community care appointments were scheduled within 30 days of the file entry date. 41VHA officials also told us that this analysis was also intended to help them identify which VAMCs would benefit from having contractors provide appointment scheduling support. 42GAO/GGD/AIMD-99-69. Page 19 GAO-23-105617 Veterans Health Care In response to the Isakson-Roe Act, in August 2021, VHA developed a VHA Has tool-a dashboard that VHA refers to as the Megabus Scheduling and Implemented New Consult Management Monitor Tool-to monitor VAMCs' performance on the scheduling timeliness standards for both VHA facility and community Tools to Monitor care appointments. According to VHA documentation, the dashboard- Scheduling which is populated with data from veterans' electronic health records- can be used by VAMCs to determine the effectiveness of current VAMC Timeliness operations, observe trends over time, and determine the root causes behind those trends. 43 For example, the dashboard, using data from each VAMC, calculates the percentage of VHA facility appointments scheduled within 3 business days of the file entry date. The dashboard is updated daily and provides VHA a snapshot of VAMCs' performance on the timeliness standards, according to VHA officials. VAMCs can use the dashboard to compare their performance on the timeliness standards to other VAMCs in their VISN. In addition, officials at four of the six VAMCs we spoke with told us they track their performance on the scheduling timeliness standards locally using this dashboard. According to officials from VHA, VAMCs can also use other available data to monitor performance on the scheduling timeliness standards. In December 2021, VHA reported that all VAMCs had accessed the dashboard. Furthermore, in June 2022, VHA implemented a new tool to provide field support to individual VAMCs that are performing poorly on both VHA facility and community care appointment scheduling timeliness, according to VHA officials. 44 Through this tool, alerts are issued for VAMCs whose average number of days from the file entry date to when an appointment is first scheduled for VHA facility and community care increases three months in a row. VHA officials told us that a VHA field support team reviews the alerts every 2 weeks to determine which VAMCs need assistance from VHA, and VAMCs receive varying levels of VHA assistance, if any, depending on the number of alerts issued. 45 VAMCs 43The data comes from VA's Corporate Data Warehouse, which is a central clearinghouse that VA has to compile data from across VA's data systems. 44The tool also issues alerts when VAMCs are performing poorly on other measures. For example, the tool issues alerts when the percent of a VAMC's referrals classified as stat (i.e., urgent) taking more than 48 hours to complete from the file entry date increases three months in a row. In total, VHA is monitoring 20 measures using this tool. 45According to VHA officials in August 2022, the agency is modifying the frequency for reviewing the alerts as VHA continues to develop and refine the processes for this oversight. Page 20 GAO-23-105617 Veterans Health Care that have less than four alerts receive no immediate support, and VAMCs with four or more alerts fall into one of four support levels, according to written documentation we received from VHA (see table 2). Table 2: Veterans Health Administration (VHA) Field Support Engagement Levels VHA support level Number of alertsa Field support actions Level 1 – Low 4-5 alerts VHA discusses with the Department of Veterans Affairs medical center (VAMC) at bi-weekly field support meeting. Level 2 – Medium 6-7 alerts VHA conducts focused review and sends recommendations. Level 3 – High 8-9 alerts VHA launches a virtual site engagement process and develops a site action plan. Level 4 – Urgent 10+ alerts VHA immediately initiates a virtual and in-person site engagement process and develops a site action plan. Source: GAO Analysis of VHA oversight documentation. | GAO-23-105617 a Alerts are issued based on VAMC performance on specific measures, including the scheduling timeliness standards. VHA initiates virtual site engagement with VAMCs that fall into the high support level, and for VAMCs in the urgent support level, VHA is to immediately initiate both virtual and in-person site support. According to VHA officials, for both virtual and in-person site visits, VHA officials meet with VAMC leadership and collaborate with them to develop a site action plan used to assist sites in completing any actions identified through VHA's field support. According to VHA officials, as of August 2022, VHA had initiated field support with 15 VAMCs at the medium and high support levels. According to VHA officials, as of September 2022, VHA is using this effort to provide support to individual VAMCs and VHA does not plan to use data from this effort to evaluate the VHA facility or community care timeliness standards. In addition, due to the recent implementation of this oversight effort, VHA officials told us that they are still updating and making modifications to the tool and how it is used. As a result, the success of this effort is not yet known. Page 21 GAO-23-105617 Veterans Health Care VHA Provides Training to Schedulers, but Does Not Require Providers to Take Training on VHA's Updated Scheduling Process VHA Provides Training to In response to a requirement in the Isakson-Roe Act, VHA revised its New and Existing training materials and implemented new training requirements. 46 VHA Scheduling Staff on the facility and community care schedulers and scheduling supervisors were required to complete these requirements between August 10, 2021, and Scheduling Process for September 30, 2021. 47 The training materials we reviewed covered the VHA Facility and scheduling process for both VHA facility and community care Community Care appointments, including information on how the RCT fits into the Appointments scheduling process. VHA also incorporated the information from this new training into the mandatory training for new scheduling staff, VHA documentation shows. 48 Each VAMC director had to certify that the required staff had completed the required training or report any exceptions by September 30, 2021. 49 According to the memo that VHA sent to VAMCs outlining the new training requirements, any staff who did not complete the training by the deadline were to lose their access to VHA's scheduling systems and 46See Pub. L. No. 116-315, tit. III, § 3101(b)(1), 134 Stat. at 5000 (2021). 47VHA's scheduling systems include the Veterans Health Information Systems and Technology Architecture system, which is VHA's scheduling system for appointments at VHA facilities, and community care's HealthShare Referral Manager. Community care staff at VAMCs use HealthShare Referral Manager to manage community care referrals. 48New staff must complete training before they are given access to VHA's scheduling systems. Officials at five of the six VAMCs we spoke with said that new schedulers typically receive additional training or support after they are placed in a VHA clinic or the community care office. 49Each VAMC also was responsible for reporting training completion for the schedulers and scheduling supervisors at other VHA facilities connected to the VAMC, such as VHA community-based outpatient clinics. Page 22 GAO-23-105617 Veterans Health Care therefore their ability to schedule VHA facility or community care appointments. 50 According to VHA officials, all VAMCs submitted certifications stating required staff had either completed the necessary training, or had an exception, by September 30, 2021. Exceptions to training completion that VAMC directors cited in the certifications we reviewed for the six VAMCs we spoke with included staff who were on extended leave and staff who had transferred to another VAMC, among other reasons. Officials at three of the six VAMCs in our review said that they only had to remove access to VHA's scheduling systems in limited instances. For example, staff who had access to the appointment scheduling system but rarely scheduled appointments, such as providers, or staff who were on extended leave had access removed. Some of the exceptions VAMCs reported in September were temporary. Accordingly, in December 2021, VHA officials followed-up with 27 VAMCs that still had active scheduling staff, less than 100 schedulers in total, who had not completed the training by the initial deadline to confirm that they had since completed the training. 51 VHA Has Developed In May 2022, VHA developed additional Referral Coordination Initiative Training on the Updated training materials for RCT staff and other stakeholders–such as referring providers–that were shared with VAMCs. 52 According to the Referral Scheduling Process for Coordination Initiative Guidebook, these materials were developed to Providers, but the Training provide the necessary training for each individual with a role in the Is Not Required scheduling process. These new training materials include a guide, which outlines the Referral Coordination Initiative and scheduling process step- by-step and who is responsible for each step. They also include role- based training curriculums that are tailored to what referring providers who enter referrals and providers on RCTs who conduct the clinical review of referrals need to understand about their role in the updated 50If staff completed the training after losing their access, they could have their access to the systems restored. 51As of January 2022, there were approximately 35,800 schedulers across VHA, which included both VHA facility and community care schedulers. 52In addition to curriculums for providers, VHA also developed role-based training curriculums that target the learning needs of other staff based on their RCT or stakeholder role, including RCT administrative staff, and all VHA staff. Page 23 GAO-23-105617 Veterans Health Care scheduling process. 53 As part of these curriculums VHA developed new training courses, including a "Referral Coordination Initiative Overview" course and a course titled "A Day in the Life of a Referral Coordination Team Member". In addition to the new training VHA developed in May 2022, VHA officials said that new resident trainees receive some referral management training when they first start working at a VAMC. However, there is no required training for other providers. VHA officials said that they expect to require training for providers when an updated VHA directive on referral processes and procedures is released, which as of September 2022, officials anticipated would be by the end of the year. 54 However, VHA officials told us that they do not plan to require referring providers or providers who are RCT clinical reviewers to utilize the new training materials. According to VHA officials, they highly encourage VAMCs to assign the new training to staff, but that this is ultimately at the discretion of the VAMCs. Officials at the six VAMCs we spoke with reported varying levels of familiarity with the new training materials and how they planned to utilize them. For example, at one VAMC, an official was unaware of the new training materials; while officials from another VAMC said that they planned to include the materials when training new RCT members and planned to inform current staff that this training is available. Moreover, it is unclear when the new training course for providers that VHA officials told us will coincide with release of the directive will be available, since the time frame VHA officials have given us for the expected release of the new directive has been repeatedly delayed during the course of our review. Staff at five of the six VAMCs we spoke with reported experiencing challenges with providers related to the updated scheduling process. For example, a clinical RCT member at one VAMC we spoke with said that the biggest challenge the RCT has is that referring providers do not know about the RCT. This can result in referrals not being routed through the RCT for review before they go to community care. At another VAMC, 53As described earlier in this report, 1) referring providers, who are VHA primary care providers, enter referrals for specialty care, and 2) a provider who is a clinical RCT member, reviews referrals for the RCT. VAMCs that cannot assign dedicated staff, such as nurses, may utilize providers as clinical reviewers for the RCT. 54The directive currently in place was published in 2016 and last updated in December 2021. See Department of Veterans Affairs, Consult Processes and Procedures, VHA Directive 1232(4) (Washington, D.C.: Aug. 24, 2016, amended Dec. 14, 2021). Page 24 GAO-23-105617 Veterans Health Care community care officials reported receiving referrals from the RCT with missing information on veterans' preferences, or clinical information, such a lab results or imaging studies. These community care officials said that referrals forwarded to them with these gaps result in the need for back and forth communication with the RCT or referring provider, affecting their ability to schedule an appointment with a community provider in a timely manner. Further, according to officials at another VAMC, referrals are sometimes incomplete because providers serving as clinical reviewers are not utilizing certain tools, such as the Decision Support Tool, that they should during their review of the referral, and that they are working to address this with providers. 55 Both referring providers and providers who are clinical reviewers for the RCT have important roles in the scheduling process, so it is essential that they understand the process and their responsibilities. 56 If VHA were to require both types of providers to take the Referral Coordination Initiative training that it developed in May 2022 and track their completion of the training it would help VHA to ensure that providers have the information they need to carry out their responsibilities and help ensure timely access to care. For example, if referring providers had the Referral Coordination Initiative training it could reduce the amount of back and forth communication between the RCT or community care staff with the referring provider; reducing the time needed for the scheduling process. In addition, if providers who are responsible for the clinical review of referrals for the RCT had training it could help ensure that the necessary clinical details, such as lab results are captured before a referral is forwarded to community care to be scheduled. This, in turn, could help improve community care's appointment scheduling timeliness. As providers' current duties include roles in the scheduling process, requiring such training would also be consistent with leading practices. GAO's guide on leading practices for assessing federal government training 55According to VHA officials, if the clinical reviewer contacts the veteran, they are supposed to fill out the Decision Support Tool to document the veteran's care options at VHA and in the community, and if eligible for community care, whether the veteran is electing to receive care at VHA or in the community. If the clinical reviewer does not contact the veteran, an administrative RCT member is responsible for using this tool to update the referral with this information. 56VHA's Referral Coordination Initiative Implementation Guidebook, which provides guidance to staff, states that ensuring that facility staff understand the role of RCTs in the referral process is a key part of implementing the Referral Coordination Initiative and recommends that staff be cross-trained in the referral and scheduling processes for both VHA facilities and community care. Page 25 GAO-23-105617 Veterans Health Care indicates that a leading practice is for the agency to consider the suitability and timeliness of who is selected to receive training given employees' current duties and existing skills and competencies. 57 VHA has faced longstanding challenges with scheduling VHA facility and Conclusions community care appointments as well as ensuring veterans' timely access to care. Since 2020, VHA has taken steps to update its appointment scheduling process for specialty care referrals-including providing veterans with more information about their options for care, creating new tools for oversight, and developing training for schedulers. However, VHA faces continued challenges in developing an appointment scheduling process that will provide veterans with timely access to care. In 2018, as part of our review of the Veterans Choice Program-which was a precursor to the current community care program-we recommended that VHA develop an appointment scheduling process for community care with related time frames, including a time frame within which appointments must occur. We maintain that it is important to establish this time frame and reiterate that VHA should fully implement this recommendation. Having such a standard in place for community care appointments, as well as a corresponding standard for VHA appointments at VHA facilities, would help ensure that veterans are receiving timely access to specialty care at VHA facilities and in the community. If VHA developed such a standard, officials could use the data to measure how long it is taking veterans to receive care and to identify where VHA could target additional resources or attention, such as expanding capacity for care at VHA facilities or expanding community care networks to help ensure timely access to care. Additionally, VHA has taken the important step of developing timeliness standards for the number of days it should take to schedule a VHA facility or community care appointment. Going forward, analyzing scheduling data from all VAMCs could help ensure that the standard VHA develops for community care is sound. Further by requiring VAMCs to implement training for referring providers and RCT clinical reviewers, VHA could help ensure that they are familiar with all steps in VHA's appointment scheduling process, thereby helping to avoid errors or delays in scheduling appointments for veterans. 57See GAO-04-546G, 49-50. Page 26 GAO-23-105617 Veterans Health Care Addressing these challenges would give VHA an opportunity to mitigate continued concerns about veterans' access to care and improve VAMCs' ability to schedule appointments for specialty care within time frames that are both achievable and consistent with VHA's timeliness standards for the care veterans receive at VHA facilities and in the community. We are making the following three recommendations to VA: Recommendations for Executive Action The Undersecretary for Health should develop a timeliness standard for the number of days within which veterans' appointments with VHA facility providers should occur. (Recommendation 1) The Undersecretary of Health should conduct a comprehensive analysis of appointment scheduling data from all VAMCs to determine whether the community care timeliness standards are achievable and revise them as necessary. (Recommendation 2) The Undersecretary of Health should require referring providers and RCT clinical reviewers to complete the role-based Referral Coordination Initiative training that VHA developed and track completion of the training to ensure familiarity with its updated scheduling process for VHA facility and community care appointments. (Recommendation 3) We provided a draft of this product to VA for review and comment. In its Agency Comments comments, reproduced in appendix IV, VA concurred with our recommendations, and identified steps it is taking to implement them. Page 27 GAO-23-105617 Veterans Health Care We are sending copies of this report to the Secretary of Veterans Affairs, appropriate congressional committees, and other interested parties. This report is also available at no charge on the GAO Web site at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-7114 or silass@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs are on the last page of this report. GAO staff who made major contributions to this report are listed in appendix V. Sharon M. Silas Director, Health Care Page 28 GAO-23-105617 Veterans Health Care Appendix I: Objectives, Scope, and Appendix I: Objectives, Scope, and Methodology Methodology Our objectives for this report were to 1) describe the Veterans Health Administration's (VHA) updated appointment scheduling process for specialty care provided at VHA facilities and through community care; 2) examine VHA's appointment scheduling timeliness standards for specialty care; 3) examine how VHA oversees Department of Veterans Affairs medical centers (VAMC) to help ensure adherence to scheduling timeliness standards; and 4) examine VHA's efforts to provide training on the updated scheduling process. For the purposes of this report, we focused on the scheduling process for specialty care referrals, which includes mental health referrals, for both VHA facility and community care appointments. 1 To describe VHA's updated appointment scheduling process for specialty care provided at VHA facilities and through community care, • we reviewed relevant VHA documentation detailing appointment scheduling policy requirements, roles, and responsibilities, including VHA directives, standard operating procedures, the Referral Coordination Initiative Guidebook, the Office of Community Care Field Guidebook, and a March 2021 report to Congress that the Department of Veterans Affairs (VA) was required to submit in response to the Johnny Isakson and David P. Roe, M.D. Veterans Health Care and Benefits Improvement Act of 2020 (Isakson-Roe Act). 2 Specifically, we reviewed the VHA directives on outpatient scheduling and consult (i.e., referral) processes and VHA's related 1According to VHA officials, referrals for primary care account for only a small portion of referrals. See appendix II for information on how new patients access care at VHA, and the scheduling of return-to-clinic orders (i.e., follow-up appointments). 2See Pub. L. No. 116-315, tit. III, § 3103(b)(2), 134 Stat. 4932, 5004-5005 (2021). According to the Referral Coordination Initiative Guidebook, the Referral Coordination Initiative is an initiative to improve scheduling timeliness through the creation of referral coordination teams at VAMCs that are responsible for reviewing referrals and communicating with veterans about their health care options. Department of Veterans Affairs, Report on Process and Requirements for Scheduling Appointments for Health Care from Department of Veterans Affairs and Non-Department Health Care (Mar. 2021). Page 29 GAO-23-105617 Veterans Health Care Appendix I: Objectives, Scope, and Methodology standard operating procedures on topics such as consult timeliness and minimum scheduling effort. 3 • We interviewed officials from VHA's Office of Integrated Veteran Care-which is responsible for establishing scheduling policy and monitoring veterans' access to care at VHA and in the community- about the appointment scheduling process, including the policy and requirements for scheduling VHA facility and community care appointments, and implementation of the Referral Coordination Initiative. • At six VAMCs selected randomly on the basis of facility complexity and geographic location, we also interviewed VAMC leadership, VHA facility scheduling supervisors and schedulers, community care scheduling supervisors and schedulers, and staff from Referral Coordination Teams (RCT) about their experiences with the appointment scheduling process and the implementation status of the Referral Coordination Initiative and use of RCTs. Details on the process we used to select the six VAMCs are at the end of this appendix under the heading "VAMC Selection Criteria". To examine VHA's appointment scheduling timeliness standards for specialty care, • we reviewed relevant VHA documentation detailing appointment scheduling policy requirements, including VHA's directives on outpatient scheduling and consult processes, VHA's standard operating procedure and fact sheet on consult timeliness, the Referral Coordination Initiative Guidebook, and the Office of Community Care Field Guidebook. • We interviewed Office of Integrated Veteran Care officials about requirements for the appointment scheduling process, including the established timeliness standards and data available for scheduling VHA facility and community care appointments. 3Minimum scheduling effort refers to the number of contact attempts and methods (e.g., phone, letter, etc.) a scheduler must make when trying to reach a veteran to schedule an appointment. See Department of Veterans Affairs, Outpatient Scheduling Management, VHA Directive 1230 (Washington, D.C.: June 1, 2022); Consult Processes and Procedures, VHA Directive 1232(4) (Washington, D.C.: Aug. 24, 2016, amended Dec. 14, 2021); Consult Timeliness, Standard Operating Procedure (Aug. 16, 2022); and Minimum Scheduling Effort for Outpatient Appointments, Standard Operating Procedure (January 2022). Page 30 GAO-23-105617 Veterans Health Care Appendix I: Objectives, Scope, and Methodology • At the six selected VAMCs, we interviewed VAMC officials about VAMC's appointment scheduling process and its associated timeliness standards. • We compared VHA's efforts related to appointment scheduling timeliness standards against Office of Management and Budget guidance and key practices we have identified in our past work. 4 • Furthermore, we analyzed VHA data on VAMCs' performance on the VHA's appointment scheduling timeliness standards for the third quarter of fiscal year 2022, which was the most recent data available at the time of our analysis. This included data on the percentage of referrals at each VAMC and within each VISN that met VHA's timeliness standards for the scheduling of a VHA facility or community care appointment. 5 • We conducted interviews with VHA officials responsible for these data and reviewed documentation related to how the data are measured and collected. We determined the data were sufficiently reliable for the purpose of providing information on VAMCs' performance on these standards. To examine how VHA oversees VAMCs to help ensure adherence to scheduling timeliness standards, • we reviewed relevant VHA documentation detailing their oversight of appointment scheduling policy requirements, including VHA directives on outpatient scheduling and consult processes, VHA's standard operating procedure and fact sheet on consult timeliness, the Referral Coordination Initiative Guidebook, the Office of Community Care Field Guidebook, and a December 2021 VA report to Congress on the 4See Office of Management and Budget, Preparation, Submission and Execution of the Budget, Circular No. A-11, part 6, § 240.3 (Revised August 2022); GAO, Managing for Results: Practices for Effective Agency Strategic Reviews, GAO-15-602 (Washington, D.C.: July 29, 2015); and GAO, Agency Performance Plans: Examples of Practices That Can Improve Usefulness to Decisionmakers, GAO/GGD/AIMD-99-69 (Washington, D.C.: Feb. 26, 1999) 5We present summary level data in the body of the report. Appendix III presents the data by VISN. Page 31 GAO-23-105617 Veterans Health Care Appendix I: Objectives, Scope, and Methodology method it was using to monitor compliance with the scheduling process. 6 • We also reviewed documentation for some of the tools that VHA has developed for oversight, including information on the dashboard on appointment scheduling timeliness standards that VHA developed in response to the Isakson-Roe Act as well as information about the field support that the Office of Integrated Veteran Care provides to VAMCs that do not meet appointment scheduling timeliness standards. 7 • We interviewed Office of Integrated Veteran Care officials about the activities they undertake to review VAMCs' performance on the established timeliness standards for scheduling VHA facility and community care appointments, the reports and dashboards they have developed for oversight of the scheduling process, including the activities of RCTs. • At each of the six selected VAMCs, we interviewed VAMC officials about the oversight they conduct to ensure that the VAMC is adhering to the timeliness standards associated with the appointment scheduling process. Specifically, we asked them about the reports and data they use to conduct their own oversight of the appointment scheduling process as well as the oversight they are subject to from the Office of Integrated Veteran Care. To examine VHA's efforts to provide training on the updated scheduling process, • we reviewed VHA documentation relevant to training on the VHA facility and community care appointment scheduling process. Specifically, we reviewed a VHA memo outlining the training requirements VHA developed in response to the Isakson-Roe Act, follow-up emails from VHA officials to VAMCs regarding training 6Department of Veterans Affairs, Congressionally Mandated Report: Report on Use of Method to Monitor Compliance with Processes and Requirements for Scheduling Appointments for Health Care from Department of Veterans Affairs and non-Department Health Care (Dec. 2021). See Pub. L. No. 116-315, tit. III, § 3103(b)(2), 134 Stat. 4932, 5004-5005 (2021). 7Within 180 days of enactment of the Isakson-Roe Act, VA was to develop a method or tool that would enable the Department to monitor of compliance with the scheduling process and requirements for VHA facility and community care appointments. See Pub. L. No. 116-315, tit. III, § 3101(c), 134 Stat. at 5000-5001 (2021). Page 32 GAO-23-105617 Veterans Health Care Appendix I: Objectives, Scope, and Methodology completion, VAMC-submitted training attestation forms, and training materials for the Referral Coordination Initiative. 8 • We interviewed Office of Integrated Veteran Care officials about the training requirements for VHA facility and community care schedulers, scheduling supervisors, referring providers, and members of the RCTs on the scheduling process. • At the six selected VAMCs, we interviewed VAMC officials about the training that staff receive, including their experiences with the training VHA developed in response to the Isakson-Roe Act. • We evaluated VHA's training in the context of leading practices for training implementation, which includes the appropriate identification of employees to receive training. 9 To collect information for all of our objectives, we conducted virtual site VAMC Selection visits from April through June 2022 with six VAMCs. 10 To ensure a Criteria balanced sampling of VAMCs we considered two criteria in making our random selection of six VAMCs. • VAMC complexity level: VHA assigns each VAMC one of five complexity levels (ranked from most complex to least complex): 1a, 1b, 1c, 2, and 3. 11 For the purposes of selection, we consolidated these complexity levels into three groups. We grouped all level 1a, 1b, and 1c facilities together into a "High" complexity group, all level 2 facilities into a "Medium" complexity group, and all level 3 facilities into a "Low" complexity group. • Geographic location: VHA's healthcare system is divided into 18 regional Veterans Integrated Service Networks (VISN) that are responsible for managing and overseeing the VAMCs within their defined area. To simplify site selection while maintaining geographic 8Within a year of enactment of the Isakson-Roe Act VA was to ensure that each individual involved in scheduling VHA facility and community care appointments certified their understanding of the scheduling process and requirements and that any new scheduling staff would also receive training on the process and requirements. See Pub. L. No. 116- 315, tit. III, § 3101(b), 134 Stat. at 5000 (2021). 9GAO, Human Capital: A Guide for Assessing Strategic Training and Development Efforts in the Federal Government, GAO-04-546G (Washington, D.C.: Mar. 1, 2004). 10For our virtual site visits, we conducted all interviews with VAMC officials via videoconference. 11The complexity level VHA assigns to a VAMC is determined on the basis of the characteristics of the patient population, clinical services offered, educational and research missions, and administrative complexity of the VAMC. Page 33 GAO-23-105617 Veterans Health Care Appendix I: Objectives, Scope, and Methodology diversity, we consolidated the 18 VISNs into four groups that correspond to the four U.S. Census regions: Northeast, South, Midwest, and West (see table 3). Table 3: Veterans Integrated Service Networks (VISN) Grouped by U.S. Census Regions U.S. Census Region Corresponding VISN Northeast 1, 2, 4 South 5, 6, 7, 8, 9, 16, 17 Midwest 10, 12, 15, 23 West 19, 20, 21, 22 Source: GAO analysis of Veterans Health Administration and U.S. Census Bureau information. | GAO-23-105617. Using the VAMC complexity level and geographic location criteria, we created six groups from which to randomly select sites. We prioritized selecting high complexity level facilities across all four U.S. Census regions, due to the higher patient volume and the extent of specialty care offered at high complexity facilities. We also randomly selected one medium complexity facility and one low complexity facility. See table 4 for the sites that we randomly selected from each group. The information we obtained from the virtual site visits is not generalizable to other VAMCs. Table 4: Department of Veterans Affairs Medical Centers (VAMC) Randomly Selected from Site Selection Groups Site selection group Total number (Census region/VAMC of VAMC in Percentage VAMC selected complexity level) group of all VAMC for virtual visita Location Northeast/high complexity 12 9 East Orange East Orange, NJ South/high complexity 38 28 San Juan San Juan, PR Midwest/high complexity 16 12 Edward Hines, Jr. Hospital Hines, IL West/high complexity 17 13 Sacramento Mather, CA All medium complexity 21 16 Central Alabama - Montgomery Montgomery, AL All low complexity 30 22 Anchorage Anchorage, AK Source: GAO analysis of Veterans Health Administration and U.S. Census Bureau information. | GAO-23-105617. a In many areas of the country, VAMCs and clinics may work together to offer services to veterans as a "health care system" in an effort to provide more efficient care. Several of the VAMCs we selected are part of a health care system: East Orange (New Jersey), San Juan (Caribbean), Sacramento (Northern California), Montgomery (Central Alabama), and Anchorage (Alaska). Some health care systems include more than one VAMC, so for the purposes of site selection, we included only the primary VAMC–called the parent station– in our selection. For this reason, we had a total of 134 VAMCs across our six site selection groups. Page 34 GAO-23-105617 Veterans Health Care Appendix II: Appointment Scheduling Appendix II: Appointment Scheduling Scenarios Other Than for Specialty Care Scenarios Other Than for Specialty Care Referrals Referrals Veterans Health Enrollment is generally the earliest step in VHA's scheduling process. If a Administration (VHA) New veteran is not already receiving care through VHA, the veteran must submit an application to enroll before they can schedule a primary care or Patient Appointment specialty care appointment. 1 VHA's Health Eligibility Center processes Scheduling applications and determines eligibility, and the new enrollee appointment request coordinators manage the new enrollee process at the Department of Veterans Affairs medical centers (VAMC). VHA requires veterans' enrollment applications be processed within five business days of receipt. After an application is fully processed and a veteran is eligible, a new enrollee scheduler must initiate scheduling within three business days. A minimum of two contact attempts must be made to the new enrollee and documented in the new appointment request system. If no contact is made by phone, a letter must be mailed out, with a wait of 14 calendar days for a response. If no response is received from the veteran within the 14-day period, the veteran's request is cancelled in the scheduling system. VHA Follow-Up After a veteran has received care, if the provider determines that the Appointment Scheduling veteran needs to return for a follow-up appointment, the provider enters what VHA refers to as a return to clinic order. The return to clinic order is entered prior to the veteran leaving the appointment and should specify scheduling instructions, including the patient indicated date, which is the date the patient and provider have agreed the patient should be seen, and whether the appointment can be a virtual appointment. VHA facility schedulers are responsible for scheduling return to clinic orders. 2 The scheduler can schedule the appointment when the veteran is checking out, or give the veteran the option to be contacted for scheduling closer to the patient indicated date. If a veteran does not check out after an appointment, the scheduler must initiate contact to schedule the return to clinic order within 2 business days of the provider 1Veterans complete VA Form 10-10EZ, Application for Health Benefits, in-person at a VHA facility, by mail, by phone, or online. 2According to VHA officials, referral coordination teams do not clinically review return to clinic orders before scheduling efforts are conducted because the Referral Coordination Initiative is only applicable to new referrals for specialty care. Page 35 GAO-23-105617 Veterans Health Care Appendix II: Appointment Scheduling Scenarios Other Than for Specialty Care Referrals entering the return to clinic order. If the contact attempt is unsuccessful, the scheduler follows the minimum scheduling effort policy. 3 Established patients can also request appointments online for primary care, mental health, and some specialties. There are specific specialties, such as audiology, optometry, and podiatry, that participate in self- scheduling, allowing veterans to schedule an appointment at a VHA facility without a return to clinic order or referral. 3Per VHA's minimum scheduling effort policy, schedulers must make a minimum of two contact attempts using different contact methods; the first can be via telephone, secure message, or email, and the second attempt can also be by letter. After 14 days from the second contact attempt, if the veteran cannot be reached, schedulers can disposition the return to clinic order, meaning the scheduler discontinues contact attempts and cancels the return to clinic order in the system. Some specialties, such as mental health, require additional contact attempts. Page 36 GAO-23-105617 Veterans Health Care Appendix III: Veterans Health Administration Appendix III: Veterans Health Administration (VHA) Data on VHA Facility and Community (VHA) Data on VHA Facility and Community Care Scheduling Timeliness Care Scheduling Timeliness Table 5: Percentage of VHA Facility and Community Care Referrals That Met Appointment Scheduling Timeliness Standards and Average Number of Days to Schedule, Third Quarter Fiscal Year 2022, by VISN VHA Facility Referrals Community Care Referrals Average number of Average number of Veterans Integrated File entry date to business days from file File entry date to calendar days from file Service Network scheduled within 3 entry date to scheduled within 7 entry date to (VISN)a business daysb scheduledc calendar daysb scheduledc Northeast VISN 1 70.7% 7.0 45.0% 17.5 VISN 2 66.9% 6.6 40.9% 23.2 VISN 4 73.7% 4.8 56.7% 12.0 South VISN 5 71.6% 7.7 43.8% 22.5 VISN 6 69.1% 6.9 30.8% 28.5 VISN 7 74.9% 5.6 26.7% 33.7 VISN 8 74.2% 5.2 46.6% 15.0 VISN 9 77.3% 6.3 36.6% 22.0 VISN 16 71.5% 6.9 36.9% 22.1 VISN 17 68.2% 8.1 43.1% 20.4 Midwest VISN 10 72.4% 5.6 51.4% 17.0 VISN 12 69.8% 7.0 44.5% 19.2 VISN 15 71.0% 5.9 44.5% 16.0 VISN 23 74.6% 6.4 44.4% 18.0 West VISN 19 61.6% 11.6 29.0% 31.1 VISN 20d 67.1% 12.2 34.5% 27.3 VISN 21 71.4% 6.1 38.7% 24.1 VISN 22 72.4% 6.2 38.8% 18.7 Source: GAO analysis of Veterans Health Administration (VHA) data on scheduling timeliness. | GAO-23-105617 Note: According to VHA officials, these data are primarily for specialty care referrals but do include a small portion of primary care referrals. Data from the third quarter of fiscal year 2022 (April 1, 2022, through June 30, 2022) were the most recent quarterly data available at the time of our analysis. a VHA's healthcare system is divided into 18 regional VISNs that are responsible for managing and overseeing the VA medical centers (VAMC) within their defined area. Due to past consolidation and reorganization of the VISNs there are no longer VISNs numbered 3, 11, 13, 14, or 18. In the table, each VISN is also grouped with the U.S. Census region its' boundaries predominantly fall within. b VHA measures referral timeliness from the file entry date to when the appointment is first scheduled. Only referrals with a patient indicated date, which is the date the patient and provider have agreed the patient should be seen, that is less than or equal to 90 days from the file entry date were used to calculate this percentage. In addition to referrals with appointments scheduled, the percentage reported here also includes referrals that were closed within this time frame. A referral could be closed if for example, it is determined that the care is no longer needed or if the referral became an e- Page 37 GAO-23-105617 Veterans Health Care Appendix III: Veterans Health Administration (VHA) Data on VHA Facility and Community Care Scheduling Timeliness consult where a provider reviewed the veteran's electronic medical record or information in the referral to provide a response in-lieu of scheduling an appointment. c VHA measures referral timeliness from the file entry date to when the appointment is first scheduled. The overall average across all VAMCs for the number of days to schedule was 6.7 business days for VHA facility referrals and 21.9 calendar days for community care referrals during the third quarter of fiscal year 2022. d Data for the Spokane, WA and Walla Walla, WA VAMCs are excluded from the VISN 20 calculations because they had implemented VHA's new electronic health record before the third quarter of fiscal year 2022. Page 38 GAO-23-105617 Veterans Health Care Appendix IV: Comments from the Appendix IV: Comments from the Department of Veterans Affairs Department of Veterans Affairs Page 39 GAO-23-105617 Veterans Health Care Appendix IV: Comments from the Department of Veterans Affairs Page 40 GAO-23-105617 Veterans Health Care Appendix V: GAO Contact and Staff Appendix V: GAO Contact and Staff Acknowledgments Acknowledgments Sharon M. Silas, (202) 512-7114 or silass@gao.gov GAO Contact In addition to the contact named above, Michael Zose (Assistant Staff Director), Alison Goetsch (Analyst-in-Charge), Lauren Anderson, and Acknowledgments Audrey Blumenfeld made key contributions to this report. Also contributing were Jennie Apter, Cynthia Khan, Jacquelyn Hamilton, Geovana Mendoza, and Roxanna Sun. Page 41 GAO-23-105617 Veterans Health Care Related GAO Products Related GAO Products Veterans Community Care Program: VA Took Action on Veterans' Access to Care, but COVID-19 Highlighted Continued Scheduling Challenges, GAO-21-476. Washington, D.C.: June 28, 2021. Veterans Community Care Program: Improvements Needed to Help Ensure Timely Access to Care, GAO-20-643. Washington, D.C.: September 28, 2020. Veterans Health Care: Opportunities Remain to Improve Appointment Scheduling within VA and through Community Care, GAO-19-687T. Washington, D.C.: July 24, 2019. Veterans Choice Program: Improvements Needed to Address Access- Related Challenges as VA Plans Consolidation of its Community Care Programs, GAO-18-281. Washington, D.C.: June 4, 2018. Veterans Health Administration: Opportunities Exist for Improving Veterans' Access to Health Care Services in the Pacific Islands, GAO-18-288. Washington, D.C.: April 12, 2018. VA Health Care: Actions Needed to Improve Newly Enrolled Veterans' Access to Primary Care, GAO-16-328. Washington, D.C.: March 18, 2016. VA Mental Health: Clearer Guidance on Access Policies and Wait-Time Data Needed, GAO-16-24. Washington, D.C.: October 28, 2015. VA Health Care: Management and Oversight of Consult Process Need Improvement to Help Ensure Veterans Receive Timely Outpatient Specialty Care, GAO-14-808. Washington, D.C.: September 30, 2014. VA Health Care: Reliability of Reported Outpatient Medical Appointment Wait Times and Scheduling Oversight Need Improvement, GAO-13-130. Washington, D.C.: December 21, 2012. (105617) Page 42 GAO-23-105617 Veterans Health Care The Government Accountability Office, the audit, evaluation, and investigative GAO's Mission arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no cost is Obtaining Copies of through our website. Each weekday afternoon, GAO posts on its website newly GAO Reports and released reports, testimony, and correspondence. You can also subscribe to GAO's email updates to receive notification of newly posted products. Testimony Order by Phone The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's website, https://www.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO on Facebook, Flickr, Twitter, and YouTube. Connect with GAO Subscribe to our RSS Feeds or Email Updates. Listen to our Podcasts. Visit GAO on the web at https://www.gao.gov. Contact FraudNet: To Report Fraud, Website: https://www.gao.gov/about/what-gao-does/fraudnet Waste, and Abuse in Automated answering system: (800) 424-5454 or (202) 512-7700 Federal Programs A. Nicole Clowers, Managing Director, ClowersA@gao.gov, (202) 512-4400, U.S. Congressional Government Accountability Office, 441 G Street NW, Room 7125, Washington, Relations DC 20548 Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 Public Affairs U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548 Stephen J. Sanford, Managing Director, spel@gao.gov, (202) 512-4707 Strategic Planning and U.S. Government Accountability Office, 441 G Street NW, Room 7814, External Liaison Washington, DC 20548 Please Print on Recycled Paper.