GAO United States Government Accountability Office Report to Congressional Committees November 2022 VA HEALTH CARE Additional Action Needed to Assess the Medical Scribe Pilot GAO-23-105712 GAO Highlights Highlights of GAO-23-105712, a report to congressional committees Why GAO Did This Study Medical scribes document patient health information in electronic health records during patient visits. By assisting providers, medical scribes may reduce providers' documentation burden and allow them to focus on their patients. However, VA researchers have found that limited information is available on provider productivity and patient satisfaction when using scribes. To assess the impact of using scribes on provider efficiency, among other things, the VA MISSION Act of 2018 directed VA to increase the use of medical scribes through a 2-year pilot. The VA MISSION Act also included a provision for GAO to report on the pilot. This report (1) examines VHA's design of the pilot and (2) describes lessons learned that VHA identified from the pilot. GAO reviewed VA and VHA pilot documentation and interviewed VHA officials, VA researchers, and contractors, as well as pilot coordinators from three VA medical centers that participated in the pilot. GAO selected the medical centers for geographic diversity and to include each specialty care area where VHA assigned scribes for the pilot. What GAO Recommends GAO is making one recommendation, that VHA assess the scalability of the medical scribe pilot. VA concurred with GAO's recommendation and identified actions to address it. View GAO-23-105712. For more information, contact Alyssa M. Hundrup at (202) 512-7114 or hundrupa@gao.gov. November 2022 VA HEALTH CARE Additional Action Needed to Assess the Medical Scribe Pilot What GAO Found The Veterans Health Administration's (VHA) scribe pilot team implemented a medical scribe pilot at 11 Department of Veterans Affairs (VA) medical centers from June 30, 2020, through July 1, 2022. Under the pilot, medical scribes assisted providers in cardiology, orthopedic, and emergency departments by updating electronic health records in real-time during patient visits. How Medical Scribes May Assist Health Care Providers in Updating Electronic Health Records During a visit scribes enter patient information in the electronic health record as directed by providers ° * Symptoms * Results * Treatments Ue) E) me. | =) After a visit providers review : and approve [ information entered Source: GAO (information); tasty_cat/stock.adobe.com (provider and patient images). | GAO-23-105712 GAO found that VHA's design of the scribe pilot was generally consistent with four of the five leading practices for pilot design that GAO had previously identified. VHA (1) established objectives, (2) communicated with stakeholders, (3) developed an assessment methodology, and (4) developed a plan for evaluating the pilot. However, VHA's design did not include plans for a scalability assessment, the fifth leading practice for pilot design. A scalability assessment would examine whether or how the pilot's approach could be expanded for broader implementation across VA. This assessment should identify criteria or standards to inform decisions, such as measures to evaluate the costs and benefits of the pilot. VHA officials told GAO they designed the pilot according to the VA MISSION Act, which did not require a scalability assessment. Conducting such an assessment would help inform VHA's decisions about whether or how it might use scribes beyond the pilot. VHA officials identified seven initial lessons learned during the implementation of the scribe pilot. Lessons included considering the use of virtual scribes and standardizing scribe onboarding training. VHA officials said they identified lessons from calls with their research partners and pilot participants-including contractors, scribes, providers, and local pilot coordinators-and from their firsthand experiences implementing the pilot. VHA officials told GAO they plan to review the pilot's final evaluations, which may lead to other potential lessons. After reviewing these evaluations, VHA officials told GAO they plan to make lessons available across VHA. Such lessons could help improve how VA medical centers use scribes outside the pilot and help inform an assessment of the scalability of the medical scribe pilot. United States Government Accountability Office Contents Letter Background 5 VHA's Design of the Scribe Pilot Was Generally Consistent with Leading Practices, But VHA Has Not Assessed Scalability 9 VHA Has Identified Initial Lessons from the Scribe Pilot 14 Conclusions 16 Recommendation for Executive Action 17 Agency Comments 17 Appendix | Medical Scribes in Clinical Settings 18 Appendix || Comments from the Department of Veterans Affairs 23 Appendix III GAO Contact and Staff Acknowledgments 26 Tables Table 1: VHA-ldentified Lessons from the Department of Veterans Affairs Medical Scribe Pilot, as of July 2022 16 Figures Figure 1: Examples of How Medical Scribes May Assist Health Care Providers in Updating Electronic Health Records 6 Figure 2: Leading Practices for Effective Pilot Design 10 Figure 3: Key Practices of a Lessons-Learned Process 15 Page i GAO-23-105712 VA Scribe Pilot Abbreviations EHR electronic health record VA Department of Veterans Affairs VHA Veterans Health Administration This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Page ii GAO-23-105712 VA Scribe Pilot GAO U.S. GOVERNMENT ACCOUNTABILITY OFFICE 441 G St. N.W. Washington, DC 20548 November 15, 2022 The Honorable Jon Tester Chairman The Honorable Jerry Moran Ranking Member Committee on Veterans' Affairs United States Senate The Honorable Mark Takano Chairman The Honorable Mike Bost Ranking Member Committee on Veterans' Affairs House of Representatives Within the Department of Veterans Affairs (VA), the Veterans Health Administration (VHA) operates one of the largest health care systems in the nation, serving about 9 million veterans annually. VA relies on electronic health records (EHR) to help provide quality care to patients and document the delivery of services provided through its medical centers and other facilities.' Although EHRs can help improve efficiency and organization, some VA providers have reported that documenting health information electronically can be time consuming and may negatively affect how they interact with patients during visits. For example, VA providers have reported that by focusing on the EHR during an exam they may make less eye contact with their patients.2 Those patients may interpret this divided focus as the provider not paying attention to them, which can harm provider-patient trust and communication. To help address these concerns, some VA facilities have used medical scribes. Scribes are individuals who assist providers by entering information into a patient's EHR in real-time during appointments, as directed by the provider. Scribes are intended to take some of the 1An EHR is a collection of information about a patient, including the health of the patient and the care provided to that patient. Examples of information included in an EHR are patient demographics, progress notes, medications, vital signs, medical history, immunizations, laboratory data, and radiology reports. 2VISN 23 PACT Demonstration Lab, lowa City VA Health Care System, Health Advocate/Medical Scribe/Dragon Evaluation Final Report, (lowa City, lowa: 2018). Page 1 GAO-23-105712 VA Scribe Pilot documentation burden off providers so that providers can focus on the diagnosis and treatment of the patient during a visit. VA researchers conducted a systematic review of studies from the private sector that suggested scribes may help improve provider productivity and improve patients' access to care and satisfaction.4 However, VA researchers have found there are limited data to support such claims, and little is known about other aspects of using scribes, including training, maintaining, and supervising large-scale use of scribes. The VA MISSION Act of 2018 (VA MISSION Act) directed VA to increase the use of medical scribes across 10 VA medical centers through a 2-year pilot.* The act directed VA to assess several metrics associated with the impact of scribes, including metrics on provider efficiency and patient wait times, among others. VHA organized a scribe pilot team that designed the VA medical scribe pilot and implemented it from June 30, 2020, through July 1, 2022. The act also included a provision for us to report on this pilot. This report 1. examines VHA's design of the scribe pilot, and 2. describes lessons learned that VHA identified from the scribe pilot. To address both objectives, we reviewed VA and VHA documents, including VA's mandated reports to Congress on the pilot's design, implementation, and evaluation."? We interviewed VHA officials from the Office of Veterans Access to Care, including those from the scribe pilot team.® We also interviewed researchers from VA's Partnered Evidence- Based Policy Resource Center and VA's Collaborative Evaluation Center, 3Department of Veterans Affairs, VA MISSION Act of 2018, Section 507, Medical Scribe Pilot Program (Washington, D.C.: Dec. 2020). 4Department of Veterans Affairs, Evidence Synthesis Program, Health Services Research and Development Service, Office of Research and Development, The Effect of Medical Scribes in Cardiology, Orthopedic, and Emergency Departments: A Systematic Review, VA ESP Project #09-009 (Washington, D.C.: Sept. 2020). 5Department of Veterans Affairs, The Effect of Medical Scribes. 6Pub. L. No. 115-182, § 507, 132 Stat. 1393, 1479 (2018). 7The VA MISSION Act required VA to submit reports to Congress no later than 180 days after the commencement of the pilot and every 180 days thereafter for the duration of the pilot. As of July 2022, VA had submitted four mandated reports to Congress. 8In 2022, VA consolidated the Office of Veterans Access to Care and Office of Community Care under a new Office for Integrated Veterans Care. According to VHA officials, this realignment had no impact on the scribe pilot. Page 2 GAO-23-105712 VA Scribe Pilot who developed quantitative and qualitative assessment and evaluation plans for the pilot. In addition, we interviewed officials from VA's Health Services Research and Development Service, who conducted a systematic review of research on scribes in cardiology, orthopedic, and emergency departments in 2020.° These are the three areas of care where VHA placed pilot scribes. We also reviewed documentation and interviewed representatives from the two contractors, C2 Technologies and Manifest Global, VHA retained to hire the contracted scribes for the pilot. 1° To examine VHA's design of the scribe pilot, we reviewed VA and VHA documents describing the design and implementation of the pilot. We also spoke with VHA officials and their research partners, VA's Partnered Evidence-Based Policy Resource Center and VA's Collaborative Evaluation Center, about the actions they took to design the pilot. We compared VHA's actions with our previous work that identified leading practices for effective pilot design.'1 Specifically, we reviewed VHA's actions and its pilot design to determine if the pilot was generally consistent with each of the leading practices for effective pilot design that we had identified. We also compared VHA's actions to federal standards for internal control related to information and communication. '12 We determined that the quality information component of internal control was significant to this objective, along with the underlying principle that management should use quality information to make informed decisions. To describe lessons learned that VHA identified from the scribe pilot, we reviewed documents and interviewed officials from VA and VHA, and representatives from the pilot contractors about any steps they took to identify lessons. We also reviewed records VHA's scribe pilot team created during pilot implementation, such as meeting agendas and notes 8Department of Veterans Affairs, The Effect of Medical Scribes. 10The VA MISSION Act required VA to seek to enter into contracts with entities to employ 20 scribes, in addition to hiring 20 new VA-term employees as scribes. 11GAO, Highway Trust Fund: Federal Highway Administration Should Develop and Apply Criteria to Assess How Pilot Projects Could Inform Expanded Use of Mileage Fee Systems, GAO-22-104299 (Washington, D.C.: Jan. 10, 2022); VA Construction: Strengthened Pilot Design and a Dedicated Team Could Improve Real Property Donation Pilot Program, GAO-19-117 (Washington, D.C.: Dec. 13, 2018); and DATA Act: Section 5 Pilot Design Issues Need to Be Addressed to Meet Goal of Reducing Recipient Reporting Burden, GAO-16-438 (Washington, D.C.: Apr. 19, 2016). 12GAO, Standards for Internal Control in the Federal Government, GAO-14-704G (Washington, D.C.: Sept. 2014). Page 3 GAO-23-105712 VA Scribe Pilot from VHA officials' meetings with contractors, VA medical center staff, and scribes. In addition, we reviewed lessons VHA documented during the pilot's implementation. We also reviewed our previous work on key practices of a lessons-learned process. '3 To gather information on lessons learned during the pilot from the VA medical center perspective, we interviewed local pilot coordinators from three of the 11 VA medical centers participating in the pilot: Clarksburg, West Virginia; Oklahoma City, Oklahoma; and Fargo, North Dakota. We selected the VA medical centers based on variation in urban and rural designation, Veterans Integrated Service Network geographic diversity, and the medical center's need for increased access or increased efficiency, as determined by VA.14 We also selected medical centers to include representation from each of the areas of care in which VHA assigned pilot scribes (cardiology, orthopedic, and emergency departments). We conducted this performance audit from January 2022 to November 2022 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. 13The key practices of a lessons-learned process are based on GAO's prior work across several reports, including GAO, VA Construction: VA Should Enhance the Lessons- Learned Process for Its Real-Property Donation Pilot Program, GAO-21-133 (Washington, D.C.: Dec. 10, 2020); Project Management: DOE and NNSA Should Improve Their Lessons-Learned Process for Capital Asset Projects, GAO-19-25 (Washington, D.C.: Dec. 21, 2018); Telecommunications: GSA Needs to Share and Prioritize Lessons Learned to Avoid Future Transition Delays, GAO-14-63 (Washington, D.C.: Dec. 5, 2013); and Department of the Army, Combined Arms Center, Center for Army Lessons Learned, Establishing a Lessons Learned Program: Observations, Insights, and Lessons (Fort Leavenworth, Kans.: June 2011). 14VA medical centers are organized under a national network of 18 Veterans Integrated Service Networks. Each Veterans Integrated Service Network is responsible for overseeing medical centers within a defined geographic area. The VA MISSION Act required VA to place 70 percent of scribes in specialty care areas with the longest patient wait times or lowest efficiency ratings, as determined by the Secretary of VA. Page 4 GAO-23-105712 VA Scribe Pilot Background A medical scribe may be an unlicensed, certified, or licensed individual who provides documentation assistance to a health care provider. '> As directed by a provider, a scribe may help to navigate the EHR, respond to various messages in the EHR, and enter information in the EHR, among other things. During a visit, a scribe may be located in the exam room with the patient and provider; outside the exam room; or virtually, while working from a remote location.'6 See figure 1 for examples of how scribes may assist providers in updating patients' EHRs. 15The Joint Commission, "Documentation Assistance Provided by Scribes," Behavioral Health Manual (July 26, 2018), accessed Aug. 22, 2022, https:/Avww.jointcommission.org/standards/standard-faqs/behavioral-health/record-of- care-treatment-and-services-rc/000002210. The Joint Commission is a nonprofit corporation responsible for setting standards that hospitals must meet to receive their accreditation; conducting surveys to determine compliance with those standards; and issuing certificates of accreditation, which are valid for a 3-year period. The Joint Commission developed broad guidelines for scribes, but neither endorses nor prohibits the use of scribes. 16A patient may also decline the presence of a scribe during a visit. Page 5 GAO-23-105712 VA Scribe Pilot Figure 1: Examples of How Medical Scribes May Assist Health Care Providers in Updating Electronic Health Records 36. History * Symptoms « Results * Treatments Before a visit During a visit After a visit Scribes pull up a patient's electronic health Scribes enter information in the electronic health Providers review, edit, sign, and date all record and prepare a template for record as directed by providers, such as patient documentation completed by scribes. documenting the visit. history and symptoms, physical examination results, and treatment plans. A patient may decline the presence of a scribe during a visit. Source: GAO (information); tasty_cat/stock.adobe.com (doctor and patient images). | GAO-23-105712 Page 6 GAO-23-105712 VA Scribe Pilot According to VA, medical scribes are intended to help providers use EHRs.'" VA's EHR system is to help support VA's ability to provide health care services to patients.18 However, some VA providers have reported that time spent updating EHRs during visits can impair their ability to connect with patients.'9 To lessen the documentation burden on providers, some VA facilities have used scribes to update EHRs during patient visits. For example, VA used scribes in primary care settings as part of its Health Advocate Proof of Concept Demonstration Project, initiated in 2014 and concluded in 2018. Under this project, health advocates provided scribe services in addition to acting as a patient's health coach. Alternatively, some VA providers use voice recognition software, which transcribe providers' voice dictations, to help them update patients' EHRs. VA MISSION Act The VA MISSION Act directed VA to increase the use of scribes across 10 VA medical centers in a 2-year pilot.2° The act required that VA do the following: e Carry out the pilot at 10 VA medical centers, including at least four such medical centers located in rural areas, at least four such medical centers located in urban areas, and two such medical centers located 17Department of Veterans Affairs, VA MISSION Act of 2018, Section 507, Medical Scribe Pilot Program. 18V/A is modernizing its more than 30-year-old EHR system to help improve health care delivery for patients and facilitate the sharing of health data. VA is in the process of deploying the same EHR system as the Department of Defense and the estimated life cycle cost of VA's modernization effort as of 2021 is $16.1 billion over 10 years. We have previously reported on challenges VA has faced in its EHR system modernization project, as well as in other information technology management efforts. See GAO, Electronic Health Records: VA Needs to Address Data Management Challenges for New System, GAO-22-103718 (Washington, D.C.: Feb. 1, 2022); Electronic Health Records: VA Has Made Progress in Preparing for New System, but Subsequent Test Findings Will Need to Be Addressed, GAO-21-224 (Washington, D.C.: Feb. 11, 2021); and Veterans Affairs: Systems Modernization, Cybersecurity, and IT Management Issues Need to Be Addressed, GAO-21-105304 (Washington, D.C.: July 1, 2021). 19VISN 23 PACT Demonstration Lab, Health Advocate/Medical Scribe/Dragon Evaluation Report. Similarly, research from the private sector indicates some providers in primary care settings experience challenges working with EHRs, such as increased administrative burden, which may affect their productivity, detract from their focus on patients, and increase burnout. See, M. Ziemann, et al., "The Use of Medical Scribes in Primary Care Settings: A Literature Synthesis," Medical Care, vol. 59, no. 10, suppl. 5 (2021): S449- S456. 20Pub. L. No. 115-182, § 507, 132 Stat. at 1479. Page 7 GAO-23-105712 VA Scribe Pilot in areas with need for increased access or increased efficiency, as determined by the Secretary of VA; e« Hire 20 new VA-term employees as scribes and seek to enter into contracts with entities to employ an additional 20 scribes; e Distribute scribes so that each VA medical center in the pilot would be assigned four scribes, with two scribes to be assigned to each of two physicians; e Place 30 percent of the scribes in emergency care and 70 percent in specialty care areas with the longest patient wait times or lowest efficiency ratings, as determined by the Secretary of VA; and e Submit reports to Congress that include a separate analysis of each of the following metrics with respect to both VA-hired and contracted scribes: provider efficiency, patient satisfaction, average wait time, the number of patients seen per day by each provider, and the amount of time required to hire and train a scribe. VHA's Office of Veterans Access to Care organized a scribe pilot team that designed and implemented the VA medical scribe pilot. VHA implemented the pilot from June 30, 2020, through July 1, 2022, at 11 of the 171 VA medical centers-six urban and five rural.21 VHA identified cardiology and orthopedic specialty care areas as having some of the longest new patient wait times and assigned scribes to these departments, in addition to the emergency department. Research and Data on Medical Scribes In September 2020, VA researchers issued a review of studies published from 2010 through 2019 on the effect of medical scribes in cardiology, orthopedic, and emergency departments, the three areas of care where VHA assigned pilot scribes.22 This review focused on the use of scribes outside of VA as researchers identified no studies of scribes in VA health care settings or among veterans. Researchers reported that their review was limited by the quantity and quality of available data in these studies and that the findings they reviewed had limited generalizability. Studies in 21According to VA's December 2020 report to Congress, 55 VA medical centers expressed an interest in participating in the pilot. VA selected 12 VA medical centers to participate in the pilot-10 to meet the requirement and two alternates, in the event that a participating VA medical center could no longer be part of the pilot. During the course of the pilot, one VA medical center dropped out, citing concerns with space in the emergency department and provider preference for using voice recognition software. VHA officials told us that nine other VA medical centers expressed interest in using scribes after the pilot was underway. 22Department of Veterans Affairs, The Effect of Medical Scribes. Page 8 GAO-23-105712 VA Scribe Pilot VHA's Design of the Scribe Pilot Was Generally Consistent with Leading Practices, But VHA Has Not Assessed Scalability VA's review identified the potential advantages associated with using scribes, including improved provider and patient satisfaction, reduction in time providers spent on EHR documentation, improved provider efficiency, and improved quality of EHR notes. VA researchers also reported that there were large gaps in evidence about the use of scribes that would require future research. There is no standard model for scribe programs and the use of scribes can vary widely. Studies in VA's review indicated that the use of scribes has increased since 2010, but there are limited data on the number of scribes employed in the U.S.23 Further, although The Joint Commission developed broad guidance for using scribes, there are no nationwide requirements related to their use.24 Private facilities and health care systems generally tailor scribe programs to meet their individual needs, leading to variations in policies, settings, program structure, roles and responsibilities, training, and compensation, among other things. See appendix | for more information on the use of scribes in the private sector and in VA's scribe pilot. Based on our review of VHA's design of the medical scribe pilot, we found it was generally consistent with four of the five leading practices we identified for effective pilot design. The fifth leading practice is to assess the scalability of the pilot; however, VHA has not done this. The leading practices that we identified for effective pilot design act as a framework to enhance the quality, credibility, and usefulness of a pilot's results. The five leading practices are to (1) establish clear and measurable objectives, (2) communicate with stakeholders throughout the pilot, (3) develop an assessment methodology that details the type and source of information needed, (4) develop a plan to evaluate the pilot's implementation and performance, and (5) assess the scalability of the pilot's design.25 (See fig. 2.) 23Department of Veterans Affairs, The Effect of Medical Scribes. 24The Joint Commission, "Documentation Assistance Provided by Scribes." 25GAO-19-117. Page 9 GAO-23-105712 VA Scribe Pilot Figure 2: Leading Practices for Effective Pilot Design communication G@mwi0o INWEK Establish well-defined, Ensure stakeholder Articulate an assessment Develop a plan that defines | Assess scalability of the appropriate, clear, and communication at allstages methodology that details how the information pilot design to inform a measurable objectives. of the pilot. Identify and the type and source of the collected will be analyzed decision on whether and involve relevant information necessary to to evaluate the pilot's how to implement a new stakeholders, and ensure evaluate the pilot, and implementation and approach in a broader that appropriate two-way methods for collecting that performance. context. communication occurs. information, including Source: GAO. | GAO-23-105712 timing and frequency. Our review of pilot documentation and interviews with VA and VHA officials found that VHA's design of the medical scribe pilot was generally consistent with four of the five leading practices for effective pilot design. Objectives. VHA officials told us their objective for the pilot was to meet the requirements defined in the VA MISSION Act. Overall, this was to increase the use of medical scribes across certain medical centers through a 2-year pilot. The act directed VA to use VA-hired and contracted scribes at ten VA medical centers in emergency and specialty care areas. Stakeholder communication. VHA officials told us they communicated with stakeholders throughout the pilot to obtain feedback on the pilot's design and to assess implementation challenges. Specifically, according to VHA officials and our review of meeting agendas and minutes, the pilot scribe team held recurring weekly, bi-weekly, and monthly calls with pilot stakeholders-including local VA medical center pilot coordinators, both VA-hired and contracted scribes, and the contractors VHA retained to hire contracted scribes. For example, stakeholders discussed challenges with hiring scribes at some medical centers during the COVID-19 pandemic, as well as potential strategies for addressing these hiring challenges. Assessment. VHA officials and VA researchers identified what data was needed to assess the pilot and how and when to collect it. VHA officials Page 10 GAO-23-105712 VA Scribe Pilot partnered with researchers from VA's Partnered Evidence-Based Policy Resource Center and VA's Collaborative Evaluation Center to develop an assessment methodology. This methodology detailed the type of quantitative and qualitative information needed to assess the pilot and how and when to collect that information throughout the pilot. For example, to measure provider efficiency, VA researchers defined how they would collect data on the number of patients seen by providers using scribes, as well as the number of patients seen by a control group of providers who did not use scribes during the pilot. In addition, VA researchers planned to conduct a series of semi-structured interviews with local pilot coordinators and scribes to gather qualitative information about their experiences with the pilot. Evaluation. VHA officials and their research partners developed plans to evaluate the implementation and performance of the scribe pilot using the metrics identified in the VA MISSION Act.26 For example, VA's first report to Congress, in December 2020, described how researchers from VA's Partnered Evidence-Based Policy Resource Center planned to use the quantitative data collected through the assessment methodology to calculate provider efficiency. It also described plans to evaluate the differences in efficiency between providers using VA-hired scribes and those using contracted scribes, between rural and urban settings, and between cardiology, orthopedic, and emergency departments. In addition, VA's report described how the Collaborative Evaluation Center planned to evaluate qualitative interview responses to identify barriers and facilitators to pilot implementation, the appropriateness of scribe training, the scribe hiring process, and implementation differences between VA-hired and contracted scribes, among other things. Based on information researchers collected and analyzed during the pilot, VA reported preliminary evaluation results in mandated reports to Congress issued from December 2020 through July 2022. Preliminary evaluation results in the July 2022 report were based on pilot data from June 2020 through December 2021.27 (See text box.) 26The VA Mission Act required VA to submit to Congress an analysis of each of the following metrics with respect to both VA-hired and contracted scribes: provider efficiency, patient satisfaction, average wait time, the number of patients seen per day by each provider, and the amount of time required to hire and train a scribe. 27In September 2022, after we sent a draft of this report to VA for comment, VA's Partnered Evidence-Based Policy Resource Center and VA's Collaborative Evaluation Center completed their final evaluations of the scribe pilot. VA highlighted some results from these final evaluations in its comment letter, which is reprinted in appendix II. Page 11 GAO-23-105712 VA Scribe Pilot Preliminary Evaluation Results of VA's Medical Scribe Pilot VA's July 2022 report to Congress on the medical scribe pilot included preliminary results of its evaluation. These results were based on analyses of quantitative and qualitative data from June 2020 through December 2021. In particular, VA reported that providers using scribes who have the knowledge and skills to chart visits in real time described improved efficiency, improved quality of notes in patients' electronic health records, more patient-centered time, and an overall better work experience. However, providers working with scribes without a background in medical terminology and prior experience reported added burden and workload. Patients shared positive feedback regarding their experiences with the presence of a scribe or described the scribe's presence as unobtrusive. In its reports to Congress, VA also described pandemic-related disruptions and challenges implementing the pilot that might have led to variation in outcomes. For example, the VA MISSION Act required VA to hire 20 new VA-term employees as scribes and seek to enter into contracts to employ an additional 20 scribes. In implementing the pilot, VA planned to hire and retain two VA-term and two contracted scribes at each of the participating VA medical centers. However, some medical centers were not able to meet these hiring goals. According to VA, this was due, at least in part, to factors such as delayed hiring, postponed pilot implementation, competing funding demands, and space constraints related to the pandemic. VA reported taking steps to overcome these challenges, such as providing options for scribes to work outside of patient exam rooms or virtually, as appropriate. Nevertheless, VA reported that such challenges affected its ability to make comparisons between VA-hired and contracted scribes. VA reported that the impact of scribes and the results of the pilot should be considered in context of the COVID-19 pandemic. In addition, while preliminary results may show positive experiences with using scribes, VA reported there was insufficient data to draw conclusions. Source: GAO analysis of Department of Veterans Affairs (VA) documents. | GAO-23-105712 However, we found that VHA's design of the medical scribe pilot was not consistent with the fifth leading practice for effective pilot design-to assess the scalability of the pilot for broader implementation across VA. Assessing scalability could inform VHA's decisions on whether or how best to use medical scribes beyond the pilot. Assessing scalability would Page 12 GAO-23-105712 VA Scribe Pilot also be consistent with federal standards for internal control, which state that management should use quality information to make informed decisions. 28 VHA officials told us an assessment of scalability was not part of the design of the pilot. They also said they did not develop plans to assess the scalability of the pilot after its completion. VHA officials explained that they designed the pilot according to the VA MISSION Act, which did not include a scalability component, and that assessing scalability was not within the scope of their design. Now that the pilot has ended, VHA Officials said that VHA leadership would make data-driven decisions about the future use of medical scribes after reviewing the pilot's final evaluations.2° They said that these decisions would include any recommendations on increasing the use of scribes. As VHA leadership considers the potential future use of scribes, it would benefit from assessing the pilot's scalability. A scalability assessment should identify criteria or standards that, when applied, help inform decisions about whether a pilot's approach could be expanded. In this case, criteria or standards could relate to the feasibility of using scribes. These could include measures to evaluate the costs associated with hiring and training scribes compared to resulting benefits, such as improved provider efficiency or patient satisfaction. In addition, a scalability assessment should also assess the generalizability of the pilot's design and potential results-that is, whether VHA could apply the pilot's model of using scribes to other areas of care or other VA medical centers and expect the same results.°° The purpose of a pilot is generally to inform a decision on whether or how to implement a new approach in a broader context. By conducting a scalability assessment, VHA would be 28GAO-14-704G. 28According to a VHA official, VHA's Governance Board, which includes VHA senior leaders and Veterans Integrated Service Network directors, would approve any adoption or expansion of medical scribes beyond the pilot program. 30We previously reported that scalability assessment criteria and standards should be observable and measureable events, actions, or characteristics that provide evidence that certain pilot objectives have been met. We also previously reported that, to assess scalability, the pilot design should represent a similar or comparable range of circumstances and populations expected in full implementation, and should be based on other related efforts known to influence implementation and performance, among other sources. For example, the sample of pilot VA medical centers or specific implementation challenges, such as COVID-19, are factors that could affect the scalability of the pilot's findings. For more information on assessing scalability, see GAO-22-104299, GAO-19-117, and GAO-16-438. Page 13 GAO-23-105712 VA Scribe Pilot VHA Has Identified Initial Lessons from the Scribe Pilot better positioned to decide the circumstances under which the use of medical scribes would be effective. Assessing Scalability After the Pilot Has Ended Although the medical scribe pilot has ended, assessing scalability of the pilot, including identifying criteria or standards to use in decision-making, is a crucial step to help ensure Veterans Health Administration leadership make the best use of information from the pilot's evaluations. For example, VA reported that it was not able to meet its objective to hire the required number of scribes in the pilot due to the COVID-19 pandemic. VA reported that it was difficult to hire scribes in rural areas, in part because there were few qualified applicants. VA also reported that contract scribes were better trained than VA-hired scribes, and that the additional time needed to train VA-hired scribes was significant. Such findings from the pilot evaluations could be used with scalability criteria or standards, such as specific costs or benefits criteria, to inform decisions about whether or how the use of scribes might be appropriate or feasible, such as in urban or rural areas or to use VA-hired or contracted scribes. Source: GAO and Department of Veterans Affairs (VA). | GAO 23 105712 VHA's scribe pilot team identified seven initial lessons learned during the implementation of the medical scribe pilot. VHA officials conducted some activities that generally aligned with key practices of a lessons-learned process to identify these lessons. A lessons-learned process is a systematic means for agencies to learn from specific events or day-to-day operations and make decisions about when and how to use that knowledge to change behavior.31 We have previously reported that a lessons-learned process is critical to ensuring that lessons endure, and we identified five key practices for this process: (1) collect information, (2) analyze information to identify lessons, (3) validate that the right lessons have been identified, (4) document lessons, and (5) disseminate lessons.*2 (See fig. 3.) 31GAO-21-133. 32See GAO-21-133; GAO, DOD Utilities Privatization: Improved Data Collection and Lessons Learned Archive Could Help Reduce Time to Award Contracts, GAO-20-104 (Washington, D.C.: Apr. 2, 2020). Page 14 GAO-23-105712 VA Scribe Pilot Figure 3: Key Practices of a Lessons-Learned Process Collect Validate Document Disseminate / \O0Y \ \ =-- \\ \ \ \ ~ } V) ) } » 4 ) VA / Capture data and information through activities such as interviews, discussions, forms, or direct observation. Source: GAO and Department of the Army. | GAOQ-23-105712 Analyze information Validate that the right Document and save Disseminate and share collected to determine lessons have been lessons, such as in lessons to pass on root causes and identify identified and determine an electronic knowledge gained, such lessons that lead to the breadth of their database, for use on as through briefings, recommendations. applicability (e.g., site- or existing and future reports, emails, or project-specific or projects. training. program-wide). Based on our review of VA and VHA documents as well as interviews with VHA officials, VA medical center staff, and contractors, we found that VHA officials conducted some activities that generally aligned with key practices of a lessons-learned process. In particular, officials from VHA's scribe pilot team collected information from calls with contractors, scribes, providers, and local pilot coordinators on their challenges, successes, and experiences during implementation of the pilot. VHA officials also said that the scribe pilot team considered its own firsthand experiences in implementing the pilot. In addition, VHA officials told us they reviewed qualitative data on the barriers and facilitators to pilot implementation that VA researchers collected during interviews with pilot participants. VHA officials said that, in conjunction with their research partners, they analyzed this information and in doing so identified seven initial lessons. (See table 1.) In addition, VHA officials told us they documented these lessons in a SharePoint file, which is accessible to all VHA staff.33 33VHA uses Microsoft SharePoint to store, organize, share, and access information sites. Page 15 GAO-23-105712 VA Scribe Pilot Table 1: VHA-Identified Lessons from the Department of Veterans Affairs Medical Scribe Pilot, as of July 2022 Lesson 1. Hire scribes at the General Schedule 6 pay scale to help recruit more qualified candidates. For example, this can be done by hiring scribes as medical support assistants or health technicians. 2. Consider virtual scribes to assist with alleviating staffing needs, which could be strained due to COVID-19 related quarantine or difficulties with recruiting local candidates. 3. Have a strong point of contact who will take ownership for the success of the scribe program at each participating Department of Veterans Affairs medical center. 4. Consider a standardized scribe onboarding training program. 5. Add the 15 most common diagnosis codes to the scribe-training package so that onboarding scribes may reference these. 6. Address complexities present in emergency departments that may not be found in specialty care areas. For example, build in greater flexibility for scribing hours in the emergency department so that scribes can support providers who do not work traditional 8:00 a.m. to 4:00 p.m. schedules. 7. Have scribes hand out information at the appointment, which could have a positive impact on patient experience while not disrupting the appointment. Source: Veterans Health Administration (VHA). | GAO-23-105712 Note: According to VHA officials, VHA's scribe pilot team identified these seven initial lessons from its firsthand experiences implementing the pilot, and from calls with pilot participants and stakeholders, including scribes and providers from the VA medical centers participating in the pilot, representatives from the contractors VHA retained to hire the contracted scribes, and VA researchers who developed quantitative and qualitative assessment and evaluation plans for the pilot. VHA officials said they were waiting to receive final evaluation results from the VA researchers who were conducting the quantitative and qualitative analyses of pilot information. VHA officials told us they would review the pilot's final evaluations, which might yield additional lessons. After reviewing the final evaluations, VHA officials said they would validate identified lessons to confirm their appropriateness and applicability beyond the pilot. Further, VHA officials told us they plan to make lessons learned available across VHA. The lessons VHA has identified could help improve how VA medical centers use scribes outside the pilot. In addition, lessons from this pilot could provide important information to the scalability assessment previously discussed and help VHA make informed decisions on whether or how to expand the use of scribes beyond this pilot. . VA's medical scribe pilot provided the department with an opportunity to Conclusions assess the use of scribes to document patient visits. However, VHA has not assessed the scalability of the pilot to see if using scribes should be expanded or applied to broader settings, an important purpose of conducting pilots. Such information is crucial to VHA's ability to interpret the pilot's results and make informed decisions about whether or how to continue or expand the use of scribes across the department. By conducting a scalability assessment, VHA has an opportunity to make Page 16 GAO-23-105712 VA Scribe Pilot Recommendation for Executive Action Agency Comments effective use of the information gained from the pilot and inform decisions that could help improve provider productivity and patients' access to care. The Under Secretary for Health should assess the scalability of the medical scribe pilot, including identifying and applying criteria to inform decisions about whether or how to use scribes beyond the pilot. (Recommendation 1) We provided a draft of this report to VA for review and comment. VA provided written comments, which are reprinted in appendix II. In its comments, VA concurred with our recommendation and identified actions to address it. VA reported that VHA recently received the final evaluations for the scribe pilot and would be taking data-driven steps to assess the appropriateness and impact that scribes would bring to VHA. VA also provided technical comments, which we incorporated as appropriate. We are sending copies of this report to the appropriate congressional committees, the Secretary of Veterans Affairs, and other interested parties. In addition, the report is available at no charge on the GAO website at https://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-7114 or hundrupa@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in appendix III. Egan 77 Str rl yy Alyssa M. Hundrup Director, Health Care Page 17 GAO-23-105712 VA Scribe Pilot Appendix |: Medical Scribes in Clinical Settings This appendix includes information on the use of medical scribes in a variety of clinical settings, including Department of Veterans Affairs (VA) medical centers participating in VA's medical scribe pilot and clinical settings in the private sector.1 We reviewed VA's research including VA's systematic review of literature on the effect of scribes in cardiology, orthopedic, and emergency departments.2 In addition, we interviewed Veterans Health Administration (VHA) officials and industry professionals, such as representatives from scribe training and management companies and representatives of professional associations and organizations familiar with the use of scribes in clinical settings in the private sector.? We selected these associations and organizations based on our background research and interviews with VHA officials with the aim of obtaining various perspectives on the use of scribes in the private sector. Our review of documents from and interviews with VA and VHA officials and industry professionals indicate that there is no single model for medical scribe programs and the use of scribes can vary widely. The Joint Commission states that a scribe may be an unlicensed, certified, or licensed person who provides documentation assistance to a provider.4 The Joint Commission has developed broad guidance for using scribes, but there are no nationwide requirements for scribes. According to industry professionals familiar with the use of scribes, differing state and local laws and facility needs also affect how scribes are used. Facilities may tailor scribe programs to meet the individual needs and requirements of their location and facility. The use of scribes in VA medical centers that participated in the pilot and in private sector programs vary in policies, settings, program structure, responsibilities, training, and compensation. 1Within VA, the Veterans Health Administration's (VHA) Office of Veterans Access to Care designed VA's medical scribe pilot and implemented it from June 30, 2020, through July 1, 2022. 2Department of Veterans Affairs, Evidence Synthesis Program, Health Services Research and Development Service, Office of Research and Development, The Effect of Medical Scribes in Cardiology, Orthopedic, and Emergency Departments: A Systematic Review, VA ESP Project #09-009 (Washington, D.C.: Sept. 2020). 3We interviewed representatives or received written responses to questions from the following professional associations: American Health Information Management Association, American College of Emergency Physicians, and American Medical Association; private sector organizations: ScribeAmerica and American Healthcare Documentation Professionals Group; and The Joint Commission, a nonprofit corporation. 4The Joint Commission, "Documentation Assistance Provided by Scribes," Behavioral Health Manual (July 26, 2018), accessed Aug. 22, 2022, https://www.jointcommission.org/standards/standard-faqs/behavioral-health/record-of-care -treatment-and-services-rc/000002210. Page 18 GAO-23-105712 VA Scribe Pilot Appendix |: Medical Scribes in Clinical Settings Policies. The Joint Commission's guidance states that organizations should develop policies or procedures for using medical scribes. For example, The Joint Commission specifies that such policies or procedures should define how to enter orders into the electronic health record (EHR), and how the provider will review and verify orders. VHA developed a Scribe Training Manual for the pilot that reflected that guidance. VHA also directed each VA medical center that participated in the pilot to include specific language in its local policies and procedures. In the private sector, organizations may develop policies to meet their particular needs. Such policies may cover topics like performance, training, or certification, among others. According to representatives from one association we spoke with, an example of a difference in policy is whether (1) scribes are allowed to enter draft orders into EHRs on providers' behalf, which a provider would later approve, or (2) only providers can enter and approve orders in medical records. VHA permitted pilot scribes to enter draft orders into EHRs in preparation for a provider to review and approve. Settings. Scribes in VA's pilot were generally located on-site at VA medical centers in a combination of urban and rural areas in cardiology, orthopedic, and emergency departments. The VA MISSION Act required VA to place 30 percent of the scribes in emergency departments and 70 percent in specialty care areas with the longest patient wait times or lowest efficiency ratings. VA identified cardiology and orthopedics as having some of the longest new patient wait times and assigned scribes to these departments. In response to the COVID-19 pandemic, VHA officials updated their Scribe Training Manual to include virtual scribing options where either the scribe, patient, or provider were in different locations. VA reported that three of the medical centers in the pilot implemented some form of virtual scribing. In the private sector, the use of scribes may vary widely. Scribes may work in a variety of clinical settings, such as hospitals, clinics, family practices, and community health centers. Scribes work in a range of departments, such as primary, emergency, and specialty care. Scribes work in both urban and rural areas, and may work on-site or off-site, such as from home on their personal computers. Some research suggests that virtual scribing options may allow rural providers greater access to scribe 5Pub. L. No. 115-182, § 507(c)(2), 132 Stat. at 1479. Page 19 GAO-23-105712 VA Scribe Pilot Appendix |: Medical Scribes in Clinical Settings support.6 Research also indicates that remote scribing may have expanded during the COVID-19 pandemic.' Program structure. The VA MISSION Act directed VA to use a combination of VA-hired scribes as limited-term employees (for the 2-year pilot) and contracted scribes in the pilot. VHA entered into contracts with C? Technologies and Manifest Global that hired the pilot's contracted scribes.° In the private sector, facilities may operate in-house scribe programs or outsource to companies who specialize in training, placing, and managing scribes. For example, private companies specializing in scribes can help a facility establish a scribe program and assist with administrative and oversight responsibilities. These companies may have a larger pool of trained scribes to draw from, and may help ensure coverage in the event of scribe turnover. Alternatively, in-house scribe programs allow organizations greater oversight of scribes, as the responsibility for the program is internal. According to representatives from one of the medical associations we spoke with, organizations that hire in-house scribes may be able to develop more solid relationships with those scribes as the scribes may be more involved in the departments and facilities for which they work. Roles and responsibilities. The role of scribes in VA's pilot was to provide documentary assistance to certain health care providers.19 For example, scribes would record a patient's current illnesses, medications, or procedures performed by the provider in the EHR. Scribes in the pilot could also assist with other non-clinical tasks, such as conducting 8S. Corby, et al., "The Future of Medical Scribes Documenting in the Electronic Health Record: Results of an Expert Consensus Conference," BMC Medical Informatics and Decision Making, vol. 21, no. 204 (2021). 7J. Gold, et al., "Do You Know What Your Scribe Did Last Spring? The Impact of COVID- 19 on Medical Scribe Workflow," Applied Clinical Informatics, vol. 11, no. 5 (2020): 807- 811. 8Pub. L. No. 115-182, § 507(c)(1), 132 Stat. at 1479 9A representative from C2 Technologies told us the company had prior experience working with VA as well as with providing medical scribes for a pilot for the United States Air Force under the Defense Health Agency. A representative from Manifest Global said the company had previous experience with scribes, as it worked under contract on a medical scribe pilot for the Air Force Medical Operations Agency in support of the Air Force Medical Home branch. 10According to VA's December 2020 report to Congress, in August 2019 VA went through the process to nationally classify a position description for medical scribes. This description included the nature of the assignment, a scribe's principle duties and responsibilities, and the knowledge required by the position. Page 20 GAO-23-105712 VA Scribe Pilot Appendix |: Medical Scribes in Clinical Settings COVID-19 screening questionnaires with patients. However, scribes in the pilot were not to perform any clinical roles. Private sector facilities may use scribes in similar ways, as document support staff, or scribes may have additional duties, some of which may be clinical. For example, depending on the job description, qualifications, and the scope of their responsibilities, scribes may be permitted to perform limited clinical work, such as providing food or drink to a patient or assisting a patient to the bathroom. In some organizations, nurses or medical assistants perform scribe duties in addition to their clinical responsibilities. For example, a nurse performing scribe duties may also complete the patient education, answer questions, print medication lists, and go over next steps in a treatment plan. According to a representative from one medical association, this model could result in a smoother handoff and reduce the number of individuals involved in communicating information about or for a patient, which could reduce the chance of errors. Training. The Joint Commission's guidance states that scribes should be educated or trained on certain topics, such as medical terminology, privacy requirements, and EHR navigation and functionality. VHA's Scribe Training Manual for the pilot identified several trainings for scribes to complete, including trainings on clinic flow and entering orders and progress notes into EHRs. Some private sector companies operate scribe training and credentialing programs. When a scribe completes this training, these companies may help place and manage scribes. On the other hand, organizations operating in-house programs may opt to train their own scribes. Facilities may also provide on-site orientation or practice-related training to help familiarize scribes with the facility or clinic flow. Compensation. VA reported that it paid VA-hired pilot scribes at a General Schedule 4 rate of approximately $15 per hour.'1 However, contracted scribes in the pilot were not subject to the same job description classifications and some may have earned as much as $19 to $25 per hour. Some VA medical centers reported challenges hiring qualified VA candidates at the General Schedule 4 pay level. In July 2022, VA reported that, in response to this challenge, VHA's scribe team 11Department of Veterans Affairs, Report on the Medical Scribe Pilot Program (Washington, D.C.: June 2021). In 2021, General Schedule 4 ranged from approximately $13 to $17 per hour. See Office of Personnel Management, Salary Table 2021-GS, https:/Avww.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/21Tabl es/html/GS_h.aspx, accessed Sept. 1, 2022. Page 21 GAO-23-105712 VA Scribe Pilot Appendix |: Medical Scribes in Clinical Settings attempted to have the scribe position reclassified at a higher pay level, but were unsuccessful. Private sector pay for scribes may vary according to experience, training, and location, with more experienced scribes in urban locations earning more. Industry professionals we spoke with estimated that scribes in the private sector might make between $10 to $30 per hour depending on training, experience, and market demand. Representatives from one organization told us that some college students working as scribes are paid as part of a university's work-study program, and some may get course or experience credit instead of financial compensation. Page 22 GAO-23-105712 VA Scribe Pilot Appendix Il: Comments from the Department of Veterans Affairs DEPARTMENT OF VETERANS AFFAIR WASHINGTON : October 25, 2022 Ms. Alyssa M. Hundrup Director Health Care U.S. Government Accountability Office 441 G Street, NW Washington, DC 20548 Dear Ms. Hundrup: The Department of Veterans Affairs (VA) has reviewed the Government Accountability Office (GAO) draft report: VA HEALTH CARE: Additional Action Needed to Assess the Medical Scribe Pilot (GAO-23-105712). The enclosure contains general and technical comments and the action plan to implement the draft report recommendation. VA appreciates the opportunity to comment on your draft report. Sincerely, -% GALL Tanya J. Bradsher Chief of Staff Enclosure Page 23 GAO-23-105712 VA Scribe Pilot Appendix Il: Comments from the Department of Veterans Affairs Enclosure Department of Veterans Affairs (VA) Comments to the Government Accountability Office (GAO) Draft Report: VA Health Care: Additional Action Needed to Assess the Medical Scribe Pilot (GAO-23-105712) Recommendation 1: The Under Secretary for Health should assess the scalability of the medical scribe pilot, including identifying and applying criteria to inform decisions about whether or how to use scribes beyond the pilot. VA Response: Concur. The Veterans Health Administration's (VHA) Office of Integrated Veteran Care is analyzing the results from the scribe pilot evaluations and will present them to VHA leadership. VHA intends to factor both final evaluation results and anticipated cost analyses into assessing the scalability of the scribe pilot. The scalability assessment will enhance the quality and usefulness of the pilot and will better inform VHA's decisions about whether or how to use scribes in the future. Target Completion Date: December 2022 Page 24 GAO-23-105712 VA Scribe Pilot Appendix Il: Comments from the Department of Veterans Affairs Enclosure Department of Veterans Affairs (VA) Comments to the Government Accountability Office (GAO) Draft Report: VA Health Care: Additional Action Needed to Assess the Medical Scribe Pilot (GAO-23-105712) General Comments: Page 12, Paragraph 1, Line 2. "VHA officials explained that they designed the pilot according to the VA MISSION Act, which did not include a scalability component, and that assessing scalability was not within the scope of their design." VA Comment: The Veterans Health Administration (VHA) agrees that the design of the scribe pilot was in adherence with the VA Maintaining Internal Systems and Strengthening Integrated Outside Networks (MISSION) Act and that scalability was not listed as a requirement for the pilot. However, VHA is taking data-driven steps to assess the appropriateness and impact that implementation of scribing would bring to VHA. VHA recently received the final evaluations for the scribe pilot. The results are being analyzed and will be discussed with VHA leadership. Additionally, VHA is awaiting a cost-benefit analysis from our research partner, Partnered Evidence-based Policy Resource Center (PEPReC), which is expected within the next several months. VHA will consider the findings of PEPReC's report in decision making around scalability of the scribe pilot. The future of scribing in VHA will be based on the information available and direction from VHA leadership. From the final evaluation that PEPReC provided in September 2022, the following positive trends were evident: e Cardiology and Orthopedics saw increases in physician productivity and no declines in patient satisfaction with randomization to scribes. e Orthopedics saw reductions in wait times with randomization to scribes. From the final evaluation that the VA Collaborative Evaluation Center provided in September 2022, the following positive trends were evident: e Improvements in efficiency were reported by providers who saw scribe value and who worked with skilled scribes. e Improvements in quality of care were reported as providers having more focused attention to patient care and improved accuracy of visit documentation. e Providers working with well-trained and/or experienced scribes reported experiencing lower stress and reduced documentation burden, while those working with less-experienced scribes reported added burden. e The quality of care and efficiency shifts were perceived to improve provider satisfaction and Veteran experience. ¢ Improvements in efficiency and quality of care were noted by some to result in better future care coordination and/or a reduced need for follow up visits. Page 25 GAO-23-105712 VA Scribe Pilot Appendix III: GAO Contact and Staff Acknowledgments GAO Contact Alyssa M. Hundrup at (202) 512-7114 or HundrupA@gao.gov Staff In addition to the contact named above, Janina Austin (Assistant Director), Kelly Turner (Analyst-In-Charge), Kye Briesath, Kathryn Acknowledgments Fledderman, and Cathy Hamann Whitmore made key contributions to this report. Also contributing were Peter Del Toro, Diona Martyn, Monica Perez-Nelson, and Roxanna Sun. (105712) Page 26 GAO-23-105712 VA Scribe Pilot GAO's Mission Obtaining Copies of GAO Reports and Testimony The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no cost is through our website. Each weekday afternoon, GAO posts on its website newly released reports, testimony, and correspondence. You can also subscribe to GAO's email updates to receive notification of newly posted products. Order by Phone Connect with GAO To Report Fraud, Waste, and Abuse in Federal Programs Congressional Relations Public Affairs Strategic Planning and External Liaison The price of each GAO publication reflects GAO's actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO's website, https:/Awww.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO on Facebook, Flickr, Twitter, and YouTube. Subscribe to our RSS Feeds or Email Updates. Listen to our Podcasts. Visit GAO on the web at https:/Awww.gao.gov. Contact FraudNet: Website: https:/Avww.gao.gov/about/what-gao-does/fraudnet Automated answering system: (800) 424-5454 or (202) 512-7700 A. Nicole Clowers, Managing Director, ClowersA@gao.gov, (202) 512-4400, U.S. Government Accountability Office, 441 G Street NW, Room 7125, Washington, DC 20548 Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548 Stephen J. Sanford, Managing Director, spel@gao.gov, (202) 512-4707 U.S. Government Accountability Office, 441 G Street NW, Room 7814, Washington, DC 20548 oy ue Please Print on Recycled Paper.