United States Government Accountability Office Testimony Before the Committee on Veterans' Affairs, House of Representatives VA DISABILITY For Release on Delivery Expected at 10:00 a.m. ET Tuesday, January 30, 2018 BENEFITS Opportunities Exist to Better Ensure Successful Appeals Reform Statement of Gene L. Dodaro, Comptroller General of the United States GAO-18-349T January 2018 VA DISABILITY BENEFITS Opportunities Exist to Better Ensure Successful Appeals Reform Highlights of GAO-18-349T, a testimony before the Committee on Veterans’ Affairs, House of Representatives Why GAO Did This Study What GAO Found VA’s disability compensation program The Department of Veterans Affairs’ (VA) plan for implementing a new disability pays cash benefits to veterans with appeals process while attending to appeals in the current process addresses disabilities connected to their military most, but not all, elements required by the Veterans Appeals Improvement and service. In recent years, the number of Modernization Act of 2017 (Act). VA’s appeals plan addresses 17 of 22 required appeals of VA’s benefit decisions has elements, partially addresses 4, and does not address 1. For example, not been rising. For decisions made on addressed is the required element to include the resources needed by the appeal in fiscal year 2017, veterans Veterans Benefits Administration (VBA) and the Board of Veterans’ Appeals waited an average of 3 years for (Board) to implement the new appeals process and address legacy appeals resolution by either VBA or the Board, under the current process. VA needs this information to certify, as specified and 7 years for resolution by the under the Act, that it has sufficient resources to implement appeals reform and Board. The Veterans Appeals Improvement and Modernization Act of make timely appeals decisions under the new and legacy processes. 2017 makes changes to VA’s current VA’s appeals plan reflects certain sound planning practices, but it could benefit (legacy) appeals process, giving from including important details in several key planning areas: veterans new options to have their claims further reviewed by VBA or Performance measurement: VA’s plan reflects steps taken to track appeal directly to the Board. The Act performance, but could articulate a more complete and balanced set of goals requires VA to submit to Congress and and measures for monitoring and assessing performance on a range of GAO a plan for implementing a new dimensions of success. Specifically, the plan reports that VA is developing a appeals process, and includes a process to track timeliness of the new and legacy processes. However, contrary provision for GAO to assess VA’s plan. to sound planning practices, the plan does not include timeliness goals for all five appeals options available to veterans, does not include goals or measures for This testimony focuses on the extent to additional aspects of performance (such as accuracy or cost), and does not which VA’s plan: (1) addresses the explain how VA will monitor or assess the new process compared to the legacy required elements in the Act, and (2) process. Unless VA clearly articulates a complete and balanced set of goals and reflects sound planning practices measures, it could inadvertently incentivize staff to focus on certain aspects of identified in prior GAO work. GAO’s appeals performance over others or fail to improve overall service to veterans. work entailed reviewing and assessing VA’s appeals plan and related Project management: VA’s plan includes a master schedule for implementing documents against sound planning the new appeals plan; however, this schedule falls short of sound practices practices, and soliciting VA’s views on because it does not include key planned activities—such as its pilot test of two of GAO’s assessments. the five appeals options. In addition, the schedule does not reflect other sound practices for guiding implementation and establishing accountability—such as What GAO Recommends articulating interim goals and needed resources for, and interdependencies In its forthcoming report, GAO is among, activities. Unless VA augments its master schedule to include all key considering recommending that VA: activities and reflect sound practices, VA may be unable to provide reasonable fully address all legally required assurance that it has the essential program management information needed for elements in its appeals plan, articulate this complex and important effort. how it will monitor and assess the new appeals process as compared to the Risk assessment: VA has taken steps to assess and mitigate some risks legacy process, augment its master related to appeals reform by, for example, pilot testing two of the five appeals schedule for implementation, and more options through its Rapid Appeals Modernization Program (RAMP). However, as fully address risk. designed, RAMP does not include key features of a well-developed and documented pilot test. For example, VA has not articulated how it will assess RAMP before proceeding with full implementation. In addition, RAMP is not pilot testing three options and, as a result, VA will not have data on the extent to which veterans will appeal directly to the Board when given the option. Unless View GAO-18-349T. For more information, contact Elizabeth Curda at (202) 512-7215 or VA identifies and mitigates key risks associated with implementing a new curdae@gao.gov. process, VA is taking a chance that untested aspects will not perform as desired. United States Government Accountability Office Letter Letter Chairman Roe, Ranking Member Walz, and Members of the Committee: I am pleased to be here today to discuss our review of the Department of Veterans Affairs’ (VA) plan for implementing a new disability appeals process while processing pending appeals under the current, or legacy, process. VA provides cash benefits to veterans for disabling conditions incurred in or aggravated by military service, paying an estimated $67 billion to about 4.3 million veterans in fiscal year 2016. 1 If veterans are dissatisfied with VA’s initial decision they can appeal—first to the Veterans Benefits Administration (VBA) and then, if not satisfied there, to the Board of Veterans’ Appeals (Board), a separate agency within VA. In recent years, the number of appeals of VA’s benefit decisions has been rising. For appeals resolved in fiscal year 2017, veterans waited an average of 3 years from the date they initiated their appeal to resolution by either VBA or the Board—and a cumulative average of 7 years for appeals resolved by the Board. 2 Due in part to the challenges VA faces managing large workloads and deciding disability claims and appeals in a timely manner, GAO in 2003 designated VA disability compensation and other federal disability programs as one of the government’s highest management risks. 3 In a March 2017 report, we examined VA’s approaches to address challenges it identified as contributing to lengthy appeals processing times—including VA efforts to hire staff, propose reform legislation to Congress in April 2016, and upgrade its information technology systems—and the extent to which those approaches were consistent with sound planning practices. 4 We made five recommendations to improve VA’s ability to implement its proposed appeals process reform while addressing a growing appeals workload. VA agreed in principle with our five recommendations, which remain open as of January 2018. We recommended, in essence, that VA develop: (1) a detailed workforce plan, (2) a complete schedule of information technology (IT) updates, (3) 1 VA Fiscal Year 2018 Congressional Budget Justification. 2 VA, Comprehensive Plan for Processing of Legacy Appeals and Implementing the Modernized Appeals System (Washington, D.C.: Nov. 22, 2017). 3 GAO, High-Risk Series: Progress on Many High Risk Areas, While Substantial Efforts Needed on Others, GAO-17-317 (Washington, D.C.: Feb. 15, 2017). 4 GAO, VA Disability Benefits: Additional Planning Would Enhance Efforts to Improve the Timeliness of Appeals Decisions, GAO-17-234 (Washington, D.C.: Mar. 23, 2017). Page 1 GAO-18-349T VA Disability Benefits better estimates of future workloads and timeliness, (4) a robust plan for monitoring appeals reform, and (5) a strategy for assessing whether the new process improves veterans’ experiences over the current process. 5 We also suggested that Congress require VA to pilot test appeals reform. Enacted on August 23, 2017, the Veterans Appeals Improvement and Modernization Act of 2017 (the Act) will make changes to VA’s appeals process. 6 Specifically, the Act replaces the current appeals process with a process that gives veterans various options to have their claim reviewed further by VBA or to bypass VBA and appeal directly to the Board. The Act also requires VA to submit a comprehensive plan for implementing the new appeals process and processing legacy appeals (appeals that remain pending in the current process prior to fully implementing appeals reform) to the appropriate committees of Congress and GAO. 7 The Act delineates the required elements of this plan, and required VA to submit its plan within 90 days of enactment. VA submitted its plan on November 22, 2017. The Act also includes a provision for GAO to assess VA’s appeals plan, including whether the plan comports with sound planning practices and/or contains gaps. My testimony today is based on work being conducted for a report that GAO expects to issue pursuant to the Act. This statement focuses on the extent to which VA’s appeals plan (1) addresses the required elements in the Act; and (2) reflects sound planning practices identified in prior GAO work. To assess the extent to which VA’s plan addresses the required elements in the Act, we identified the required elements for VA’s comprehensive plan under section 3(a) and (b) of the Act; compared the required elements against VA’s appeals plan and supplemental materials VA provided at our request; and made a preliminary determination as to 5 While agreeing in principle, VA stated in its written comments that all five recommendations should be closed because its efforts to date already addressed the recommendations, or further progress was contingent upon passage of appeals reform legislation. We disagreed and maintained that VA needed to take further action on all recommendations. 6 Pub. L. No. 115-55, § 2, 131 Stat. 1105, 1105. (2017) 7 The Act defines “appropriate committees of Congress” to be the Committee on Veterans’ Affairs and the Committee on Appropriations of the Senate, and the Committee on Veterans’ Affairs and the Committee on Appropriations of the House of Representatives. Page 2 GAO-18-349T VA Disability Benefits whether VA’s plan addressed, partially addressed, or did not address each element. We then shared the results of this review with VA officials, and considered their comments in arriving at our assessment. 8 To address the extent to which VA’s plan reflects sound planning practices, we compared the appeals plan and supplemental materials against relevant sound planning practices and other criteria identified in our prior work. 9 Our analyses focused on the information and elements VA presented in its appeals plan and supplemental materials provided by VA, rather than auditing the underlying information. We conducted the work for this statement in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. Background VA’s Disability VA’s process for deciding veterans’ eligibility for disability compensation Compensation Claims begins when a veteran submits a claim to VA. The veteran submits his or her claim to one of VBA’s 56 regional offices, where staff members assist Process the veteran by gathering additional evidence, such as military and medical records, that is needed to evaluate the claim. Based on this evidence, VBA decides whether the veteran is entitled to compensation and, if so, how much. 8 We developed a checklist and decision rules to determine whether VA’s appeals plan addressed, partially addressed, or did not address each required element, or all of the sub-parts of each required element of the Act. Using these tools, an analyst reviewed VA’s plan against each required element and documented a judgment about whether that element was addressed in VA’s appeals plan. An attorney reviewed the analyst’s assessment and documented agreement or disagreement with the initial assessment. Where the attorney and analyst did not agree, a second attorney reviewed and made a final decision. We sent our preliminary assessment to officials at VA for comment and verification, which we considered in arriving at our assessment. 9 See GAO-17-234. Specifically, that report identified best practices and other criteria through a review of relevant literature, such as federal internal control standards and prior GAO reports where we defined a number of desirable characteristics of an effective, results-oriented plan, or components of sound planning practices. For a list of the sources of sound practices and criteria used to evaluate VA’s approaches, see the Related GAO and Other Products section of GAO-17-234 (page 61). Page 3 GAO-18-349T VA Disability Benefits A veteran dissatisfied with the initial claim decision can generally appeal within 1 year from the date of the notification letter VBA sends to the veteran. Under the current appeals process (now referred to by VA as the legacy process), an appeal begins with the veteran filing a Notice of Disagreement. VBA then re-examines the case and generally issues a Statement of the Case that represents its decision. 10 A veteran dissatisfied with VBA’s decision can file an appeal with the Board. In filing that appeal, the veteran can indicate whether a Board hearing is desired. Before the Board reviews the appeal, VBA prepares the file and certifies it as ready for Board review. If the veteran requests a hearing to present new evidence or arguments, the Board will hold a hearing by videoconference or at a local VBA regional office. The Board’s members, also known as Veterans Law Judges, review the evidence and either issue a decision to grant or deny the veteran’s appeal or refer (or remand) the appeal back to VBA for further work. New Appeals Process The 2017 Act made changes to VA’s legacy appeals process that will generally take effect no earlier than February 2019, which is approximately 18 months from the date of enactment. 11 According to its appeals plan, VA intends to implement the Act by replacing the current appeals process with a process offering veterans who are dissatisfied with VBA’s decision on their claim one of five options: two of those 10 According to VA officials, the veteran can elect either a traditional VBA review or a VBA review by a Decision Review Officer (DRO). Under the traditional review option, the veteran may present new evidence and receive a formal hearing. In general, the review can change VBA’s original decision based only on new evidence, or a clear and unmistakable error in the original decision. Alternatively, the veteran may elect a review by a DRO, who reviews the record without deference to VBA’s original decision, and can revise that decision based on a difference of opinion. If needed, the DRO may also pursue additional evidence or discuss the appeal informally with the veteran or the veteran’s representative. 11 Under the Act, the legal changes to VA’s appeals process will generally take effect on or after the later of (1) 540 days (approximately 18 months) after enactment and (2) 30 days after the Secretary of Veterans Affairs submits to the appropriate committees of Congress (i) a certification that VA has the resources, personnel, office space, procedures, and IT required to carry out the new appeals system and to timely address appeals under the new appeals system as well as pending legacy appeals, and (ii) a summary of the expected performance outcomes used in making the certification with respect to legacy claims and a comparison of these expected outcomes with actual program performance with respect to the appeals under the legacy system (before the new system is implemented). Pub. L. No. 115-55, § 2, 131 Stat. 1105, 1105 (2017). Page 4 GAO-18-349T VA Disability Benefits options afford the veteran an opportunity for an additional review of VBA’s decision within VBA, and the other three options afford them the opportunity to bypass additional VBA review and appeal directly to the Board. 12 • Under the new appeals process, the two VBA options will be: 1. Request higher-level review: The veteran asks VBA to review its initial decision based on the same evidence but with a higher-level official reviewing and issuing a new decision. 2. File supplemental claim: The veteran provides additional evidence and files a supplemental claim with VBA for a new decision on the claim. • The three Board options will be: 3. Request Board review of existing record: The veteran appeals to the Board and asks it to review only the existing record without a hearing. 4. Request Board review of additional evidence, without a hearing. 5. Request Board review of additional evidence, with a hearing. VA’s Appeals Plan The Act also requires VA to submit to the appropriate committees of Congress and GAO, within 90 days of the date of enactment, a comprehensive plan for (1) processing appeals under the legacy process until there are no more to process, (2) implementing the new appeals process, (3) processing of claims under the new appeals process in a timely manner, and (4) monitoring implementation of the new appeals process. In addition to these four broad elements, the Act lists 18 elements required to be included in the plan that relate to, among other things: • staffing, information technology (IT), and other resources required to implement the plan; • estimated timelines for hiring and training VA employees; and 12 Veterans who are dissatisfied with VBA’s decision on their appeal may file another appeal to the Board using one of these three options, according to VA officials. Page 5 GAO-18-349T VA Disability Benefits • a description of risks associated with each element of the plan. 13 The Act also includes a provision for GAO to assess the plan within 90 days after VA submits it. 14 The Act also requires VA to provide progress reports to the appropriate committees of Congress and GAO at least once every 90 days (starting after VA submits its plan), until the date the Act’s legal changes to the appeals process generally go into effect and then at least once every 180 days after this date for 7 years. Rapid Appeals The Act also authorized VA to carry out a program to test any Modernization Program assumptions relied upon in developing its comprehensive plan and test the feasibility and advisability of any facet of the new appeals process. In (RAMP) its appeals plan, VA reported its decision to pilot test two of the five new options by allowing veterans with pending appeals in the legacy process (known as legacy appeals) to elect the VBA supplemental claim or the higher-level review options beginning in November 2017. This program, which VA refers to as RAMP, is intended to reduce legacy appeals by providing veterans with a chance for early resolution of their claims within VBA while the Board focuses on reducing its inventory of legacy appeals, according to VA. Participation in RAMP is voluntary, but veterans must withdraw their pending legacy appeal to participate, according to VA. Veterans dissatisfied with their RAMP decisions must wait until VA fully implements the new appeals process (in February 2019 at the earliest) before pursuing an appeal with the Board under the new process, according to VA officials. 13 Pub. L. No. 115-55, § 3(a) and (b), 131 Stat. 1105, 1116. (2017) 14 VA submitted its plan to GAO on November 22, 2017. Page 6 GAO-18-349T VA Disability Benefits VA’s appeals plan addresses 17 of the Act’s 22 required elements, VA’s Plan Addresses partially addresses 4 related to monitoring implementation and workforce Most of the Act’s planning, and does not address 1 element related to identifying total resources. For example, VA’s appeals plan addresses the required Required Elements elements related to, among others, identifying legal authorities for hiring for the New and and removing employees, estimating timelines for hiring and training employees, and outlining the outreach VA expects to conduct. For the Legacy Disability elements in the Act that VA’s appeals plan partially addresses or does not Appeals Processes address, see table 1. For a detailed list of the 22 required elements in the Act, see appendix I. Table 1: The Department of Veterans Affairs’ (VA) Appeals Plan Partially Addresses or Does Not Address 5 of 22 Required Elements of the Veterans Appeals Improvement and Modernization Act of 2017 VA’s comments and our Required element in the Act What VA provides Our assessment response Section 3(a)(4) VA’s plan states that the Partially addressed: VA’s VA officials stated that they A comprehensive plan for agency convened an appeals plan does not contain disagree with our assessment monitoring the implementation enterprise-wide governance sufficient details about metrics and that their appeals plan of the new appeals system, workgroup to oversee for tracking the progress of addresses this required including metrics and goals— implementation and develop implementation element. Their comments metrics and track timeliness of (subparagraph (A)) or the generally restated what is (A) to track the progress of appeals in the legacy system metrics and goals for contained in VA’s plan, except implementation; and the new process. identifying potential issues they added that the agency is (B) to evaluate the efficiency The plan also references VA’s related to implementation developing more detailed and effectiveness of the intended use of certain (subparagraph (C)). project plans and schedules. implementation; and metrics, such as average We continue to believe VA’s (C) to identify potential issues processing time for Veterans plan partially addresses this relating to the implementation. Benefits Administration (VBA) required element. options. Page 7 GAO-18-349T VA Disability Benefits VA’s comments and our Required element in the Act What VA provides Our assessment response Section 3(b)(2) VA’s plan states the number Partially addressed: VA’s VA officials stated that they Delineation of the personnel of personnel required by VBA appeals plan does not provide disagree with our assessment requirements of the and the Board of Veterans’ an estimate of personnel and that their appeals plan Administration and the Board, Appeals (Board) for required for either VBA or the addresses this required including staffing levels during concurrently processing Board for the period when the element. the— legacy and new appeals in agency would no longer be Their comments generally fiscal year 2018 (1,495 and processing legacy appeals. restated what is contained in (A) period in which the 1,050 full-time equivalents, (subparagraph (B).) Administration and the Board VA’s plan. respectively). are concurrently processing— We continue to believe VA’s The plan also references plan partially addresses this (i) appeals of decisions on using data from the Rapid legacy claims; and required element. Appeals Modernization (ii) appeals of decisions on Program (RAMP) and the new non-legacy claims under the system—once fully new appeals system; and implemented—to inform its (B) the period during which personnel needs for when VA the Administration and the is no longer processing legacy Board are no longer appeals.a Further, VA’s processing any appeals of statements suggest the decisions on legacy claims. agency will wait for RAMP results and budget outcomes to estimate future personnel requirements, rather than develop estimates based on a range of assumptions. Page 8 GAO-18-349T VA Disability Benefits VA’s comments and our Required element in the Act What VA provides Our assessment response Section 3(b)(12) For VBA, VA provided Partially addressed: VA’s VA officials stated that they Projections for the productivity information on productivity appeals plan does not contain disagree with our assessment of individual employees at the projections for processing projected productivity and that their appeals plan Administration and the Board legacy appeals in fiscal year information for VBA addresses this required in carrying out tasks relating 2018 that are not in the plan. processing of appeals under element. In their comments, to the processing of appeals Also, VA’s plan states VA the new process. VA officials provided of decisions on legacy claims would use data from RAMP additional information on and appeals under the new and the new process—once productivity projections that appeals system, taking into fully implemented—to develop are not in the plan. For VBA, account the experience level productivity information for the VA provided projections for of new employees and the new process. the legacy process for fiscal enhanced notice requirements For the Board, VA’s plan year 2018. For the Board, VA under section 5104(b) of title provides productivity restated that productivity will 38, United States Code, as projections for processing be 79 appeals per full-time amended by section 2(e). legacy appeals in fiscal year equivalent for fiscal year 2018 2018 and projects that and projects that productivity productivity will be higher will be higher than 79 in the under the new process. new appeals process. We continue to believe VA’s plan partially addresses this required element due to the absence of projected productivity information for VBA processing appeals under the new process, which VA anticipates will be implemented in fiscal year 2019. Section 3(b)(17) VA’s plan states that the rate Partially addressed: VA’s VA officials stated that they Delineation of the key goals at which the legacy appeals appeals plan does not contain disagree with our assessment and milestones for reducing inventory can be resolved milestones for reducing legacy and that this element is the number of pending depends on numerous factors, appeals. The plan also does addressed in their plan. For appeals that are not such as future funding and the not describe the expected the Board, VA also provided processed under the new rate that veterans opt into number of appeals, remands, updated information that appeals system, including the RAMP. VA’s plan indicates and hearing requests at VBA, contained aggregated appeals expected number of appeals, that the agency would collect or the expected number of and remands expected in remands, and hearing information from RAMP and hearing requests at the Board fiscal years 2018 and 2019. requests at the Administration as the Board begins deciding each year. Further, the follow- We continue to believe VA’s and the Board each year, appeals in the new process to up information VA provided for plan partially addresses this beginning with the one year develop accurate goals and VBA and the Board does not required element due to the period beginning on the date milestones for reducing the extend beyond fiscal year absence of (1) milestones for of the enactment of this Act, number of pending legacy 2019, yet VA indicates that reducing legacy appeals; (2) until there are no longer any appeals at VBA and the the Board will be processing expected number of appeals, appeals pending before the Board. For the Board, VA legacy appeals for several remands, and hearing Administration or the Board provided aggregated years. requests at VBA, or the for a decision on a legacy information on appeals and expected number of hearing claim. remands that were not in the requests at the Board each plan. year; and (3) key goals and milestones beyond fiscal year 2019. Page 9 GAO-18-349T VA Disability Benefits VA’s comments and our Required element in the Act What VA provides Our assessment response Section 3(b)(1) VA’s plan states that the Not addressed: VA’s appeals VA officials stated that they Delineation of the total agency will use existing plan does not include a disagree with our assessment resource requirements of the resources devoted to its delineation of the total and that this element is Veterans Benefits workforce, IT systems, and resources required by VBA addressed in their plan. Their Administration and the Board performance tracking, among and the Board to implement comments generally restated of Veterans’ Appeals, other areas, to implement the new system while what is contained in VA’s disaggregated by resources parts of the new appeals addressing pending legacy plan. required to implement and process. VA’s plan indicates it appeals. We continue to believe VA’s administer the new appeals will collect and use data from plan does not address this system and resources RAMP—for example, on the required element. required to address the (1) percentage of veterans appeals of decisions on who opt-in to this program, (2) legacy claims. veterans’ choices within the new VBA options, and (3) VA employees’ productivity—to inform future funding requests through the annual appropriations process. In its plan, VA states that it will efficiently allocate resources to the new appeals process and allocate all remaining resources to address legacy appeals. Source: GAO analysis of VA’s November 2017 appeals plan and supporting documents VA provided on or before January 18, 2018, the Veterans Appeals Improvement and Modernization Act of 2017, and agency input by VA. | GAO-18-349T Note: We identified 22 required elements for VA’s comprehensive plan under section 3(a) and (b) of the Act. Specifically, subsection (a) contains 4 elements, and subsection (b) requires the appeals plan to address 18 elements. Although this table shows the required elements that VA’s plan partially addresses or does not address, VA’s appeals plan addressed most (17 of 22) of the required elements in the Act. In determining that VA’s plan addresses these requirements, we assessed the plan and its elements as presented, rather than auditing the underlying information. a RAMP involves VA inviting certain veterans with pending appeals—starting with those appeals pending the longest—among other eligibility criteria to participate in the higher-level review or supplemental claims options with VBA, which are two of the five options that will be available under the new appeals process to all veterans. When we provided VA with our preliminary assessment, VA officials said they disagreed with our assessment and that their appeals plan addresses all 22 of the required elements. In general, they said that data are not available, and VA cannot yet forecast the information required by the Act until aspects of the new appeals process are tested or implemented. We continue to believe the information as presented in VA’s appeals plan and supplemental materials addresses 17 of the required elements, partially addresses 4, and does not address 1 element. Without complete information on all 22 of the required elements, Congress does not have the information it needs to fully conduct oversight of VA’s appeals plan Page 10 GAO-18-349T VA Disability Benefits and the agency’s efforts to implement and administer the new process while addressing legacy appeals. VA also is required to provide information on resources, among other areas, before it can certify that the agency is prepared to carry out timely processing of appeals under the new and legacy appeals process. 15 Further, as discussed below, addressing required elements through a more comprehensive plan and underlying analysis is consistent with sound planning practices and would better position VA to implement the new appeals process while attending to legacy appeals; for example, a plan that provides for carefully monitoring the new and legacy appeals processes against balanced goals and metrics, and clearly articulates resources, milestones and other information needed for effective program management. 15 VA’s plan states that the agency intends to begin full implementation in February 2019. Page 11 GAO-18-349T VA Disability Benefits VA’s appeals plan reflects certain sound planning practices, such as VA’s Appeals Plan convening a working group on performance tracking; however, the plan Reflects Certain could benefit from including important details related to three key planning areas: Sound Planning Practices, but Could 1. articulating a balanced set of goals and related measures to monitor and assess the performance of the new appeals process, Improve on Others in conjunction with the legacy process; 2. developing a high-quality and reliable implementation schedule to manage key steps and activities of the project; and 3. assessing key risks in a comprehensive manner, including respective mitigation strategies, and articulating clear criteria and an assessment plan for RAMP, and more fully testing or analyzing all appeal options. VA’s Appeals Plan VA’s appeals plan reflects steps taken to track performance, but it could Indicates Steps to Assess improve its planning practices related to monitoring and assessing performance on a range of key dimensions of success. Sound planning Process Changes, but practices suggest that agencies develop overall goals tied to meaningful Should Also Include Goals and balanced performance measures. These measures include a mix of and Measures to Provide outcome, output, and efficiency measures to ensure that an Full Picture of Success organization’s priorities—as well as government-wide priorities such as quality, timeliness, and cost of service—are addressed. 16 VA’s appeals plan reports that the agency convened a working group to design a process for tracking timeliness of both the legacy appeals and appeals within the new process. In supporting documentation that we requested, VA officials stated they are also determining the best way to measure veterans’ satisfaction with the new appeals process. VA’s appeals plan and supporting documentation also identify timeliness goals for the two VBA-only options and one of the three Board options. Nevertheless, its appeals plan does not articulate a set of goals and measures that cover all aspects of its new appeals process, such as accuracy of decisions and cost. The plan also does not provide details on 16 See GAO, Tax Administration: IRS Needs to Further Refine Its Tax Filing Season Performance Measures, GAO-03-143 (Washington, D.C.: Nov. 22, 2002). In addition, the Act requires VA’s plan to contain goals and related measures that can be used to manage and assess implementation of VA’s new appeals process, and key goals and milestones for reducing the number of pending appeals under the legacy process. Page 12 GAO-18-349T VA Disability Benefits the metrics the agency will develop, how it will assess if the new appeals process is an improvement over the legacy appeals process, and how it will monitor the allocation of resources between legacy and new appeals claims. More specifically: • VA’s reported timeliness measures are incomplete: VA’s appeals plan outlines timeliness goals for the two VBA options (average processing time of 125 days) and for the Board option that does not include new evidence or a hearing (average processing time of 365 days). 17 However, VA’s plan does not establish timeliness goals for the other two Board options: Board review of additional evidence without a hearing and Board review of additional evidence with a hearing. In commenting on our assessment, while VA officials indicated they expect the new process to be more efficient than the legacy process (and, therefore, more timely), data to inform goal setting for all Board options will not be available until VA fully implements these options. However, establishing timeliness goals for all options would provide a more complete picture of VA’s vision for the new appeals process, and help VA to develop concrete, objective, and observable performance measures to show progress in achieving that vision, as well as inform resource estimates. • VA’s reported measures lack adequate balance: Other than including certain timeliness goals, VA’s appeals plan does not articulate additional aspects of performance important for managing appeals, such as accuracy of decisions, veteran satisfaction with the process, or cost. We previously reported that VA officials said that they wanted to also use veteran survey results, wait times, and inventories as sources of information to measure progress under the new appeals process. 18 Further, VA’s fiscal year 2018 annual performance plan includes an overall customer satisfaction score for veterans’ benefits. 19 However, these and other potential measures of success are not specified in VA’s appeals plan for monitoring the new 17 In supporting documentation, VA indicated that it will measure average veteran wait times in the new appeals system as one indicator of success. For example, an average processing time of 365 days will be the timeliness goal for the option in which the Board reviews the existing record without a hearing. VA indicated it would develop measures and determine appropriate resource allocation for the other options in the future. 18 GAO-17-234. 19 Specifically, VA’s Voice of the Veteran Customer Satisfaction Survey tracks veterans’ and their beneficiaries’ overall satisfaction on a 1,000-point scale. Page 13 GAO-18-349T VA Disability Benefits appeals process as compared with legacy appeals. By not articulating a set of comprehensive and balanced goals and measures in its appeals plan, VA could be inadvertently creating skewed incentives by focusing on one area of program performance to the detriment of other areas (e.g., processing claims quickly but inaccurately). In commenting on our assessment, VA officials recognized the need to develop additional goals and measures and indicated, for example, that they are developing and testing whether the existing quality assurance goal—requiring 92 percent accuracy—is appropriate for the new process. According to VA officials, once they have developed these other goals and measures, VA will communicate this information as part of the required progress reports to the appropriate committees of Congress and GAO. • VA’s plan does not reflect how it will establish baseline data: VA’s approach for evaluating the efficiency and effectiveness of the implementation of the new appeals process falls short of sound practices for using baseline data to assess performance. Our prior work has demonstrated that by tracking and developing a performance baseline for all measures, including those that demonstrate the effectiveness of a program, agencies can better evaluate progress made and whether or not goals are being achieved. 20 However, VA’s appeals plan did not provide important details about what aspects of the new appeals process’ performance will be compared to what aspects of the legacy process’ performance. In particular, section 5 of the Act lists a number of metrics VA is required to report periodically, including some that could be used as baseline measures. 21 For example, VA is required to periodically publish on its website the average time that elapsed between the filing of an initial claim and the final resolution of the claim, for legacy appeals as well as appeals under the new system, which is consistent 20 Baseline and trend data can also be used to inform and refine performance goals, according to our prior work. See GAO, Agency Performance Plans: Examples of Practices That Can Improve Usefulness to Decisionmakers, GAO/GGD/AIMD 99 69 (Washington, D.C.: Feb. 26, 1999). 21 Pub. L. No. 115-55, § 5, 131 Stat. 1105, 1123 (2017). Page 14 GAO-18-349T VA Disability Benefits with our prior recommendation. 22 However, VA’s appeals plan does not explain how or when the agency would collect and use these or other data about the legacy and new processes’ performance—such as accuracy, veteran satisfaction, and cost—to assess their relative performance. As we had previously reported, VA’s business case for reform in some instances relied on unproven assumptions and limited analyses of its legacy process to identify root causes of performance problems. 23 Specifically, VA determined that the open-ended nature of its legacy appeals process, whereby a veteran can submit additional evidence numerous times at any point during the VA appeals process, can cause additional cycles of re-adjudication, a process VA refers to as “churning.” According to VA, this re-adjudication can occur multiple times and can add years to the time needed to reach a final decision on an appeal. Without fully articulating a plan for collecting and using baseline and trend data, VA cannot determine the extent to which the new appeals process, which also allows for multiple appeal opportunities, will achieve final resolution of veterans’ appeals sooner, on average, than the legacy process. In commenting on our assessment, VA indicated that it is working toward capturing the metrics listed in section 5 of the Act. VA officials also noted that reporting on the new appeals process will require IT system functionality that currently does not exist, but stated that efforts are underway to add this functionality. • VA’s plan does not explain how the agency will monitor processing of legacy versus new appeals: In addition, VA’s appeals plan does not fully articulate how the agency will monitor whether resources are being appropriately devoted to both the new and legacy appeals process and how it will track both sets of 22 We previously recommended that VA develop a strategy for assessing process reform— relative to the current process—that ensures transparency on the extent to which VA is improving veterans’ experiences with its disability appeals process. This includes measuring the average time to reach final resolution of veterans’ appeals and reflects the fact that veterans may file multiple appeals under the new appeals process before achieving resolution of their claims. GAO-17-234. 23 See GAO-17-234. Page 15 GAO-18-349T VA Disability Benefits workloads. 24 An appeals plan that does not specifically articulate how VA will manage the two processes in parallel exposes the agency to risk that veterans with appeals in the legacy process may experience significant delays or otherwise poor results relative to those in the new appeals process or vice versa. In commenting on our assessment, VA officials noted that VA was not required under section 3 of the Act to provide a description of its plans to capture metrics listed in section 5. Even if not required by the Act, developing an approach for carefully monitoring the management of new and legacy appeals would help VA track progress being made and achievement of goals. Until VA establishes complete and balanced goals and measures, identifies baseline data, and develops a plan for monitoring and assessing both the new and legacy processes, VA runs the risk of promoting skewed behaviors, or not fully understanding whether the new process is an improvement or whether veterans with appeals in the legacy process are experiencing poor results. VA’s Appeals Plan Needs VA’s appeals plan reflects certain aspects of sound planning practices a Reliable Implementation related to managing the implementation of process change; however, other key components are not addressed. Sound planning practices for Schedule to Manage the implementing process change suggest establishing a transition team. 25 Project Consistent with such practices, VA’s appeals plan states that the agency convened an agency-wide governance structure to coordinate implementation of its new appeals process; it is comprised of senior-level employees with authority to make necessary decisions to keep the project on track. VA’s appeals plan also includes a copy of a master schedule. In its plan, VA asserts that the master schedule reflects timelines, interim goals and milestones, reporting requirements, and established deadlines, and that it will be used to guide implementation. VA’s appeals plan also reports that VA is consulting with project management professionals, who are using the master schedule, among other tools, to monitor implementation. In addition, VA made progress addressing some of the issues we previously identified by developing steps and timetables for 24 As noted earlier, we also determined that VA’s appeals plan does not contain sufficient details about metrics and goals related to identifying potential issues related to implementing the new appeals system, and therefore only partially addresses the required element at section 3(a)(4) of the Act. 25 GAO, Business Process Reengineering Assessment Guide—Version 3, GAO/AIMD-10.1.15 (Washington, D.C. May 1997). In this testimony, we refer to reengineering as appeals process “redesign.” Page 16 GAO-18-349T VA Disability Benefits updating training in anticipation of implementing the new appeals process. 26 However, VA’s master schedule for implementing reform is missing elements of a high-quality and reliable implementation schedule for key activities. We have previously reported that having a well-planned schedule is a fundamental management tool. Generally recognized sound practices from the Project Management Institute (PMI) and GAO call for organizations to employ an integrated and reliable master schedule that defines when work activities will occur, who will complete the work, how long they will take, how they are related to one another, and the constraints affecting the start and completion of work elements, as well as whether resources will be available when they are needed. 27 Such a project management schedule not only provides a road map for systematic project execution, but also provides the means by which to gauge progress, identify and address potential problems, and promote accountability. The master schedule VA provided in its appeals plan should have included other sound practices for project management related to a reliable schedule. Specifically: • Key activities and their duration are not included: VA’s master schedule does not capture the Rapid Appeals Modernization Program (RAMP) activities, even though this pilot test is occurring at the same time VA is preparing for full implementation of appeals options at VBA and the Board. In addition, specific Board-related activities are missing from the schedule, such as efforts to develop metrics, and the schedule and other project plans we reviewed do not go beyond February 2019. For example, the schedule does not indicate the period of time when VA expects to no longer be processing legacy appeals. 28 When all key and necessary activities are not included, it raises questions about whether all activities are scheduled in the 26 See GAO-17-234. 27 Project Management Institute, Inc., A Guide to the Project Management Body of Knowledge (PMBOK Guide); GAO, GAO Schedule Assessment Guide: Best Practices for Project Schedules, GAO-16-89G (Washington, D.C.: December 2015); and, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009). 28 Sound practices suggest that the schedule should reflect the duration of each activity and have specific start and end dates. Page 17 GAO-18-349T VA Disability Benefits correct order, resources are properly allocated, or the estimated completion dates are reliable. 29 In addition, if the schedule does not fully and accurately reflect VA’s efforts, it will not serve as an appropriate basis for analysis and may result in unreliable completion dates and delays. • Sequencing and linkages among activities are not identified: For the high-level activities VA’s appeals plan identifies, VA’s master schedule does not indicate whether there were linkages or sequencing among them, which is not consistent with sound scheduling practices. Linkages and sequencing would show, for example, if any of these activities or sub-activities must finish prior to the start of other activities, or the amount of time an activity could be delayed before the delay affects VA’s estimated implementation date. 30 For example, VA cannot train new employees until after it hires them. The activities VA identifies also do not appear supported by lower- level project schedules. Specifically, when we requested documentation to support VA’s high-level summary of activities and milestones, VA officials did not provide intermediate or more detailed schedules that reflected these practices. In particular, VA’s appeals plan lacks a complete schedule for IT modifications that clearly defines what is to be achieved and the time frames for achievement. 31 We previously recommended that VA develop a schedule for IT updates that explicitly addresses when and how process reform will 29 In addition, if some necessary activities are missing from an agency’s implementation schedule, it is much more difficult for the agency to adhere to other best practices. Our prior work has identified 10 practices associated with developing and maintaining a reliable schedule. These practices are (1) capturing all activities, (2) sequencing all activities, (3) assigning resources to all activities, (4) establishing the duration of all activities, (5) integrating schedule activities horizontally and vertically, (6) establishing the critical path for activities, (7) identifying float between activities, (8) conducting a schedule risk analysis, and (9) updating the schedule using actual progress and logic, and (10) maintaining a baseline schedule. Float is the amount of time an activity can slip before affecting the critical path. The critical path is the longest path through the schedule. If an activity on the critical path slips, the entire project will be delayed. See GAO-16-89G. 30 GAO-16-89G. 31 Federal internal control standards state that program managers should define objectives clearly to enable the identification of risks that may impede the achievement of program objectives. This includes clearly defining what is to be achieved and the time frames for achievement. See GAO-14-704G. Additionally, IT investment best practices stress the need for oversight regarding a project’s progress toward predefined schedule expectations. See GAO-04-394G. Page 18 GAO-18-349T VA Disability Benefits be integrated into new systems and when these systems will be ready to support the new appeals process at its onset. 32 For example, VA’s appeals plan references several required IT modifications that do not appear in its master schedule. 33 Schedules that are defined at too high a level may disguise risk that is inherent in lower-level activities. 34 • Interim goals are not reflected: VA officials stated that they have interim goals and milestones, though VA’s appeals plan and supporting documentation generally do not include this information. Sound planning and redesign practices suggest closely monitoring implementation and developing project goals that include a mix of intermediate goals to be met at various stages. VA’s appeals plan does not include this information. We previously made a recommendation that VA develop a more robust plan for closely monitoring implementation of process reform, including metrics and interim goals to help track progress, evaluate efficiency and effectiveness, and identify trouble spots—all of which are consistent with sound planning practices. 35 • Resources are not assigned to all identified activities: The high- level summary schedule that VA provided us also lacks details regarding the assignment of resources for all activities. Specifically, while the plan identifies workgroups responsible for coordinating elements in the plan, such as regulations, training, and outreach, the schedule does not assign resources to the 40 listed activities. As discussed previously, VA’s appeals plan also does not provide information on the total resources required for this reform effort. Assigning resources to the listed activities, as well as providing other information, could provide a better indication of the estimated total 32 GAO-17-234. 33 VA’s master schedule lists three IT requirements; however, VA’s appeals plan references several other requirements not included in the master schedule: eFolder; Caseflow Work Queue, Caseflow Hearings Scheduling, and Reporting Functionality. In supplementary materials, VA provided an IT “road map” that lists nine IT requirements; however, several of these requirements (e.g., Caseflow Dispatch) do not appear in either VA’s appeals plan or master schedule. In addition the “road map” reflects 6 months of planning and does not extend to or beyond the end dates reflected in VA’s master schedule. 34 GAO-16-89G. 35 GAO-17-234. Page 19 GAO-18-349T VA Disability Benefits resources required to implement the new appeals process and address legacy appeals. In commenting on our assessment, VA officials stated that the agency is developing lower-level project schedules for key activities—such as RAMP and IT requirements—and will provide these schedules as part of the required progress reports to the appropriate committees of Congress and GAO. VA officials also noted that future updates will include additional dependencies and risks, which VBA and the Board are still developing. Until VA has a robust integrated master schedule, supported by detailed project plans that adhere to sound practices, VA’s appeals plan does not provide reasonable assurance that decision makers have the essential program management information needed for this complex and important effort. VA’s Plan Addresses VA’s appeals plan includes an assessment of risks involved in Some but Not All Key implementing the new appeals system, but could more comprehensively reflect key risks posed by such a significant reform effort. 36 VA’s appeals Risks Related to the New plan and supplementary materials include a “risk register” that describes Appeals Process risks associated with many elements of its plan and the remaining level of risk after its planned response to these risks. VA’s appeals plan also states that senior leaders will receive regular updates of risks and mitigation strategies. However, because VA has not yet articulated a balanced set of performance goals and measures in its appeals plan, it is hindered in its ability to identify and assess risks. Federal internal control standards state, and our previous work at VA and other agencies demonstrates, that establishing clear performance goals and objectives is a necessary pre-condition to effectively assessing risk. 37 Having, for example, more complete timeliness goals, and goals and 36 A risk assessment is the identification and analysis of risks related to achieving the defined objectives. This assessment provides the basis for developing appropriate risk responses. GAO, Standards for Internal Control in the Federal Government, GAO-14-704G (Washington, D.C.: September 2014). 37 GAO-14-704G. See the following reports for examples of agencies that did not have measureable performance goals to assess and mitigate risks: GAO, Veterans Justice Outreach Program: VA Could Improve Management by Establishing Performance Measures and Fully Assessing Risks, GAO-16-393 (Washington, D.C.: Apr. 28, 2016; and GAO, Telecommunications Relay Service: FCC Should Strengthen Its Management of Program to Assist Persons with Hearing or Speech Disabilities, GAO-15-409 (Washington, D.C.: Apr. 29, 2015). Page 20 GAO-18-349T VA Disability Benefits measures reflecting other areas of performance, would allow VA to better identify and target risks associated with managing two processes in parallel, including the potential that veterans with appeals in the legacy process may experience significant delays relative to those in the new appeals process. Importantly, VA is missing an opportunity to fully benefit from RAMP by not testing and assessing other aspects of the new appeals process. The Act authorizes VA to test the feasibility and advisability of any facet of the new appeals process, and VA is taking a positive step to mitigate some risks by testing the two review options available within VBA (review of a claim by a higher-level official based on the same evidence and review of a supplemental claim with additional evidence) through RAMP. In November 2017, VA began RAMP by inviting 500 veterans whose appeals have been pending the longest to participate. According to VA officials, each month VA plans to continue offering RAMP to additional eligible veterans with pending legacy appeals until January 2019—a month before VA anticipates fully implementing the new appeals system. However, as designed, RAMP does not include features that—consistent with a well-developed and documented pilot test program—would provide VA with an opportunity to evaluate fully the soundness of new processes and practices on a smaller scale. 38 Specifically: • VA’s plan does not clearly define success criteria for RAMP: VA’s appeals plan states that the agency will collect certain data from RAMP, such as the rate at which eligible veterans opt into the process, timeliness of claims processing, and individual employee productivity. VA also established an overall average processing time goal of 125 days for the two VBA options; however, the plan and supporting documentation do not clearly articulate whether RAMP reviews are expected to meet this timeliness goal. The plan also did not identify other success criteria for RAMP or the types of results expected before fully implementing the new appeals process. For example, VA’s plan does not articulate the expected number and type of subsequent appeals to the Board that result from RAMP. In commenting on this assessment, VA noted that its intent in implementing RAMP was to collect data and test aspects of the new 38 GAO, Business Process Reengineering Assessment Guide, GAO/AIMD-10.1.15 (Washington, D.C.: May 1997) and GAO, Data Act: Section 5 Pilot Design Issues Need to Be Addressed to Meet Goal of Reducing Recipient Reporting Burden, GAO-16-438 (Washington, D.C.: Apr. 19, 2016). Page 21 GAO-18-349T VA Disability Benefits process, and that RAMP was not an initiative in and of itself. However, developing performance measures and data gathering procedures and defining success criteria for a pilot test before proceeding to full implementation are sound practices for process redesign and pilot testing. In addition, because RAMP was not included in VA’s risk assessment, we asked VA if it had identified any risks or mitigation strategies specific to RAMP. In its supplemental materials, VA stated that the greatest risk to RAMP is a low participation rate among eligible veterans with legacy claims. VA also indicated that it would need 10 percent of eligible veterans to opt into RAMP to yield meaningful results. However, this threshold is not articulated in VA’s appeals plan as an explicit success criterion or objective. According to data provided by VA, as of January 22, 2018, 238 veterans opted in. 39 Of veterans with pending claims in RAMP, two-thirds chose the higher-level review option. VA also reported that 47 RAMP decisions have been made so far. As of yet, no appeals of RAMP decisions have been filed. • VA’s plan does not articulate how it will assess RAMP before proceeding with full implementation: Although VA’s appeals plan describes a “close-out” phase in which VA intends to assess the results of RAMP, it does not detail the conditions that would have to be met (or not met) to trigger changes. For example, VA’s plan does not explain when or how it might respond to low opt-in rates for RAMP—other than stating it will increase outreach to eligible veterans—or to unexpectedly high appeal rates to the Board resulting from RAMP decisions. Sound redesign and change management practices both suggest that pilot tests be rigorously monitored and evaluated, and that further roll-out occur only after an agency’s transition team takes any needed corrective action and determines that the new process is achieving previously identified success criteria. Without fully articulating its plan for deciding how and when to roll out changes more broadly, it is not clear whether VA would be prepared to fully implement a new appeals process that achieves its aim of better serving veterans. • RAMP does not test all aspects of the new appeals process: RAMP provides an opportunity to learn about experiences at VBA under the new system, such as the rate at which eligible veterans 39 According to VA, as of January 22, 2018, RAMP invitations were sent to 15,500 veterans via three separate mailings—500 in November 2017, 5,000 in December 2017, and 10,000 in January 2018. As of January 22, 2018, VA reported that the opt-in rate was about 3 percent for the November and December mailing, and 1 percent for January. Page 22 GAO-18-349T VA Disability Benefits choose those options and the resources that will be required to process their appeals. However, RAMP was not designed to test how many veterans would choose to appeal directly to the Board and, therefore, it will not provide comparable information on the Board appeals options. Sound workforce planning practices suggest that agencies identify the total resources needed to manage the risk of implementing new processes and conduct scenario planning to determine those needs. 40 In addition, although we previously recommended VA conduct additional sensitivity analyses to inform projections of future appeals inventories, 41 VA’s appeals plan does not reflect VA’s use or intended use of sensitivity analyses when projecting staffing needs for new appeals options at the Board. 42 In commenting on our assessment, VA officials said they do not plan to conduct additional sensitivity analyses to project future workloads until they have more information from RAMP to inform their assumptions. As a result, VA will lack data on scenarios in which veterans may overwhelmingly choose options available at the Board over those at VBA when the appeals plan is fully implemented. This presents a risk that VA’s early production projections and initial resource allocations may not be properly balanced between the Board and VBA. This, in turn, may result in an unexpectedly large number of appeals pending with the Board, and corresponding lengthy average wait and decision times for some, if not all, Board options. Having information on the number of veterans who are likely to appeal to the Board is particularly critical, given that similar efforts to create additional review options at VBA did not achieve their goals of reducing the percentage of appeals that continue on to the Board. In 2001, VA established the Decision Review Officer (DRO) process—in which senior staff have the authority to overturn an initial disability claim decision without any new evidence—to resolve more appeals at 40 GAO-09-3SP. 41 GAO-17-234. Sensitivity analysis—used in scenario planning to, for example, determine the resources needed for implementing a new process—is an analysis to determine how sensitive outcomes are to changes in assumptions, such as those used to determine resource needs. The assumptions that deserve the most attention should depend on the dominant benefit and cost elements and the areas of greatest uncertainty of the program or process being analyzed. See GAO-09-3SP. 42 As noted previously, we determined that VA’s plan does not estimate personnel required by either VBA or the Board for the period when they are not processing legacy appeals. Sensitivity analyses could aid VA in forecasting such resources. Page 23 GAO-18-349T VA Disability Benefits the regional level and avoid long waits at the Board. However, we reported in 2011 that, although the DRO process helped some veterans get additional benefits at the regional office level, it did not accomplish the program’s primary goal of reducing the percentage of appeals continuing on to the Board. 43 In responding to our assessment, VA officials reiterated their plans to increase outreach in the event of low opt-in rates for RAMP and indicated they recently began to send follow-up RAMP invitation letters. With respect to assessing all appeal options, VA officials stated that, while no legal bar prevents testing of the Board options, the Board is focused on reducing its inventory of pending appeals while RAMP provides early resolution of appeals within the new VBA-only options. Officials conceded that this approach means they cannot collect data on the rate at which veterans opt to appeal directly to the Board (e.g., bypassing additional VBA review) until the new process is fully implemented. However, they noted that they can collect some data on the rate at which veterans whose appeals go through RAMP file subsequent appeals to the Board, even though the Board will not begin processing those appeals until full implementation. By pursuing an approach that does not identify or mitigate significant risks associated with implementing a new process, VA is taking a chance that untested aspects will not perform as desired. The Act provides VA authority to pilot aspects of the process and flexibility on the timing of implementing the new process, which could allow some additional time for VA to carefully measure performance under RAMP and determine whether any corrective actions are necessary. If VA does not take full advantage of this authority, it risks moving forward without knowing whether the new appeals process improves experiences for veterans, and potentially implementing a process that is more expensive or results in longer wait times than originally anticipated. In conclusion, in implementing appeals reform after the enactment of the Veterans Appeals Improvement and Modernization Act of 2017, VA is undertaking a complex endeavor that has the potential to affect the lives of hundreds of thousands of veterans with service-connected disabilities. 43 GAO, Veterans Disability Benefits: Clearer Information for Veterans and Additional Performance Measures Could Improve Appeal Process, GAO-11-812 (Washington, D.C.: Sept. 29, 2011). Page 24 GAO-18-349T VA Disability Benefits Such an endeavor demands a commensurate level of planning to be successful. While the Act required VA to submit its plan within 90 days of enactment, VA had proposed and began to plan for appeals reform much earlier, and had our March 2017 recommendations to guide its planning efforts from a foundation of sound practices. VA’s November 2017 appeals plan is a positive step forward. Certain elements of the plan—such as establishing an agency-wide governance structure to oversee implementation and testing aspects of reform prior to full implementation—are notable gains since our March 2017 report. At the same time, the plan partially addresses or does not address five of the required elements called for by the Act, such as delineating the total resources required by VBA and the Board to implement and administer the new appeals process and address legacy appeals. The plan also is not fully responsive to our past recommendations and does not reflect a number of sound planning practices that are essential for gauging progress, establishing accountability, and linking resources to results. One such key practice is articulating a desired “end state”—a vision for what successful implementation would look like for the new appeals process as well as the wind-down of the legacy process, such as accurate and timely processing of appeals while ensuring veteran satisfaction. Without establishing a complete and balanced set of goals and related performance measures to achieve this end state and monitoring and assessing progress along the way, VA risks falling short of its overarching objective—to improve timeliness of appeals decisions for veterans overall. By not fully articulating how it plans to monitor workloads and devote resources to both the new and legacy processes, VA runs the risk of disadvantaging veterans with legacy appeals relative to those in the new process, or vice versa. Just as important is establishing a robust integrated master schedule— rather than a high-level timeline—that is built upon and clearly reflects extensive detailed planning and includes all of the activities necessary to execute the program and interdependencies between these activities. Without such a road map, VA’s appeals plan does not provide reasonable assurance that decision makers have the essential information needed to manage this complex and important program. We are encouraged that VA has taken some steps toward assessing risks, including establishing a risk register and implementing RAMP to collect information on the two VBA appeals options; however, unless VA assesses risks against a balanced set of goals and measures, VA may Page 25 GAO-18-349T VA Disability Benefits not be fully aware of risks that may impede successful implementation of appeals reform. Further, although VA will undoubtedly learn from the RAMP experience, it may not learn all that it should from its efforts without (1) establishing clear criteria for what success looks like (or the circumstances that would cause VA to consider making course corrections) and (2) building in time to take stock of the lessons learned before moving to full implementation. VA’s plan places a lot of weight on RAMP to, among other efforts, mitigate risk and generate estimates of the resources needed for successful implementation after fiscal year 2018, even though RAMP does not fully test options for appealing to the Board that will be available to veterans after full implementation. Unless VA addresses key risks associated with fully implementing appeals reform—by either testing or conducting sensitivity analyses for all five appeals options, to better understand potential workloads at the Board—VA runs the risk of fully implementing the process without knowing if it is improving the process for veterans. In our forthcoming report, we anticipate making recommendations to address these issues. Specifically, we are preliminarily considering recommending that the Secretary of Veterans Affairs: • address all of the required elements in the Act in VA’s appeals plan to Congress—including delineating resources required for all VBA and Board appeals options—using sensitivity analyses and RAMP results, where appropriate and needed. • clearly articulate in VA’s appeals plan how VA will monitor and assess the new appeals process compared to the legacy process, including specifying a balanced set of goals and measures—such as timeliness goals for all VBA appeals options and Board dockets, and measures of accuracy, veteran satisfaction, and cost—and related baseline data. • augment the master schedule for VA’s appeals plan to reflect all activities—such as RAMP and modifications to IT systems—as well as assigned responsibilities, interdependencies, start and end dates for key activities for each workgroup, and resources, to establish accountability and reduce overall risk of implementation failures. • ensure that the appeals plan more fully addresses risk associated with appeals reform—for example, by assessing risks against a balanced set of goals and measures, articulating success criteria and an assessment plan for RAMP, and testing or conducting sensitivity Page 26 GAO-18-349T VA Disability Benefits analyses of all appeal options—prior to fully implementing the new appeals process. Chairman Roe, Ranking Member Walz, and Members of the Committee, this concludes my prepared statement. I would be pleased to respond to any questions that you may have at this time. For further information about this testimony, please contact Elizabeth GAO Contact and Curda at (202) 512-7215 or curdae@gao.gov. Contact points for our Staff Offices of Congressional Relations and Public Affairs may be found on the last page of this testimony. Other key contributors to this testimony Acknowledgments include Michele Grgich (Assistant Director), James Whitcomb (Analyst in Charge), and Rachael Chamberlin. In addition, key support was provided by Susan Aschoff, Mark Bird, David Chrisinger, Daniel Concepcion, Clifton Douglas, Alex Galuten, Nisha Hazra, Melissa Jaynes, Benjamin Licht, Patricia McClure, Sheila McCoy, Lorin Obler, Gloria Proa, Almeta Spencer, James Sweetman, Walter Vance, and Greg Whitney. Page 27 GAO-18-349T VA Disability Benefits Appendix I: Our Assessment of VA’s Appeals Appendix I: Our Assessment of VA’s Appeals Plan Against Required Elements in the Act Plan Against Required Elements in the Act To assess the extent to which VA’s appeals plan addresses the required elements in the Veterans Appeals Improvement and Modernization Act of 2017 (the Act), we first identified and developed a checklist reflecting each required element for VA’s appeals plan (including sub-parts) under section 3(a) and (b) of the Act. To compare the required elements and their sub-parts against VA’s appeals plan and supplemental materials provided, we developed decision rules for determining whether the VA’s appeals plan addressed, partially addressed, or did not address each required element. Specifically, we concluded that VA’s plan addressed (or partially addressed) a required element if the plan included information related to all (or some) subparts of the requirement. We focused on the plan as presented, rather than auditing the information VA relied on in developing the plan. For example, the Act’s section 3(b)(10) required VA’s plan to include a description of the modifications to the IT systems that VBA and the Board require to carry out the new appeals system, including cost estimates and a timeline for making the IT modifications. We concluded that VA’s plan addressed all sub-parts of this element because it provided a description of required IT modifications, a reference to costs included in the Appeals Modernization IT budget, and a timeline. However, our determination that VA addressed this element should not be construed to necessarily mean that VA fully identified or described all IT requirements, or provided complete estimated costs and timelines associated with those requirements, or that the information in VA’s appeals plan comported with sound planning practices. This type of assessment was outside the scope of this objective. Table 2 summarizes our assessment of VA’s appeals plan against the 22 required elements in the Act. Page 28 GAO-18-349T VA Disability Benefits Appendix I: Our Assessment of VA’s Appeals Plan Against Required Elements in the Act Table 2: Summary of GAO’s Assessment of VA’s Appeals Plan Against the Required Elements in the Veterans Appeals Improvement and Modernization Act of 2017 (Act) Required elements of plan (from sec. 3(a) and (b) of the Act) Summary of GAO’s assessment Section 3(a)(1) Addressed [A comprehensive plan for] the processing of appeals of decisions on legacy claims that the Secretary considers pending. Section 3(a)(2) Addressed [A comprehensive plan for] implementing the new appeals system. Section 3(a)(3) Addressed [A comprehensive plan for] timely processing, under the new appeals system, of— (A) supplemental claims under section 5108 of title 38, United States Code, as amended by section 2(i); (B) requests for higher-level review under section 5104B of such title, as added by section 2(g); and (C) appeals on any docket maintained under section 7107 of such title, as amended by section 2(t). Section 3(a)(4) Partially addressed [A comprehensive plan for] monitoring the implementation of the new appeals system, including metrics and goals— (A) to track the progress of implementation; (B) to evaluate the efficiency and effectiveness of the implementation; and (C) to identify potential issues relating to the implementation. Section 3(b)(1) Not addressed Delineation of the total resource requirements of the Veterans Benefits Administration and the Board of Veterans’ Appeals, disaggregated by resources required to implement and administer the new appeals system and resources required to address the appeals of decisions on legacy claims. Section 3(b)(2) Partially addressed Delineation of the personnel requirements of the Administration and the Board, including staffing levels during the— (A) period in which the Administration and the Board are concurrently processing— (i) appeals of decisions on legacy claims; and (ii) appeals of decisions on non-legacy claims under the new appeals system; and (B) the period during which the Administration and the Board are no longer processing any appeals of decisions on legacy claims. Page 29 GAO-18-349T VA Disability Benefits Appendix I: Our Assessment of VA’s Appeals Plan Against Required Elements in the Act Required elements of plan (from sec. 3(a) and (b) of the Act) Summary of GAO’s assessment Section 3(b)(3) Addressed Identification of the legal authorities under which the Administration or the Board may— (A) hire additional employees to conduct the concurrent processing described in paragraph (2)(A); and (B) remove employees who are no longer required by the Administration or the Board once the Administration and the Board are no longer processing any appeals of decisions on legacy claims. Section 3(b)(4) Addressed An estimate of the amount of time the Administration and the Board will require to hire additional employees as described in paragraph (3)(A) once funding has been made available for such purpose, including a comparison of such estimate and the historical average time required by the Administration and the Board to hire additional employees. Section 3(b)(5) Addressed A description of the amount of training and experience that will be required of individuals conducting higher-level reviews under section 5104B of title 38, United States Code, as added by section 2(g). Section 3(b)(6) Addressed An estimate of the percentage of higher-level adjudicators who will be employees of the Department of Veterans Affairs who were Decision Review Officers on the day before the new appeals system takes effect or had experience, as of such date, comparable to that of one who was a Decision Review Officer. Section 3(b)(7) Addressed A description of the functions that will be performed after the date on which the new appeals system takes effect by Decision Review Officers who were Decision Review Officers on the day before the date the new appeals system takes effect. Section 3(b)(8) Addressed Identification of and a timeline for— (A) any training that may be required as a result of hiring new employees to carry out the new appeals system or to process appeals of decisions on legacy claims; and (B) any retraining of existing employees that may be required to carry out such system or to process such claims. Section 3(b)(9) Addressed Identification of the costs to the Department of Veterans Affairs of the training identified under paragraph (8) and any additional training staff and any additional training facilities that will be required to provide such training. Section 3(b)(10) Addressed A description of the modifications to the information technology systems of the Administration and the Board that the Administration and the Board require to carry out the new appeals system, including cost estimates and a timeline for making the modifications. Page 30 GAO-18-349T VA Disability Benefits Appendix I: Our Assessment of VA’s Appeals Plan Against Required Elements in the Act Required elements of plan (from sec. 3(a) and (b) of the Act) Summary of GAO’s assessment Section 3(b)(11) Addressed An estimate of the office space the Administration and the Board will require during each of the periods described in paragraph (2), including— (A) an estimate of the amount of time the Administration and the Board will require to acquire any additional office space to carry out processing of appeals of decisions on legacy claims and processing of appeals under the new appeals system; (B) a comparison of the estimate under subparagraph (A) and the historical average time required by the Administration and the Board to acquire new office space; and (C) a plan for using telework to accommodate staff exceeding available office space, including how the Administration and the Board will provide training and oversight with respect to such teleworking. Section 3(b)(12) Partially addressed Projections for the productivity of individual employees at the Administration and the Board in carrying out tasks relating to the processing of appeals of decisions on legacy claims and appeals under the new appeals system, taking into account the experience level of new employees and the enhanced notice requirements under section 5104(b) of title 38, United States Code, as amended by section 2(e). Section 3(b)(13) Addressed An outline of the outreach the Secretary expects to conduct to inform veterans, families of veterans, survivors of veterans, veterans service organizations, military service organizations, congressional caseworkers, advocates for veterans, and such other stakeholders as the Secretary considers appropriate about the new appeals system, including— (A) a description of the resources required to conduct such outreach; and (B) timelines for conducting such outreach. Section 3(b)(14) Addressed Timelines for updating any policy guidance, Internet websites, and official forms that may be necessary to carry out the new appeals system, including— (A) identification of which offices and entities will be involved in efforts relating to such updating; and (B) historical information about how long similar update efforts have taken. Section 3(b)(15) Addressed A timeline, including interim milestones, for promulgating such regulations as may be necessary to carry out the new appeals system and a comparison with historical averages for time required to promulgate regulations of similar complexity and scope. Section 3(b)(16) Addressed An outline of the circumstances under which claimants with pending appeals of decisions on legacy claims would be authorized to have their appeals reviewed under the new appeals system. Page 31 GAO-18-349T VA Disability Benefits Appendix I: Our Assessment of VA’s Appeals Plan Against Required Elements in the Act Required elements of plan (from sec. 3(a) and (b) of the Act) Summary of GAO’s assessment Section 3(b)(17) Partially addressed A delineation of the key goals and milestones for reducing the number of pending appeals that are not processed under the new appeals system, including the expected number of appeals, remands, and hearing requests at the Administration and the Board each year, beginning with the one year period beginning on the date of the enactment of this Act, until there are no longer any appeals pending before the Administration or the Board for a decision on a legacy claim. Section 3(b)(18) Addressed A description of each risk factor associated with each element of the plan and a contingency plan to minimize each such risk. Source: GAO analysis of VA’s November 2017 appeals plan and any supporting documents VA provided on or before January 18, 2018, the Veterans Appeals Improvement and Modernization Act of 2017, and agency input by VA. | GAO-18-349T (102558) Page 32 GAO-18-349T VA Disability Benefits This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. The Government Accountability Office, the audit, evaluation, and investigative GAO’s Mission arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no cost is Obtaining Copies of through GAO’s website (https://www.gao.gov). Each weekday afternoon, GAO GAO Reports and posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted products, go to https://www.gao.gov Testimony and select “E-mail Updates.” Order by Phone The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, https://www.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO on Facebook, Flickr, Twitter, and YouTube. Connect with GAO Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. Visit GAO on the web at https://www.gao.gov. Contact: To Report Fraud, Website: https://www.gao.gov/fraudnet/fraudnet.htm Waste, and Abuse in E-mail: fraudnet@gao.gov Federal Programs Automated answering system: (800) 424-5454 or (202) 512-7470 Orice Williams Brown, Managing Director, WilliamsO@gao.gov, (202) 512-4400, Congressional U.S. Government Accountability Office, 441 G Street NW, Room 7125, Relations Washington, DC 20548 Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800 Public Affairs U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548 James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707 Strategic Planning and U.S. Government Accountability Office, 441 G Street NW, Room 7814, External Liaison Washington, DC 20548 Please Print on Recycled Paper.