Cn MEMORANDUM TO MEMBERS OF RE VIEW COMMITTER Sub3: The Review Process The agenda for the Review Coumittee meeting of July 30 and 31 will include a discussion of the review process. This process has been a subject of study during my assignment to the Division for this year. The study has been directed toward the system of review which would reflect the unusual nature of Regional Medical Prograns and would. serve best their objectives. The study has focused on a process that would be appropriate fcr 1971, when RMP has becone firmly established and substantially funded, with gracual inplementation over the next two ears. Most of the recommended chan es would be incorporated in a k * a 1 proposed system of “anniversary review." As background for cur discussion, the following material is enclosed ’ 1. An abstract froma preliminary report proposing general changes in the review process, some of which have been instituted. (Exhibit I). 2. "The Transfer of Regions to Anniversary Review" - a background paper being used for discussion. (Exh pds Ebit td — 3. "Anniversary Review" - a detailed system and proposed steps to implementation. (Hxhibit ITi This subject is presented, to you, in line with discussioas in the April * meeting for thorough censidereton and advice in advance of its su 4 pede te the Natioual Advisory Counc Robart P. Lawton Special Assistent to the Director ON er a ee eg ae ce pi “ wee re ge ° 7 7 ete SxaiBir I MEMORANDUM To: Members of the Review Committee Frou: Special Assistant to the Director Subj: The Review Process - An abstract from a preliminary report In preparation for a discussion of the RMP review process at the Committee neeting on July 30, it seems appropriate to provide you with an abstract of the findings and recommendations from one of the interim reports that L have made to the Director. These findings reflect a gErong consensus from scores of persons I have interviewed. In addition to receiving advice from members of national review groups and Division staff, I have visited ‘sixteen regions and otherwise discussed the review process with the leaders of all of the regions in interregional meetings and coordinators seminars. 7 It is not news that the present review process is overloaded and too lengthy. It does not do justice to the Program or achieve the incisiveness required by limited funds. ‘vais is not criticism of the effort at the national level. The volume of material for review has just swamped the available troops and the system. There is much frustration and desire for change. It is unanimously felt that peer review is an essential element of the RMP review process and must be retained. However, peer review as practiced in individual research grants is grossly inadequate to a program as complex and multi-institutional as RMP. It must be expanded and supplemented : ote * eres Te es en = a -2- * extensively while being maintained as the essence of qualitative evaluation. Program peers, as contrasted with categorical project peers, are in growing demand but they are in short supply. The regional review processes whieh vary from sophistication and statesmenship to superficiality and irresponsibility. However, regional review is improving everywhere as the regions face the crunch of limited funding and priorities. More reliance on regional review is very important and desirable but it must become responsible and relatively uniform throughout the 55 regions. - The uncertainties of objectives, money, manpower and priorities have all caught RMP in second gear. If RMP is to fulfill its considerable promise to improve the health care system, the process by which it is evaluated, funded and otherwise nurtured and encouraged needs to be one of great quality and efficiency - and needs to become that very rapidly. PRELIMINARY RECOMMENDATIONS The preliminary recommendations come under four headings. A. REORGANIZATION OF THE DIVISION -- Proper staffing and organization | ‘of the Division are fundamental to a good review process. The Division has been undergoing organizational changes and the Director is planning other changes. I can only say that the Division is undermanned and overworked and express the hope that the absorption of new manpower can be accomplished by fall. CHANCES IN THE REVIEW CYCLE ~~ The combination of review at the regional and federal. levels frequently consumes a full'year from conception of a project to its funding. There is considerable overlap between regional and federal review and among some components of federal review. Therefore, the following changes are proposed: 1. © An anniversary system under which a Region and its program and projects have a high quality review once a year (every three years) culminating in Committee and Council action. Three federal review cycles with Committee and Council review one-third and action on gne-ninth)of the regions in each cycle. Shortening the time for project review by better filtration, sharper definition of roles and more regional autonomy (see change D). Better "filtration" prior to Review Committee action: (a) shorter and better applications. (b) More staff involvement in review. (c) Better field visits (see change C). (d) Panel reviews. Terminating the qualitative, scientific review with the Review Conmittee, reserving all policy decisions, including national priorities and choice among qualified projects, to the Advisory Council. (This has been largely accomplished.) D. SITE VISITS -- Site visits should become more comprehensive The staff assistance visit will and should grow to approach the importance of the site visit. Each should be: 1. Highly structured and well planned, based on experiment and experience -- as in Alabama and California. 2:--At multiple locations where program and projects are being applied throughout the region. 3, Basically constructive as well as judgmental. 4, More program oriented. 5, Authorized to communicate its findings to the Regions, with competence and discretion, in a way to expedite and improve _the Region's program without compromising higher review or approval authority. REGIONAL AUTONOMY AND RESPONSIBILITY -- . There can be quicker, greater impact, more efficiency and political gain from changes to increase regional autonomy and responsibility for programs and review. Proposed changes are: i. A uniform system of first caliber regional review, so highly responsible it can relieve the federal review process in some ways. A system of fiscal flexibility to allow regions to initiate new projects between federal reviews. Regional management, under good guidelines, of the basic support grant for support of core staff, continuing planning and possibly for some "ongoing" programs. ” | A system by which Regions may terminate unproductive projects and transfer funds, within limits, to other projects. -5- 5. A system of formal, written certification by the Regional Advisory Groups on certain essential matters, including: ‘(a) The qualifications, functions and authority of the Regional Advisory Group itself. (b) The regional pian. (c) The existence of formal, cooperative arrangements __ among institutions, (d) Coordination with other programs. (e) The impact or influence of the RMP on health care in the Region. 17/11/69 Robert P. Lawton - Sn ; . ‘ Draft TRANSFER OF REGIONS TO ANNIVERSARY REVIEW. There has been some initial discussion of putting Regional Medical Programs on a system of anniversary review. It is time to decide if we will effect this change but it poses two difficult problems for which 4 solutions are being sought. . The practice of having applications for grants for over thirty of the fifty-five regions in each quarterly review cycle must be changed for many obvious reasons. The study of the review process indicates that our long range goal should be a combination of high quality regional and federal review of each Region's program and projects on its anniversary each year or every two years. Placing some regions on anniversary review not only smooths and lessens the overwhelming workload of review but is a means of testing many recommended changes which have come out of the study of the review process. Division Staff has been studying the initiation of the anniversary ‘review system now with selected regions. It is proposed that we transfer at least five, and not more than ten, of carefully selected regions to a 4 system of annual review for the simultaneous funding of all operations within each regicn--i.e., core support, the continuation of appropriate central resources and operational projects. It is further proposed that we do this under a system which would allow the region to initiate or substitute new projects, which are a recognizable part of the approved regional plan, within the financial limits of the annual award. The ways in which a region could acquire fiscal flexibility between anniversaries, and the manner in which they qualify for and manage this gutouony, are extremely important. At the present time, the Division has the authority, delegated by the Council, to make awards in amounts totalling up to $50,000 annually for new projects which are clearly related to program and projects already approved by Council. This is potentially quick and efficient but falls far short of the amount of money needed to give a region minimum flexibility. There seem to be . two major ways of project funding between anniversary awards. One is to make a substantial award for "anticipated" new projects arising from _ ‘a plan presented at the anniversary. Another is to grant a region the right to reinvest the money recaptured from terminated projects. f The first of these, which could be called a "facilitating award", would be a fixed sum which would undergo the complete process of review w ‘and final approval by the Council. However, the facilitating award would not be tied to specific projects. It would be restructed to future projects which clearly relate to approved program and would be subject to audit against the region's overall plan. Expenditures for new projects out of the facilitating award would be subject to advance administrative approval by DRM. This would be essentially a formalized extension and enlargement of the delegated authority. The right, within limits, to reinvest the proceeds of terminated projects is an attractive means of fiscal flexibility. Unfortunately, most regions are not yet in “a position to terminate projects. This is not generally a satisfactory solution to the flexibility problem for those a Se 3 - . five to ten regions that are most eligible for anniversary review. Nevertheless it is not premature to draft tentative guidelines of ‘this - subject. | The chief hazard of placing regions on anniversary review is that ‘it. could react to their disadvantage, rather than advantage. Such regions would lose the several ways now open to apply for added funds. Consequently, the Division would have to take steps to prevent the other regions from being better funded than those on anniversary review. A second problem has to do with setting the level of the flexible portion of a region's grant, whether this is a facilitating award or ae ‘the right to reinvest money. Based on experience, eteanization and leadership, five regions are “ suggested for designation as "AR" regions, meaning eligible to apply for classification as fully operational and transfer to anniversary review. eintatirentneenr eM ey These are Georgia, North Carolina, Tennessee-Mid South, Washington-Alaska_ —eeencrncunaerter and Wisconsin.° reset sce ED There are differences among these five regions which should be noted in considering them for preferred status. They range in population size from 2.7 to 5.1 million and have current levels of annual support ranging -from 1.3 million to 2.5 million dollars. Per capita RMP investment ranges from .31 to .58 for total support. Per capita investment in core ranges from .10 to .33. Nevertheless, good program is well established in all of them and cualifies them for consideration in this new classification. Five others, California, Kansas, Michigan, Oregon: and Western New Yark are potential candidates for later in the year. ANNIVERSARY REVIEW THE MERITS OF ANNIVERSARY REVIEW -- There are a number of reasons why a system of anniversary review would be good for Regional Medical Programs, both on the federal and regional levels. 1 It gives recognition to regions which are the most effective by putting them-in a new category with a reward for good program design, excellent leadership and for a successful and prolonged implementation of their projects. The special designation, extension of core support for a minimum of three years and some regional autonomy “for the review process and the management of money will give a special stability to the regional program. More importantly it will stabilize the staff and improve retention and recruitment. Anniversary review will stimulate a higher level of regional responsibility for its program, for a high guality regional review process and for the setting of | priorities. The management of these responsibilities will generate respect within the region and thus create support ‘for RMP. A system of anniversary review emphasizes program over projects and stimulates the completion of projects with the resulting turnover of funds. This enables ecucial and valuable regeneration of projects and studies which exemplify good program. Lastly, but extremely important, the proposed system of anniversary review smooths out the review process and the review load in the Division. A SUGGESTED SYSTEM - There follows a list of the essential elements and a method of operating a system of anniversary review with fiscal flexibility between dnniversaries. A. Criteria for Eligibility for Anniversary Review Designation - A region must have: 1. 2. One year of operations] status and experience. A high quality regional review process, system of approval and setting of priorities. B. “D. 5. 6. ~-2-— 'An overall plan .or ‘concept for the Region, well identified and well vresented. "Good balance between two, well identified program components — (a) direction, planning and professional services and (b) projects. . Demonstrated competence in fiscal. management. A single grant anniversary. Application - A region would make special application for designation as AR and the application would request awards for 1. 2. 3. _ the following components: Direction, planning and professional services New projects A facilitating award Note: Ongoing projects would be reported, with particular emphasis on their status and a relation to overall program. Nature of Award - An approved AR award would include funding ‘for the following time intervals: 1. 2, 3. “Direction, planning and professional services - three years. New projects - as individually approved, up to three years. Facilitating award - one year. Note: Ongoing projects would continue for the - duration of previous approval. First Anniversary - The following action would take place at the first annual review: 1. 2. Renewal of facilitating award for one more year. Approval of new projects initiated under the facilitating award. 3, General review of DP&PS (and possible extension for ‘the fourth year). 4. General review of ongoing projects. ‘E. Third Anniversary ~ At the end of three years, the following review and approval would take place: ; 1. Extension of DP&PS for three years. 2. Approval of new projects, initiated by facilitating award or newly submitted. ~ 3. Renewal of the facilitating award for one year. ‘4. A review of ongoing projects. F. Sources of Funds - Sources of funds for initiating new projects will include: 1. Program savings 2. Turnover money 3. Facilitating award HYPOTHETICAL EXAMPLE ~ A description of the manner in which a hypothetical region would operate as an AR Region is enclosed as Appendix A. SITE VISITS ~ Deep, highly structured site visits are an essential component of a system of anniversary review. They should place more emphasis on program review, include several. locations within a region ‘and be advisory as well as evaluative. Such a site visit should be held before a region is designated AR and as often on subsequent anniversaries as the Committee and Council may require. While the testing of anniversary review may require site visits in ‘successive years, the expectation would be every three years. This would be consistent with a long range objective, when all regions are firmly established, of total review of each region every three years. This means review of eighteen regions per year, six per review cycle - just about the right load for thorough review of a complex program. STEPS TO IMPLEMENT ANNIVERSARY REVIEW ~ I suggest that the steps to be followed to implement a system of anniversary review are the following: ~-4- Approval of definitions of program components — as per May Council. (Appendix B) Approval in principle by National Advisory Council of Anniversary Review. ' Approval by National Advisory Council of an extension - of the delegated authority. Selection of regions to apply for AR classification. Develop format for AR application. Invite regions to apply. Review Committee conducts program site visit in regions. National Advisory Council approval of regions as AR with facilitating award. g10 AWARD _ EXPENOTTURE ALANCE Dp, Pl. & P.S. é 800,000 .. $ 800,000 : O 2 Projects (A-G) . 1,200,000 | ——-2,,100,000 ¢ (100,000) <— PAL 200,000 150,000 ~50,000¢— ~ $ 2,200,000 Z catenin te 397. REQUEST / APPROVE AWARD Dp. Pl. & PS. $ 800,000 $ 800,000 - - $ 800,000 Projects (D-G) . 1,000,000, 1,000,000 - 1,000,000 - . (200,060) < — - F.A.#2 ~ 200,000 200,000 150 ,000¢-~ (1 yr) Study - 200,000 100,000 100,000 <——~ New Projects J en . JI —. 200,000 200,000 200,000 L- - ~~ 300,000 --0- . — =0- Ni oo "350,000 350,000 ___350,000_ $2,550,000 1. Region has a grant of $800,000 for D.Pi. & PS, $1,200,000 fron projects A thru G, and a facilitating award of $200,000 (FA #1). 2. During 1970 they utilize $150,000 of the facilitating award to initiate, in 1971, projects I, J, & K (which require $200,000 annual level). Also during the year they achieve a. $100,000 savings in projects which they use to fund a cne year feasibility study H. , w N, In 1971 the continuation of projects is only $1,000,000, since projects ABC are completed - leaving 200,000 in ttrenover money which is used to fund the new projects I,J, and K. 5. Also in 1971 they receive $150,000 to add to the $50,000 remaining in FA’ #1 to restore the facilitating award to $200,000 (HA #2). They also have a new project approved (M) and receive a supplemental award of $350,000. The'totdl award for 1971 is now $2,650,000 but the annual level is only $2,550,000 since the $100,000 carryover carries no continuing commitment. , Mf, PROGRAM DIRECTION AND PROFESSIONAL SERVICES SUGGESTED DIFFERENTIAL DEBINITIONS OF COMPONENT'S OF , Ue REGIONAL MEDICAL PROGRAMS aoe we, The central staff which is directly responsible, and the sub-regional -or area staff which is indirectly responsible, to the chief administrative officer of the Regional Medical Program (coordinator or director); and those administrative and professional activities directly related to carrying out those responsibilities, but not including the contribution, in time or other costs, to approved and funded specific projects or studies (as defined below). - - rr. GENTRAL REGIONAL RESOURCE Program components which are required for the development and support "y of the Regional Medical Program, as approved, and which will continue to require grant funds as their principal means of support for as long as they serve the Program of which they are a part. This category can include program evaluation units, data collection and processing resources, etc., as well as service activities designed for immediate response to ‘a regional need (as opposed to establishment of permanent staff services “to meet continuing needs). ry Oo FEASIBILITY STUDY Specifically definable planning activities which ‘require a separate allocation of funds for a sub-contract or other "purchase" of a planning capability or resource, which are time limited and intended to lead to the development of one or more projects. These are sometimes administered. by the central staff, but do not require a permanent increase in staff or represent a continuing cost to the Regional Medical Program. IV. GPERATIONAL PROJECT'S . Time-limited activities which, when completed, can be shown to have. . . RMPS:GCP 04/25/59 Identified and met a continuing need in the Region so that it can be expected to receive future support from local funding resources, or Defined a program direction to be followed up by a new project approach, or have accomplished a short- term Program goal, and will be terminated.