7 Dee # ay

MAGE NAGEMENT
3s BD 2
 

SUMIARY OF EVALUATION CONFERENCE AND RonKsuar

A National RMP Conference and Workshop on Evaluation was held September -
28 - 30 at the University of Chicago Center for Continuing Education.

Attendance

Over 250 persons attended. O£ these about 160 were RMP Staff (coordina-
tors, evaluation directors, and others); another 70 or so were participants
and special invitees; with the remainder largely accounted for by RMPS
staff. Several National Advisory Council (Mrs. Wyckoff and Dr. Cannon)

and Review Committee. members also were present.

 

Background and Objectives of Conference

The Conference was planned by an ad hoc steering committee of RMP
evaluation directors and coordinators in conjunction with the RMPS

Office of Program Planning and Evaluation. A survey of regional problems,
needs, and expertise in evaluation resulted in the committee's decision
that the primary objectives of the conference should be to:

. Present from various perspectives the differing needs for,
and uses of, RMP evaluative material.

. Re-emphasize (or introduce) the concept of program evaluation.

- Present philosophies and methodologies of evaluation in
broad programmatic areas. ° .

. Present methodologies of evaluation in narrower subject
and project~-oriented areas.

. Provide both formal and informal opportunity for consultation
with evaluation experts.

+ Provide RMP representatives concerned with evaluation an
opportunity to become familiar with other RMP personnel
in similar positions and with RMPS and outside evaluation
resources,

- Discuss with participants the need and means for further
formal mechanisms to improve RMP evaluation.

Program

A copy of the program, listing all speakers, is enclosed. A copy of
the paper presented by Dr. Schon in the Monday plenary session is
enclosed elsewhere in your agenda book. Enclosed herewith are
copies of the remarks of Dr. Peter Fox (Office of Management and
Budget) and Mr. John Blamphin (MEDICAL WORLD NEWS).

te teens

dag Bema
 

 

Page 2

Proceedings of the conference, ineludins the remarks of Pre. Samide
and Wilbur at the Monday plenary session, Der. Margulies! luncheon
remarks and those of Dr. Wilson on Wednesday, as well as the papers
presented in the workshop and special interest sessions on Tuesday,
will be available in about six to eight weeks. Copies will be sent
to Council members then.

 

 
 

NATIONAL REGIONAL MEDICAL PROGRAM CONFERENCE
AND WORKSHOP ON EVALUATION

The University of Chicago
Center for Continuing Education
1307 East 60th Street
Chicago, Illinois 60637

 

SUNDAY - September 27
6-10pm........ REGISTRATION Lobby

 

 
 

 

MONDAY - September 28 _ . a

 

7:30 - 9am. ... . . . REGISTRATION : : Lobby
_ : Qam........ . PLENARY SESSION Assembly Room
Presiding

John Glasgow, Ph.D.
Associate Coordinator,

Research and Evaluation
Connecticut Regional Medical Program

Welcome
Morton C. Creditor, M.D.
Program Coordinator

Illinois Regional Medical Program

Introduction and Purpose

Roland L. Peterson

Assistant Director for Program Planning
and Evaluation

Regional Medical Programs Service

"Program Evaluation and Systems Transformation"
Donald A. Schon, Ph.D.

 

 

resident
Organization for Social and Technological
Innovation
10:20 am... . . . . .~COFFEE BREAK
10:45 a.m. . . .. . . . PLENARY SESSION (continued)

“How Others see Regional Medical Programs and
Evaluation"

Alexander M. Schmidt, M.D.

Dean, Abraham Lincoln School of Medicine
University of Illinois

Member, National Review Committee
Regional Medical Programs

Peter Fox, Ph.D.

Senior Economist - Health Evaluation
Office of Management and Budget
Office of the President

John Chase, M.D.

Assistant Chief Medical Director for
Professional Services

Veterans Administration

 
MONDAY - September 28 (continued)

Richard S. Wilbur, M.D.
Assistant Executive Vice President
American Medical Association

John M. Blamphin
Assistant Bureati Chief - Washington Office

MEDICAL WORLD NEWS

 

12:30 p.m. LUNCHEON Dining Room

Speaker

Harold Margulies, M.D.
ting Director

Regional Medical Programs Service

 

2pm... 2. eee eee DISCUSSION GROUP MEETINGS As Assigned

Right separate assigned groups will be asked to
react, discuss and critique the strategies and
policies of evaluation against the background
of morning presentations. (See assignments in
registration packet.)

 

 

 

3:30pm... ee COFFEE BREAK
Spm... eee eee DISCUSSION GROUP MEETINGS (continued)
5:30 p.m... ee ee SOCIAL HOUR Second Floor Lounge

 

TUESDAY - September 29
8:30 a.m. - 3pm... . WORKSHOPS See Appendix A

Workshops in five specific areas will be offered
in three separate sessions. (See Appendix A -
WORKSHOP TOPICS AND ROOMS). These five work-
shops will run concurrently, but attendance may
be divided at the option of the participants.

 

8:30 a.m... we ee SESSION I

 

0:15am... . 2.2.4. COFFEE BREAK

 

10:30 am... . we, SESSION IT

 

 

 
TUESDAY - September 29 (continued)

 

 

 

12 noon. .... . . » + LUNCHEON - Not arranged

1:30 pmo ee ee ee “SESSION IIT

3:15 pam we ee COFFEE BREAK

3:30 pms we ee SPECIAL INTEREST MEETINGS See Appendix B

Meetings covering 12 selected areas of special
interest will be offered and run concurrently.
(See Appendix B - SPECIAL INTEREST MEETING
TOPICS AND ROOMS.)

 

Spm... we ew ewe ADJOURNMENT - sign up for consultation sessions in
Second Floor Lounge area

 

WEDNESDAY - September 30
9am... ee ee ew ee PLENARY SESSION Assembly Roan

Introduction
Harold Margulies, M.D.
cting Director
Regional Medical Programs Service
t

"HSMHA - An Instrument for Improvement

In Heaith Services"

Vernon E. Wilson, M.D.

Administrator

Health Services and Mental Health
Administration

“RMP Evaluation - Where Have We Been; and
Where Do We Go From Here?"
(Panelists to be announced.)

 

10:20 a.m... ww ee ee COFFEE BREAK

 

10:30 am... 1 ee ee FOLLOW- THROUGH MEETING Room 2-EF

Members of the Ad Hoc Steering Canmittee which
planned the conference and selected RMPS staff
and conference discussion leaders will meet to
discuss plans for follow-through. Others are
welcome to attend.

 

 
 

 

10:30 a.m. ee ee

WEDNESDAY - September 30 (continued)

. sINDIVIDUAL CONSULTATION SESSIONS

Participants may request brief consultation
sessions with conference presentors or any
other participants. Scheduling may be done
anytime Monday or Tuesday until 8:00 p.m.
at the consultations desk in the Second
Floor Lounge area. Requests may be made
for specific consultants or by area of
evaluation expertise, in which case an
RMPS staff member will help in selecting
an appropriate consultant.

 

 

 

 

 
 

 

Appendix 1-A

WORKSHOP TOPICS AND ROOMS

Tuesday, September 29, 1970
8:30 a.m. - 3:15 p.m,

8:30 10:30 1:30

to to te
WORKSHOPS § SESSIONS 10:15 12:00 =3:15
A - DATA (Room 2-D)
I. The Values and Limitations of National Data X
II, Data for Ambulatory Care Planning x
III. The Computer and Health Information System X
B - MEASURING CHANGES IN BEHAVIOR (Room 1-AB)
I, Measuring Changes in Knowledge X
II. Measuring Changes in Attitude X
III. Measuring Changes in Clinical Performance X
C - EVALUATION OF CHANGING HEALTH STATUS (Room 2-EF)
I. The Value of Health Status Measures X
II. How to Measure Health Status x
ITI, The Relation of Process and End Result
Evaluation X
D - PROGRAM EVALUATION (Assembly Room)
I. Concerns in Program Evaluation . xX
Ii. Design of Program Evaluation Xx
III. Implementation of Program Evaluation X
E - RESOURCE ALLOCATION/ECONOMICS (Room 2-BC)
I. Social Scientists and RMP Evaluation:
Role, Techniques and Usefulness x
TI. Cost Benefit - Cost Effectiveness xX
TIT. Resource Allocation and Management X

(See Appendix A-2 - A-10 for details of each Workshop Session)

 

 
 

Workshop A - DATA Room 2-D

Appendix A-2

é

Relevance of data to RMP planning and evaluation
will be discussed. The Workshop will be carefully
structured to provide for a free and informal

~ exchange of ideas among RMP personnel concerned
with data, rather than to teach statistical and
epidemiological techniques.

 

 

Moderator: Arthur R. Jacobs, M.D., M.P.H.
Director of Statistical and
Evaluation Unit
Rochester Regional Medical Program
Associate Moderator: Sarah J. Peterson
Biostatistician
Session A-I The Values and Limitations of National
8:30 Data (U.S. Census Data and National
to Health Survey Data)
10:15
Room 2-D Charles A Metzner, Ph.D.

Professor of Medical Care Organization

University of Michigan

 

Session A-II
10:30
to
12:00
Room 2-D

Data for Ambulatory Care Planning
J. William Gavett, Ph.D.

Associate Professor, College of
Management and Department of
Preventive Medicine and Community
Health School of Medicine

University of Rochester

 

Session A-IITI
1:30
to
3:15
Room 2-D

The Computer and Health Information System

David E. Reed, M.D.

Assistant Director of Evaluation

Western Pennsylvania Regional Medical
Program

Katherine G. Bauer

Research Associate

Harvard Center for Community Health and
Medical Care

John E. Wennberg, M.D. ,
Coordinator, Northern New England Regional
Medical Program

- A Regional Health Planning and Management
Data Base

 
 

Appendix A-3

Workshop B - MEASURING CHANGES IN BEHAVIOR Room 1-AB

Moderator: John S. Lloyd, Ph.D.
Assistant Coodinator, Evaluation

Area V - California Regional
Medical Program

 

 

Session B-I Measuring Changes in Knowledge

8:30

to The discussion of techniques for measuring
10:15 changes in knowledge -- some well known and
Room 1-AB some not so well known. Topics included

will be paper-and-pencil measures as well
as measures which have been developed for
Situations in which paper-and-pencil
technqiues are inappropriate.

William R. Crawford, Ed.D.

Associate

Evaluation Studies Section

Center for Study of Medical Education
University of Illinois

 

 

Session B-II Measuring Changes in Attitude

10:30

to Display of methods and materials - the how-
12:00 to-do-it of attitude assessments of
Room 1-AB medical care providers, including ways

attitude change can be objectively
fomulated, evaluated, and documented
to show project and program impact.
Opportunity for questions, problems, and
needs in area of attitude evaluation.

Mitchell Schorow

Assistant Coordinator for Educational
Planning and Evaluation

Intermountain Regional Medical Program

 

Session B-III Measuring Changes in Clinical Performance
1:30

to Discussion of various direct and indirect
3:15 techniques of assessing clinical performance.
Room 1-AB Advantages and disadvantages of each as well
(continued)

 
 

 

Appendix A-4

Session B-III (continued)

‘ as criteria for the selection of specific
measurement instruments will be discussed,
with special attention given to the use and
construction of "patient management problems."

Barbara J. Andrew, Ph.D.
Assistant Professor Medical Education Research

Division of Research in Medical Education
University of Southern California

 

 

 
 

Appendix A-5

Workshop C - THE EVALUATION OF CHANGING HEALTH STATUS Room 2-EF

 

Moderator: Robert R. Carpenter, M.D.

 

Director, Western Pennsylvania

Regional Medical Program

 

Session C-1
8:30

10:15
Room 2-EF

The Value of Health Status Measures

This Workshop will consider the measurement
of the "end result"! of health care efforts

in terms of patient and population well
being. The difficulty of such measurements
and their value is to be discussed. Examples
of methods required to make and interpret
measures of health status will be considered.

Sam Shapiro ,
Director of Research and Statistics
Health Insurance Plan of Greater New York

Maureen M. Henderson, M.D.

Professor, Preventive Medicine

Department of Preventive Medicine and
Rehabilitation

University of Maryland

 

 

Session C-II
10:30
to
12:00
Room Z2-EF

How to Measure Health Status

‘Howard R. Kelman, Ph.D.

Department of Preventive Medicine
and Public Health
New York Medical College

 

Session C-IIT
1:30
to
3:15
Room 2-EF

The Relation of Process and End Result Evaluation

Charles E. Lewis, M.D.
Professor and Head

Health Administration Division
School of Public Health
 

Appendix A-6

Workshop D - PROGRAM EVALUATION Assembly Room

This workship is planned for those who have an appreciation

of the need for program evaluation but who are not familiar
with the common methods or approaches. By participating in
this Workshop, each person should: (1) know the methods for
determining program evaluation priorities, (2) know the com-
mon method for obtaining program evaluation information, (3)

be more confident of his ability to conceptualize program eval-
uation design, and (4) be able to design with or without the
help of consultants a program evaluation design for his own
program.

Moderator: Harold W. Keairnes, M.D.
Coordinator for Evaluation

Tri-State Regional Medical Program

 

Session D-I Concerns in Program Evaluation
A. Approaches to Program Evaluation: Lecture
8:30 and Discussion.
to
10:15 This lecture will introduce the workshop

Assembly Room participants to: (1) The role of program
evaluation in program development and
funding procurement, (2) The common methods
for obtaining and reporting program eval-
uation information and, (3) Method of as-
sessing the utilization of program eval-
uation information. It will create the
foundation for one of the workshop sessions.

Harold W. Keairnes, M.D.
Coordinator for Evaluation

Tri-State Regional Medical Program

B. Questions to be Answered by Program Evaluation:
Panel Discussion.

Through the technique of panel discussion, a
series of questions will be developed which
reflect the program evaluation needs of an
illustrative RMP. The process will clarify
the method by which evaluators can begin the
task of designing their program evaluation
activities.

Harold W. Keairnes, M.D.

Coordinator for Evaluation

Tri-State Regional Medical Program

(continued)

 
 

Session D-I

 

Appendix A-7

B (continued)

Avery M. Colt /
Field Associate of Eastern Massachusetts
Tri-State Regional Medical Program

Claire G. Farrise
Special Projects Coordinator
Tri-State Regional Medical Program

Osler L. Peterson, M.D.

Associate Director for Data Collection
Tri-State Regional Medical Program and
Acting Chaiman .
Department of Preventive Medicine
Harvard Medical School

Dean J. Siebert, M.D.
sociate Coordinator for Dartmouth
Medical School

Tri-State Regional Medical Program

Ruth B. Mott
sear sociate for Data Collection
Tri-State Regional Medical Program

 

Session D-II
10:30
to
12:00
Assembly Room

Design of Program Evaluation

The Workshop will divide into three small
discussion groups. Each group will consist

of one or two staff members from the illus-
trative RMP, and three consultants with specific
interest in, and special concern with, program
evaluation. (Up to ten workshop participants
and an unlimited number of observers. Advance
registration of participants is recommended. )

Priorities for evaluation activities will be
developed by discussions in each small group.

The consultant will provide detailed information
about methods and techniques for obtaining the
information that answers the high priority
questions. Some emphasis will be placed upon

the problems of program evaluation and planning
that have been precipitated by the new Anniversary
Review Guidelines.

 

 

 
 

Appendix 4-8

Session D-II (continued)

Seven people listed above in Session I
Plus

James J. DP Ph.D.

Arthur D. Little, inc.

Robert K. Ausman, M.D.

Deputy Director

Florida Regional Medical Program

Jack E. Thomson, Ed.D
00 ator of Evaluation
California Regional Medical Program

Robert Beckman, Ph.D.
Director of Research

Nassau-Suffolk Regional Medical Program

William R. Thompson

Deputy Director

Washington/Alaska Regional Medical
Program

Paul E. White, Ph.D.

Division of Behavioral Sciences

Johns Hopkins School of Hygiene and
Public Health

Jeannette Forsyth, Ph.D.

roject inistrator, Information Support
System

Tri-State Regional Medical Program

 

Session D-III
1:30
to A.
3:15
Assembly Room

Implementation of Program Evaluation
Making program evaluation useful.

The small groups will continue discussion

by focusing on some of the operation problems
of program evaluation: e.g., funding, reporting
turn-around-time, special requirements, etc.

The range of solutions in the small groups will
join together to compare the results of two
discussion sessions and to demonstrate the
range of possible solutions to common problems.

All consultants from the first two sessions will
participate.
Appendix A-9

Workshop E - RESOURCE ALLOCATION/ECONOMICS Room 2-BC

Moderator:

John Glasgow, Ph.D.

Resoctate Caos:

Associate Coordinator, Research

and Evaluation

Connecticut Regional Medical Program

 

Session E-I
8:30
to
10:15
Room 2-BC

Social Scientists and RMP Evaluation:
Role, Techniques, and Usefulness

Panel discussion to clarify purpose of
evaluation, the potential role of the social
scientist in terms of available tools and
their applicability, and development of a
consensus concerning framework within which
a socioeconomic analysis may be profitably
applied.

Panel Members

A. A. Florin, M.D.
Coordinator, New Jersey Regional Medical
Program

Michael Zubcoff, Ph.D.

Head, Health Economics Section

Tennessee Mid-South Regional Medical
Program

James R. Jeffers, Ph.D.
arector, ica. onomics Research Center
University of Iowa

John E. Wennberg, M.D,
Coordinator, Northern New England Regional

Medical Program

Conrad Seipp, Ph.D. .

Associate brorsscor, Health Services Research
Center ,

University of North Carolina

 

Session E-II
10:30

12:00
Room 2-BC

Cost Benefit - Cost Effectiveness Studies

Explanation of the purpose, objectives and
methodology of such studies with particular
attention to the values and defects of these
studies when applied to health care issues.

(continued)

 
 

Appendix A-10

Session E-II (continued)

Robert L. Berg, M.D.

Professor and Chairman, Department of
Preventive Medicine and Community Health

University of Rochester

 

Session E-IITI
1:30
to
3:15
Room 2-BC

Resource Allocation and Management

Consideration of the economic allocation of
scarce resources within the contest of ,
alternate planning approaches of resource
allocation, particularly PPBS.

Panel Members

Wayne A. Kimmel
Program Analyst, Office of the Assistant

Secretary for Planning and Evaluation
Department of Health, Education and Welfare

Charles W. Caldwell, M.P.A.
Associate Coordinator, Iowa Regional
Medical Program

 

Charles L. Joiner, Ph.D.

Director, Bureau of Research and Community
Service

Alabama Regional Medical Program

 

 
 

Nn
«

> Ww
. .

10.
11.
12.

Appendix B-1

SPECIAL INTEREST MEETING TOPICS AND ROOMS

a Tuesday, September 29, 1970
3:30 - 5 p.m.

Statistical Models and Operations Research............... Conference Lounge
Coronary Care Training Evaluation...........- sae e eee n ec ees Room 1-A

A Systems Approach to Coronary Care Evaluation ee ee cece eee Press Room
Evaluation of Instructional Technology Projects........++ Room 2-EF
Evaluation of Physician Education.......... Ann Room 1-C
Evaluation of Mul tiphasic Screening....... tte e eee seeeeees Room 1-B
Evaluation of Stroke - Rehabilitation Projects..........- Roam 2-A
Evaluation of Cancer Registries......... eee w ec ceneeeenee Library
Evaluation of Regional Advisory Groups..... see e eee eoeces Room 2-B
Medical Care Evaluation........-seeeeees eee e ence eee s eee Room 2-C
Evaluation of New Categories of Manpower......csseeeeueee Room 2-D
Training for Evaluators..... Sac ec cece cence ren seaseeeece Assembly Room
 

Appendix B-2

SPECIAL INTEREST MEETINGS
Tuesday, September 29, 1970
3:30 - 5:00 p.m.

STATISTICAL MODELS AND OPERATIONS RESEARCH Conference Lounge
Moderator: Francis C. Ichniowski

Acting Chief, Systems Management, RMPS
Paper: David H. Gustafson, Ph.D.

Assistant Professor

Industrial Engineering Division
University of Wisconsin

Vernon E. Weckworth, Ph.D.

Director, Systems Development Project
University of Minnesota

Member, Regional Advisory Group
Northlands Regional Medical Program

Discussion

EVALUATION OF CORONARY CARE TRAINING Roon 1-A

Moderator: Rodger M. Shepherd, M.D.
sistant gran Director, Continuing
Education

Presbyterian Medical Center
San Francisco, California

Report: Daniele Deverin
Cybern Education, Inc.
. Report on Xerox Study of Twelve National
Coronary Care Training Centers

Presentation: Mariella Larter, M.S.
Coordinator, Washington/Alaska Coronary

Care Unit Nurse Training Program

Discussion

A SYSTEMS APPROACH TO CORONARY CARE EVALUATION Press Room
Moderator: Morton Robins
Acting Chief, Study Design and Analysis
Staff, RMPS

(continued on next page)
 

 

Appendix B-3

Report: M.A. Rockwell, M.D.
Director, Rand Health Program
Rand Corporation
. A Study of Coronary Care Unit Effectiveness
Progress Report and Proposal for Further
Work

Presentation: Morton Robins
Acting Chief, Study Design and Analysis
Staff, RMPS
. Evaluation of Mobile Coronary Care
Ambul ances

Discussion

EVALUATION OF INSTRUCTIONAL TECHNOLOGY PROJECTS Room 2-EF

Moderator: James E. Dyson, Ph.D.
irector, tinuing Education Division

Colorado-Wyoming Regional Medical
Program

Short
Presentations: Cecelia C. Conrath
i1ef, Continuing Education and
Training Branch, RMPS
- RMPS Policy on Instructional Technology

James Barrett, Ph.D.
Continuing Education Division

Colorado-Wyoming’ Regional Medical Program
. Assessment of Interest of Participants

M. Gene Aldridge

ontinuing cation Division
Colorado-Wyoming Regional Medical Program
+ Feedback Projected

Consultations Via Audio and Visual Telephone Conference:

William J. Paisl Ph.D.
’ Director, ERIC Clearinghouse on Educational
Media and Technology
Stanford University

Elizabeth Norman, Ph.D

Northeastern University College of Nursing

Frederick Breitenfeld, Jr., Ph.D.
Executive Director
Maryland ‘Center for Public Broadcasting

 

(continued on next page)

 

 
 

Appendix b-4

EVALUATION OF INSTRUCTIONAL TECHNOLOGY PROJECTS (continued)
Consultations Via Audio and Visual Telephone Conference:

Gerald W. Gaston, D.D.S.

Project Supervisor, Computer Assisted
~ Instruction (CAT)

Ohio State Regional Medical Program

David L. Beil
i er
Altadena, California

Summary: William Engbretson, Ph.D.
esident, Governor's State University

Park Forest, Illinois
EVALUATION OF PHYSICIAN EDUCATION : Room 1-C
Moderator: Daniel Fleisher, M.D.

Director of Health Professions

Temple University
Resource: William B. Munier, M.D.

Staff Assistant, Continuing Education

and Training Branch, RMPS

Audience will study and discuss three surrogate projects.

EVALUATION OF MULTIPHASIC SCREENING Room 1-B

Presentations: Donald N. Logsdon, M.D.
sociate Director, tiphasic Health
Screening Center
Department of Community Health
Brookdale Hospital Center
- State of the Art of Multiphasic Screening
Evaluation

Prank R. Mark, M.D.

Chief, Operations Research and Systems
Analysis, RMPS
. Suggested Evaluation of MPS Projects

Discussion of predistributed case history

EVALUATION OF STROKE - REHABILITATION Room 2-A

Moderator: Jerome Tobis, M.D.
Chairman Department of Physican Medicine
and Rehabilitation
University of California at Irvine

(continued on next page)

 
 

Appendix B-5

EVALUATION OF STROKE - REHABILITATION (continued)

Presentations: Charles M. pee Ph.D.
roftessor, ic Health Administration
University of Michigan

. Overview of Stroke and Rehabilitation
Evaluation

Bertram L. Tesman, M.D.

Assistant Coordinator, Stroke Program

California Regional Medical Program - Area VIII

. A Comprehensive Stroke Program, Irvine
California

B. Lionel Truscott, M.D.

Director, Stroke Program

North Carolina Regional Medical Program

. A Comprehensive Stroke Program,
North Carolina

Philip A. Klieger, M.D.
Assistant Director for Organizational
Liaison for Stroke and Rehabilitation, RMPS
. Stroke and Rehabilitation - The
National Picture

Discussion

EVALUATION OF CANCER REGISTRIES Library

Moderator: Charles R. Key, M.D.
Assistant Director for Cancer

New Mexico Regional Medical Program

Presentations: Charles R. Smart, M.D.
Director, Intermountain Tumor Registry
Salt Lake City, Utah
- Methodologies for Evaluating Effectiveness
and Value of Registries

George Linden -

thiet, California Tumor Registry

State Department of Health

Berkeley, California

. Alternative Methodologies for Evaluation
of Registries

Discussion
 

10.

 

Appendix B-6

EVALUATION OF REGIONAL ADVISORY GROUPS . Room 2-B

+

Moderator: James P. Harkness, Ph.D.
Deputy Coordinator, New Jersey Regional

Medical Program

Presentations: David A. Pearson, M.P.H.
School o} ic Hea
Yale University
- RAGs as a Factor in the Regionalization
Process

Homer Hagedorn, Ph.D.

Arthur D. Little, Inc.

- Findings of the A.D.L. Study of RMPs
Related to the Functioning and

Characteristics of RAGs.

Paul E. White, Ph.D.
Division of Behavioral Sciences
Johns Hopkins School of Hygiene and
Public Health
- Regional Advisory Groups and Decision
ing - Preliminary Findings From
a Contractual Study

Discussion

MEDICAL CARE EVALUATION Room 2-C

Moderator: Glen E. Hastings, M.D.
Coordinator, Nesta Suffolk
Regional Medical Program

Resource .
Consultant: John W. Williamson, M.D.
Professor, Department of Medical Care
and Hospitals
Johns Hopkins School of Hygiene and
Public Health

Discussion: Group will consider end results evaluation
and its applicability to assessing quality
of health care, including definition of
categories of appropriate measures, examples
of end results measures, and implementation
of end results evaluation.

 

 
 

li.

12.

Appendix B-7

EVALUATION OF NEW CATEGORTES OF MANPOWER Room 2-D

Panelists: ‘Harriet Kitzman
Department of Pediatrics
Strong Memorial Hospital
University of Rochester
. Expanded Role of the Nurse

Sarah Mazelas

Evaluation Director

California Regional Medical Program - Area I
. The Physician Assistant Program

Resource: Veronica L. Conley, Ph.D.
Head, Allied Health Section, Continuing

Education and Training Branch, RMPS

Discussion

TRAINING FOR EVALUATORS Assembly Room

Presentations: George E. Miller, M.D.
Director for Study of Medical
Education
University of Illinois

Donald Pochyly, Ph.D.

Chief of Training, Center for Study
of Medical Education

University of Illinois

Resource: Marian E. Leach, Ph.D.
Head, Educational Sciences Section,
Continuing Education and Training
Branch, RMPS

This interactive: session will focus on two kinds of training:
a) that which prepares operational staff with basic under-
standings about evaluation and, b) that which is applicable
to persons with special training and already highly qualified
in their own areas. Examples of the Center's work in evalua-
tion and training with individual-Regions will be presented.

 
 

 

oe a me 10/26/70

STATUS REPORT
EARMARKED EVALUATION FUNDS

Background

The 1968 legislative extension of Regional Medical Programs earmarked
up to one percent of appropriated funds for evaluation to be utilized
at the discretion of the Secretary of HEW. This legislative proviso
has become increasingly common to most health, education, and other ,
DHEW service-oriented grant programs. It is.expected that this pro-
vision will remain in the legislation for the foreseeable future and
that the Regional Medical Programs Service will be asked to program
approximately one-half of the one percent earmarked funds, with DEW
and HSMHA splitting the other one-half for programming by them.

Available Funds and Implementation

It is estimated that during FY-71 approximately $700,000 will be
available to the Regional Medical Programs Service for the earmarked
evaluation studies. The final amount available is, of course, dependent
upon the appropriation now under Congressional consideration. Depart-
mental policy requires that the funds be used for grants, contracts

and consultant services; they may not be used to support RMPS staff

and in-house evaluation activities. In this connection, the new
multiprogram grants (Section 910 of the Regional Medical Programs
Service legislation) may be one possible mechanism to support these
activities, and the recently-issued guidelines on these grants included
a statement to this effect.

The Office of the Secretary, DHEW, must approve the use of all evaluation
earmarked funds, and each Program has been asked to develop an evaluation
. plan on how it tentatively anticipates to program the funds.

Departmental guidelines on the development of the plans were broad

and suggested that the evaluation activities be primarily (1) retro-
spective, (2) results oriented and, (3) of assistance to the decision-
making levels of the Federal establishment and perhaps to the grantees
(collectively) themselves. "Quick and dirty' studies as well as long-
range studies should be considered. In addition, evaluation methodologies,
information systems, and some types of training were considered
appropriate if they were judged te be priority areas.

On-going and Proposed Activities

A, On-going

Of the estimated $700,000 available to the Regional Medical
Programs Service in FY-71 for this activity, approximately $200,000
may be considered already committed to two contracts initiated in late
FY-70 and an additional $20,000 has been spent on the evaluation
conference held in Chicago in late September. The two on-going contracts
include: .

 

 

 
 

(1) Information Support System for Regional Medical Programs --
Harvard Center for Community Health

Director: Harold Keairnes, M.D.
Amount: $110,000

This contract is testing and developing a socio-political
information support system for management and self-appraisal
to assist a Region in assessing regional status, identifying
changes in the medical system resulting from Regional ,
Medical Program activities, and comparing the effectiveness
of organizational processes in promoting change. Four
Regional Medical Programs are now participating in this
study begun July 1, 1970, including one Region which

"bought into" the study. on an independent basis. The

first series of interviews with Regional Advisory Group
members, selected key regional health leaders and core
staff members have been held and feedback to the Program
Coordinators is now underway. Document analysis is under-.
way and the field interviewer for each RMP has been
selected from the core staffs.

(2) Training of Health Professionals in Characteristics of
Effective Evaluation -- University of Illinois College
of Medicine ,

Director: George Miller, M.D.
Amount: $85 ,000

This contract supports short-term training for an
estimated 250 evaluators and health professionals in
Regional Medical Programs on the use of special evaluative
techniques. One workshop has already been held and four
more are planned for the fiscal year.

B, Proposed

The RMPS evaluation plan covers a wide range of subject areas,
including (1) the development of management information systems, (2)
evaluating the extent and effectiveness of cooperative arrangements,
(3) evaluating the role of RMPs in the implementation and adoption

of the heart guidelines, and (4) assessing demonstration activities.
This plan is now under review by the Department and final clearance
is expected shortly. Several of these proposed evaluation activities
(items 1 and 3) are under active consideration.