FOR RELEASE TUESDAY, A.M.

JANUARY 17, 1967

 

"PROGRAM EVALUATION''*

 

*Speech by: Vernon E, Wilson, M.D., Dean,
' University of Missouri School
of Medicine, Columbia, Missouri,
and Coordinator Regional Medical.
Program of Missouri Region.

Delivered on: January 16, 1967 (3 p.m.)

Delivered at: Conference on Regional Medical
Programs at Washington Hilton Hotel
Washington, D.C.
January 15-17, 1967
"PROGRAM EVALUATION” |
Vernon E. Wilson, M.D.

Dean and Director
University of Missouri Medical Center

Columbia, Missouri
The dilemma of a dean from the day of his appointment is to know when
to speak out and when to remain silent, Speaking requires at least an

acknowledged topic and at best a brief, flavorful and meaty content. In

pursing the somewhat evanescent title assigned for this topic - which -

evolved from "Program," to "Program and Evaluation," to "Program Evaluation,'

I must confess that the merit of silence loomed ever more attractive,

Since detailed discussion of technical evaluation procedures would not
be appropriate under our limits of time, let us compromise and discuss some
well known principles of program and, for the health field, some relatively

unused principles of evaluation, We will examine both in the light of

opportunities presented by the Regional Medical Program,

The challenge to Regional Medical Programs, as I see it, is to
demonstrate that this new endeavor established primarily in behalf of heart,
stroke, cancer, and related. diseases, is more than a static assemblage of

existing resources, This in itself is a basis for very careful thought.

Most of the principles and programs which can be considered in the field of

health and health care have been studied by one or another of the existing:

governmental, academic, professional or voluntary groups. Thus, at the outset

it seems apparent that the aim of the Regional Medical Program must be one of

synthesis, an effort to combine these various factors into a whole which will

be greater than the sum of the parts.

We have already heard that.the appearance of the Regional Medical Program
through federal legislation was a direct result of growing public ‘and
professional unrest centered around the slow rate at which new knowledge was

being put to use, -- This concern is not unique to the health field but it is new

as a major emphasis among the concerns of the health care professions, The
agricultural and engineering experiment stations, long an integral part of
the land-grant colleges, represent one attempt to deal with this problem, The
Engineers already have a term for it. They label this activity the "transfer
of technology."

It would appear then that the special mission of ‘Regional Medical
‘Programs is primarily one of research in the distribution of health care with
the focus placed firmly upon the patient's needs, rather than upon those of

the. institution or the health professions.
Until the early part of this century the healing arts possessed a dismally

small amount of informations censequently the need was primarily for basic

medical knowledge, With the
give increased emphasis to indirect factors, such as population size, number

momentum now established in basic research we can now

of related organizations and groups, increased capabilities in communication
facilities, and an ever accelerating rate of obsolesence of knowledge, The
magnitude of recent Congressional appropriations indicates the need for
immediate action, Additional and similar legislation is under serious
consideration, The comprehensive health planning act provides a logical
outlet for knowledge developed under Regional Medical Programs. Thus
research being done in the more limited field of Regional Medical Programs
can be of value throughout the total health care field.

Because of the large amount of time and money to be expended,
realistic evaluation of the results is mandatory, Unfortunately, we are
hampered by a lack of effective measurement tools, We must start by
using available techniques, while admitting their inadequacies, It is
essential that collaborative research in system design for the distribution
of health care be initiated in concern with those academic disciplines
who have a long tradition in simulation, systems research, and . communication
research,

. Existing resources for use in the design of such systems are impressive
indeed. If one looks at the great array of governmental health agencies, .
academic institutions, voluntary and professional groups, as well as
supportive organizations iike welfare agencies, community action groups and
others, it readily becomes apparent that the major problem is not that of
creating resources which could appropriately handle the problem but rather
a coordination of those resources inte an effective unit, Although to some
the comparison may be a bit unpalatable, I submit that this is a market and
distribution process and should be handled as such, An approach of this kind
does not deny the essential nature of professional and academic .
contributions; it will require a formal and scientific search for an
appropriate relaticnship between al1 academicians and professionals whose
skills can be helpful, Concurrently, the integrity of the academic and
research community must be preserved, both as an internal system and as a
part of society at large, Thus ,the analogy of marketing is in all probability

.

much more than an analogy, It may prove to be an actual pattern which will
 

-3-

provide us with illustrations and some basic principles for fruitful pursuit
of the tasks ahead,

The Distribution Process.: As a layman in this special field, may I
offer the oversimplified explanation that. the production and distribution
process amounts to a coordination of many disciplines, assembled for the _
contribution which each can make to a-single goal, While such grouping of
resources, particularly in the research process,, suggests the antithesis of
the traditional departmental organization, the.concept.is not unfamiliar to
academic institutions, It is. exemplified.frequently in institutes on |
university campuses, in land grant experiment stations, and research
centers, These patterns allow many. disciplines to proceed in a systematic
fashion in searching for new information and combining, that information

into an orderly whole. . a

Taking the marketing analogy one step further, the rational distribution .
process would be simulated and developed as foilows: .

The first step is the establishment of néed, either. recognized or
unrecognized,. The next step, after tne need is. ‘determined, is to define it
and to create recognition of that specific need in both the consumer and
producer, Here we have a direct. parallel, with the opportunities open to.
Regional: Medical Programs. - =. .

Third, © Having: identified. a. specific need .or. needs, it is necessary to
undertake basic and applied. research.in materials, resources and their
synthesis, The medical profession has expended proportionately small amounts
of its own-energies in‘this endeavor and at. the same time has frequently
poorly utilized the contributions which..could be made by other disciplines,
“Fourth, Having completed. the basic research and formulated working
models, the next step is the production and delivery of materials and services
which may come from:a variety of places, In the analogy the patient may |
move to the resources, Or. the resources may be brought to the. patient, but
finally the delivery. process -requires ‘that the,end product of health
care be synthesized in a coordinated and.personalized manner for the benefit

of thé consumer. ae oe | ae
Market Identification, ‘If we consider health care.in the. light of the

 

patient's need, recognized or unrecognized, the first painful but necessary |

step will be a shift in emphasis,. Much basic research has been sponsored _—
upon the assumption that improvement of the professions and institutions

will automatically benefit patients, However, it may be that the goals
of the patient and those of the profession are not always the same, To

accomplish our task we must now direct extensive study toward the

patient and his needs within the context of his normal pattern of living.
Professional action has classically been one of response after the
patient requests and is given access to the formal health care system, We
must now accept responsibility for health care of the public as a dynamic,
intimate part of daily performance,
Identification of needs for concentrated research endeavors will
he development of end points or goals against which the effect of

Unfortunately, at present,

require t

change in qualitative performance can be measured,

such end points are few and largely unproven,

‘Most of the measurement systems currently used in the health professions

are quantitative rather than qualitative in nature. We can measure quite

adequately deaths, morbidity, numbers of personnel, and similar items,

but we have few means by which we can test the impact of health care upon

the daily performance of a given individual. Thus, our first requirement is

for a measurement system which can assess the ability of the individual .to

perform as a useful member of society and his own attitude toward that

_performance, Also required will be a measurement of the social or peer

's estimate of the value of the individual's contribution to the

group
No single factor can be

group and their attitude toward that contribution,

used as the sole parameter, but when assembled as a pattern these factors

should provide at least the first steps in a qualitative measurement of health

care,
also a part of market definition and since diagnosis

onal individuals, early detection of

Since diagnosis is

opens communications between professi

disease would appear to be a logical first research effort for improvement

in the distribution of health care, .Such research avoids the necessity of -

premature decisions having to do with delivery of health care and would allow .

a "tooling up" of the communications system under reduced emotional tension-

‘Much diagnostic support can be provided to individual practitioners with

a minimal change in their present practice patterns.

~
-5-

Status of the patient needs study. Interaction between individuals is
heavily influenced by the status, stated or felt, of each person, We are
preparing major changes in the status of the patient in the health care

system, This calls for a "shorthand" method interwoven in the system
itself to assess status, and change in status, particularly of the patient.

An interesting correletion exists between the way we use the time of
others and our estimate of their importance. Consequently, accurate
determination of our expenditure ef the patient's time through the design
of health services is accessible, measureable, and potentially valuable.

Another little used field of knowledge is that developed in advertising
research, Significant portions of established knowledge about health is not
utilized even by those best acquainted with it, Advertising research has
a rich body of basic knowledge and techniques dealing with facilitators,
or why people choose one service or product as opposed to another, These
toals and techniques used so successfully in advertising could be adapted
and should be useful in broadening public education and personal responsibility
in health care, .

Turning to the third item in our analogy, research in materials and
resources, we should comment first on basic research which has. a long
university tradition and is the foundation upon which applied research is
conducted, Basic research in almost all academic disciplines will make
important contributions to health care, High on the list should be
research in synthesis of systems, including model building, ‘In our
past, testing through models has had little systematic and comprehensive
attention, It could produce large savings in time, as well as funds, but
will require the talents of a variety of existing disciplines, the engineers
for example, who until recently were eeldom formally invited into the health
research conversation, ,

An interesting facet of the dilemma related to manpower shows in the
fact that although we. are faced with a tremendous shortage of health personnel
and a low level of national ‘unemployment, we as a health care group have
largely ignored one of our greatest potentials - the patient himself, He is
usually the most involved, often the better educated and certainly the most
highly motivated party in the interchange, yet we have assigned him the most
passive role, Patients, I submit, may not be so helpless as some of our

practices would seem to imply. Our friends in sociology should be able to

help us here,
In the fourth and final phase of our analogy, we will face a variety of

problems in the delivery of health care, These include implementation of

research and development in distribution, All patients should have access

to the best source of care regardless of geography, financial resources,
or special interests of particular ‘professional groups. New patterns are
required, So ,

The relationship between centers of excellence and the population

which they would serve will need’ to be defined, Most organizations which

support health care use politically determined boundaries, i. e,, the city,
county or state, The probability of gaining coordinated support from all’
interested organizations for the assistance of a single and specific’
individual will be’ enhanced ‘by a maximum overlap in geographical areas
of designed responsibility, oe

A second problem ‘to be considered deals with control,

distribution systems ‘be ‘totally under the control of ‘the health professions?

Should ‘such °

If not, how much of the Process ‘should be Gonducted in cooperation with other

interested groups? “When should ‘control be turned to them?

_ A third problem concerns the obsolescent mind, both as it relates
to the medical profession itself and to the public at large, It is clear
that planned, continuing education for the profession and the public is
necessary, A searching look at potential integration of such education

with the care process seems éalled for. “Feed back mechanisms must be'

established for a progressive analysis of cause and effect, or, at least,
correlation between continuing education and change. *
A successful distribution system will itself require an integrated

information service, “Information should be derived from the home, from the °

avenue of access to the health care system, the ‘local hospital, and the large
medical center, It will require the development of common identification ‘
systems and vocabularies, Many of us hope that in the very near future the —

social security number will be issued at the time of birth, or entry into

the country, ‘and will provide such identification, “The proposed information

system should be designed to utilize, assist ‘and refine present systems, not ~
compete with them, Oo " oe
The decision for diagnosis “and treatinent of the patient will take into

account his desires which, among other things, relate to the distance from’

health care and the patient! s ‘knowledge of and confidence in the récommended ©

resource,

Other considerations are the adequacy of the ‘health care resources, ’-°:
the cost to the patient and the involved agencies, and the maximum benefit

_ from the care process which includes such by-products as education, research,

and economic impact upon the community at large.

Finally as we have already heard, no matter how one may describe a
Medical Region, it must interact with other regions. Mechanisms must be

developed which will minimize the mechanical problems of inter-regional

relationships and permit us to focus upon the patient.

THE EXAMPLE, With no claims to assured success, the Missouri Regional
Program has attempted to face these challenges in the planning process.
Projects will arise from community groups and be funneled through a refinement

process, This should encourage maximum motivation and participation at the

grassroots level,
A general objective of the program is the development of models of

early detection integrated with continuing education,

Primary emphasis will be placed on those endeavors which can be
quantitatively evaluated and the initial assumption is made that adequate
information and communication will provide qualitative improvement. The °

long range plan provides for qualitative measurement of delivered health

care,

Only a few projects are. proposed for studies of delivery of care, It
is our intent simply to be supportive to existing care patterns while setting
up the necessary information gathering mechanisms, Under this plan, a
request for information by the physician will be met by specific answer to
the question, along with additional synoptic background information or
bibliographies which should be helpful in his continuing education, Such
_ inquiries will also serve as a guide to the physician's needs, In this
manner diagnostic and delivery patterns of health care can quickly be
modified in detail when research indicates the desirability of doing so.

The data handling facility developed at the University of Missouri: for
the purpose of extending the competency of the physician will be integrated
with cooperative data handling programs established by hospitals, physician's
offices and state agencies, This integrated system is expected to furnish
feed back and monitoring which will make it possible to provide the desired

information while studying and coordinating the total process in an objective

and efficient manner,
A University multidiscipline research unit is developing new tools with
which. to measure achievement, Its staff members have joint appointments with ©
other schools on campus, including Nursing, Education, Engineering, Journalisn,
Business and Public Administration, Liberal Arts, and Veterinary Medicine,
Presently members of this unit are studying two different communities in
which they will measure efforts toward community health goals, such as
rehabilitation of the patient, family reactions and the like,

In conclusion, let us review, quite briefly, s>me goals'worthy of
consideration, These geals were picked because progress toward them can be
measured, Their evaluatien should give us some insight into whether or not --
we are moving in the direction that may be most efféctive in meeting the
actual needs of patients, _ i
i. The primary goal is to deliver the highest percentage of quality
patient care as close to the patient as possible, This is ‘not only
economical in the tctal picture but ‘in keeping with the desires of most
patients, Certainly the latter assumption merits study,

2. Every patient should have equal access to any needed national
resource, For very special services which are not avilable in the area,
patients can be sent to centers of excellence elsewhere, ‘thus eliminating -
the necessity for duplication of expensive equipment, staff and facilities.

3, Maximum coordination will be sought between the inputs of those:
who provide health care directly, as well as those involved in- supporting
that care, such as welfare, communi ty resources, eavironmentat control
groups, and others. , a

4, The development of programs ‘to assist in early and effective
detection of disease will be a primary goal. The information gained can
be used to effect changes in delivery of health care,’ both through personnel -
and systems, Early detection is perhaps least threatening to the present
health care professions and is “among: the easiest ‘procedures to measure

quantitatively. Tt also possesses the highest potential for successful

qualitative measurements of health care,
5. Post graduate edueation should be ‘integrated with detection and -

health care systems,
6, Lay health education will be a vital part of the regional program,

Existing programs in adult education, extension and voluntary organizations
will be utilized so that the potential recipient of care may be informed as
to the role which his physician, the hospital, and the various supporting
‘agencies will play and to the things which he, the patient, can expect, We
need more scientifically designed studies of public attitudes toward health
care,

7, Finally, in my view, a crucial goal will be for each of the
several regions to take a unique approach to the special needs of their
particular areas, Through meetings such as this one, we can share ideas
so that a minimum of waste will ensue as we seek to meet our respective
responsibilities.

New paths, I would remind you, are seldom explored by faint hearts.
Perhaps we need to be mindful that, in the development of new systems,
one may at times work with less than perfect parts in order to set the
system itself in operation, It is possible, even desirable, to have- "proof
runs", a practice long utilized by the printing industry. From less than
perfect initial operations, changes and corrections can be made to improve the
final product,

As participants in this national program I believe we dare not do less
than marshall the best available talents, from whatever quarters, to join

in this quest for improved health care. The opportunities are attractive

and challenging to say the least.