Section 9.2.6 PUFF/VM Project

of pulmonary disease in terms of measured data, referral diagnosis, and-
patient characteristics. The program must operate effectively over a wide
range of pathological conditions with a broad clinical perspective about
the possible complexity of the pathology.

B. Medical Relevance and Collaboration

Interpretation of standard pulmonary function tests involves
attempting to identify the presence of obstructive airways disease (OAD:
indicated by reduced flow rates during forced exhalation), restrictive Tung
disease (RLD: indicated by reduced lung volumes), and alveolar-capillary
diffusion defect (DD: indicated by reduced diffusivity of inhaled CO into
the blood). Obstruction and restriction may exist concurrently, and the
presence of one mediates the severity of the other. Obstruction of several
types can exist. In the laboratory at the Pacific Medical Center (PMC),
about 50 parameters are calculated from measurement of lung volumes, flow
rates, and diffusion capacity. In addition to these measurements, the
physician may also consider patient history and referral diagnosis in
interpreting the test results and diagnosing the presence and severity of
pulmonary disease.

Currently PUFF contains a set of about 250 physiologically based
interpretation "rules". Each rule is of the form "IF <condition> THEN
<conclusion>". Each rule relates physiological measurements or states to a
conclusion about the physiological significance of the measurement or
State.

The interpretation system operates in a batch mode, accepting input
data and printing a report for each patient. The report includes: (1)
Interpretation of the physiological meaning of the test results, the
limitation on the interpretation because of bad or missing data; the
response to bronchodilators if used; and the consistency of the findings _
and referral diagnosis. (2) clinical findings, including the applicability
of the use of bronchodilators, the consistency of multiple indications for
airway obstruction, the relation between test results, patient
characteristics and referral diagnosis. (3) Interpretation Summary, which
consists of the diagnosis of presence and severity of abnormality of
pulmonary function.

C. Progress Summary

Knowledge base:

PUFF is implemented on the PDP-10 in a version of the MYCIN system
which is designed to accept rules from new task domains. A typical rule
is:

Priviteged Communication 251 E. A. Feigenbaum
PUFF/VM Project . Section 9.2.6

If (FVC>=80) and (FEV1/FVC<predicted-5) then PEAK FLOW RATES ARE
REDUCED, SUGGESTING AIRWAY OBSTRUCTION OF DEGREE
if (predicted-15<= FEV1/FVC <predicted-5) MILD
if (predicted-25<=FEVI/FVC <predicted-15) MODERATE
if (predicted-35<=FEVI/FVC <predicted-25) MODERATE TO SEVERE
if (FEVI/FVC <predicted-35) SEVERE

This rule compares the ratio of FEV1, the amount of air that can be
forced out in the first second of exhalation with the total "forced vital
capacity” (FVC) or total amount of jung volume that can be exhaled. The
inability to force out a large percentage of air in the critical first
second implies the presence of an obstruction in the airway.

Results

The results of the PUFF system are reviewed in more detail in the
1978 SUMEX annual report and [Kunz 78]. A version of the PUFF system is
now in routine daily use at Pacific Medical Center. Reports are reviewed
by a physician pulmonary physiologist. Over 85 % of the reports are
accepted by the physician without change; they are signed and entered into
the patient record. Most of the remaining reports are edited on-line to
modify a small point in the test interpretation.

Table 1 reviews a study of the agreement in severity of diagnoses
made by two MD's and by PUFF rules. This study was made with a less
complete rule base than what is currently available in the pulmonary Tab.

In 94% of 144 cases analyzed in a prospective study, the degree of severity
(O=none; 1l=mild; 2=moderate; 3=moderately-severe; 4=severe) of OAD
diagnosed by the first MD was within a single degree of severity of OAD
diagnosed by the second MD. In 96% of the 79 cases for which the first MD
diagnosed OAD, the second MD diagnosed the severity of OAD within one level
of the severity diagnosed by the first MD. Agreement within one degree of
severity of the diagnoses by the first and second MD's was substantially
lower in RLD and DD cases. These discrepancies occurred because the second
MD consistently called RLD more severe than did the first MD, and he
consistently did not diagnose diffusion defects when the first MD diagnosed
DD of moderate or greater degree.

E. A. Feigenbaum 262 Privileged Communication
Section 9.2.6 PUFF/VM Project
Percent Agreement
with ist MD
All 144 cases 1st MD made Dx

Second PUFF Second PUFF

Diagnosis M.D. Rules M.D. Rules
Normal

OAD 0.94 0.99 0.96 0.97
RLD 0.92 0.97 0.77 1.00
DD 0.87 0.87 0.60 0.80
Total 0.91 0.94 0,86 0.94

Table 1, Percent agreement within one degree of severity of diagnoses
Approximately 1500 patients have been interpreted by the system.
by two MD's and by the first MD and rules.

In addition to the use of PUFF as a working clinical tool, it has
been very useful for evaluation of knowledge representation methods. The
original PUFF knowledge base (around 60 rules) represents realistic medical
knowledge but is small enough to use for experiments. The PUFF knowledge
has been used in the AGE system, the CENTAUR system using a combination of
rules and prototypes, and the WHEEZE system, a UNIT-based approach to
knowledge-representation.

D. Relevant publications:

[1] "A Physiological Rule-Based System for Interpreting Pulmonary Function
Test Results", J.C. Kunz, R.J. Fallat, D.H. McClung, B.A. Votteri,
J.S.  Aikins, H.P. Nii, L.M. Fagan, E.A. Feigenbaum, HPP 78-154,
Stanford Heuristic Programming Project, 1978.

[2] "Prototypes: An Approach to Knowledge Representation for Hypothesis
Formation", Aikins, J.S., HPP-79-10 (working paper), 1979. Also Int.
Joint Conf. on Artif. Intel1., Tokyo, Japan, August, 1979.

[3] "A Physiological Rule-Based System for Interpreting Pulmonary Function
Test Results", J.C. Kunz, R.J. Fallat, D.H. McClung, B.A. Votteri,
J.S. Aikins, H.P. Nii, L.M. Fagan, E.A. Feigenbaum, Proceedings of
Computers in Critical Care and Pulmonary Medicine, IEEE Press, 1979.

[4] “The Art of Artificial Intelligence: Themes and Case Studies of
Knowledge Engineering", £.A. Feigenbaum, Proceedings of the IJCAI,

(1977). (Also Stanford Computer Science Department Memo STAN-CS-77-
612).

Privileged Communication 253 E. A. Feigenbaum
PUFF/VM Project Section 9.2.6

VM
A. Technical Goals

The Ventilator Manager program (VM) interprets the clinical
Significance of time varying quantitative physiological data from patients
in the ICU. This data is used to manage patients receiving ventilatory
assistance. An extension of a physiological monitoring system, VM (1)
provides a summary of the patient's physiological status appropriate for
the clinician; (2) recognizes untoward events in the patient/machine system
and provides suggestions for corrective action; (3) suggests adjustments to
ventilatory therapy based on a long-term assessment of the patient status
and therapeutic goals; (4) detects possible measurement errors; and, (5)
maintains a set of patient-specific expectations and goals for future
evaluation, The program produces interpretations of the physiological
measurements over time, using a model of the therapeutic procedures in the
ICU and clinical knowledge about the diagnostic implications of the data.
These therapeutic guidelines are represented by a knowledge base of rules
created by clinicians with extensive ICU experience.

The PMC and SUMEX computers will be linked by telephone. The
physiological measurements are generated every 2-10 minutes by the PMC
computer system. It will be provided to VM in real time using the phone
link. Information, suggestions to the clinicians, and/or requests for
additional information will be sent back to the ICU for action.

B. Medical Relevance and Collaboration

To assist in the interpretation process, VM must be able to recognize
unusual or unexpected clinical events (including machine malfunction) in a
manner specifically tailored to the patient in question. The interpretation
task is viewed as an ongoing process in the ICU, so that the physiological
measurements must be continually reevaluated producing a current clinical
picture.

This picture can then be compared with previous summary of patient
Status to recognize changes in patient condition upon which therapy
selection and modifications can be made. The program must also determine
when the measurements are most likely to be sensitive to error or when
external measurements would be of diagnostic significance.

VM offers a new approach toward more accurate recognition of alarm
conditions by utilizing the history and situation of the patient in the
analysis. This is in contrast to the use of static limits applied to
measurements generated to fit the "typical patient" under normal
conditions. Our program uses a model of interpretation process, including
the types and levels of conclusions drawn manually from the measurements to
provide a summary of patient condition and trends. The program generated
conclusions are stated at levels more abstract than the raw data: for
example, the presence of hemodynamic stability/instability rather than in
terms of heart rate and mean arterial pressure. When the data is not
reliable enough to make these conclusions, additional tests may be

E. A. Feigenbaum 254 Privileged Communication
Section 9.2.6 PUFF/VM Project

suggested. The recognition of important conclusion for which external
verification is sought, will also elicit the suggestion for confirming
tests from the program.

C. Progress Summary

VM has been demonstrated using actual patient data recorded on
magnetic tape. Yhe input to VM is the values of 30 physiological
measurements provided on a 2~- or 10-minute bases by a automatic monitoring
system. The output is in the form of suggestions to clinicians and periodic
summaries {see example case below).

Example Case

The following case demonstrates the current state of development of
the system. The data used in this example were obtained from a post-
cardiac surgery patient from the ICU at Pacific Medical Center. The terms
VOLUME, ASSIST, CONTROLLED MANDATORY VENTILATION (CMV), and T-PIECE refer
to specific types of ventilatory assistance. The output format is:(a)
..time of day.., (b) generated comments for clinicians, Starting with "**",
and (c) commentary in {}. ,

..1350.. ..1351..
** SYSTEM ASSUMES PATIENT STARTING VOLUME VENTILATION.
{monitoring started}

** HYPERVENTILATION {diagnostic conclusions
** TACHYCARDIA based on monitored data}
** PATIENT HYPERVENTILATING. {suggested therapy based on
** SUGGEST REDUCING MINUTE VOLUME diagnosis}
..1400..,

1450...

** HYPERVENTILATION

** TACHYCARDIA

** PATIENT HYPERVENTILATING.

** SUGGEST REDUCING MINUTE VOLUME
..16500..

** HYPERVENTILATION

** PATIENT HYPERVENTILATING.

** SUGGEST REDUCING MINUTE VOLUME

Current conclusions: {summary information}
HYPOTENSION PRESENT for 41 MINUTES

HYPERVENTILATION PRESENT for 33 MINUTES

SYSTOLIC B.P. LOW for 46 MINUTES

{etc.}

Privileged Communication 255 E. A. Feigenbaum
PUFF/VM Project Section 9.2.6

Conclusions: {time of day} J..--- J...-. J..... ] .

HEMODYNAMICS -- STABLE
HYPERVENTILATION ~- PRESENT = =
HYPOTENSION -- PRESENT SSeS SaESSses
TACHYCARDIA -~ PRESENT Fass =e

patient is on ASSIST Bans ==
patient is on CMV mans sax
patient is on VOLUME =a

patient is on NOT-MONITORED ase

Goal is CMV Bounmanstonmsnssss2
Goal is VOLUME sans

The availability of new measurements requires updated interpretations
based on the changing values and trends. As the patient setting changes--
@.g., aS a patient starts to breathe on his own during removal (weaning)
from the ventilator--the same measurement values lead to different
interpretations. In order to properly interpret data collected during /
changing therapeutic contexts, the knowledge base includes a model of the
Stages that a patient follows from admission to the unit through the end of
the critical monitoring phase. Recognition of the appropriate patient
context is an essential step in determining the meaning of most
physiological measurements. ,

The majority of the knowledge of the VM program is concerned with the
relations between the various concepts known by the program. These
concepts include: measurement values, typical therapeutic decisions,
diagnostic labels, and physiological states. The connections between
concepts are represented by a form of production rules using the structure
"IF premise THEN action."

The rules in VM are of the form:
IF facts about measurements or previous conclusions are true
THEN
1) Make a conclusion based on these facts;
2) Print out suggestions for the clinician;

3) Establish expectations about the future values
of measurements,

E. A. Feigenbaum 256 Privileged Communication
Section 9.2.6 PUFF/VM Project

A sample VM rule is shown below.

STATUS RULE: STABLE-HEMODYNAMICS
DEFINITION: Defines stable hemodynamics for most settings
APPLIES to patients on VOLUME, CMV, ASSIST, T-PIECE
COMMENT: Look at mean arterial pressure for changes in
blood pressure and systolic blood pressure for maximum
oressures.
IF
HEART RATE is ACCEPTABLE :
PULSE RATE does NOT CHANGE by 20 beats/minute in 15 minutes
MEAN ARTERTAL PRESSURE is ACCEPTABLE
MEAN ARTERIAL PRESSURE does NOT CHANGE by 15 torr in 15 minutes
SYSTOLIC BLOOD PRESSURE is ACCEPTABLE
THEN
The HEMODYNAMICS are STABLE

Figure 1. Sample VM Interpretation Rule. The meaning of ‘ACCEPTABLE’
varies with the clinical context--i.e., whether the patient is receiving
VOLUME or CMV ventilation, etc. This rule makes a conclusion for internal
System use. Similar rules also make suggestions to the user.

An extended description of the VM program can be found in a Ph.D.
thesis to be available shortly as a Stanford technical memo.

D. Relevant publications:

Fagan, L.M., Kunz, J.C., Feigenbaum, E.A. and Osborn, J.J.: A symbolic
processing approach to measurement interpretation in the intensive care
unit. Proc. Third Annual Symposium Computer Applications in Medical
Care, Silver Spring, Maryland, October, 1979, pp. 30-33.

Fagan, L.M., Shortliffe, E.H. and Buchanan, B.G.: Computer-based medical
decision making: From MYCIN to VM. Automedica 3(2), 1980.

Fagan, L.M.: VM: Representing Time-Dependent Relations in a Medical
Setting, Ph.D. dissertation, Stanford University (forthcoming).

Osborn, J.J., Fagan, L.M., Fallat, R.J., et al: Managing the data from
respiratory measurements. Med. Instrumentation, November-December,
1979. (Winner of the ‘Best Article of the Year' Award for AAMI - 1979.)

II. Research Plans
A. Long Range goals and plans

The main emphasis of this project has switched from the development
of the PUFF system to the extension and evaluation of the VM system. This
change is consistent with the goals of the NIH proposal, the current use of
PUFF in a clinical setting and the research questions that remain in the VM
portion of the project. Some long term interests, such as consensus
building between experts, will be examined using both application areas.

Privileged Communication 257 E. A. Feigenbaum
PUFF/VM Project Section 9.2.6

The Tong range goal of the VM project is to develop and evaluate an
interpretation system that will improve patient care in the ICU. Toward
this goal, we plan to extend the rule set, provide better models of
physiology and therapy, and start a forma? evaluation of the program's
therapeutic advice.

The rule set in VM will be extended to handle a greater number of
patients. The current emphasis of the program has been on the management
of post-surgical patients with normal pre-operative status. We will
continue to concentrate on post-surgical patients, but the knowledge base
will be augmented to handle patients with additional problems noted before
surgery or those who have an unusual response to therapy after surgery.
The majority of this knowledge will be used to create a more detailed
classification of the patient population and the corresponding generation
of expectations.

These rule set extensions will ultimately be limited by
representation of the underlying cardiopulmonary physiology and the
therapeutic plans used in the ICU. Still other improvements wil? come from
a better model of the mechanical ventilator and other instrumentation.

Each of these models will provide a structure upon which to build the rule
base, and are motivated by the special problems of evaluating the patient's
status in a dynamic clinical setting. These problems include the
evaluation of the relationship between actual and anticipated response to
therapy and the the recognition of a particular therapy step in the context
of a larger therapeutic plan (e.g., the process of removing a patient from
the ventilator when the patient has an underlying lung disease).

In order to determine the appropriate areas for these model building
activities and to insure acceptance by physicians, a careful prospective
validation wilt be carried out to identify the accuracy of the advice of
the program.

IIi. Interactions With The SUMEX-AIM Resource
A. Collaborations and medical use of programs via SUMEX

The PUFF/VM project requires very close collaboration between
investigators at two institutions separated by fifty miles. This kind of
collaboration, in which program development and testing proceeds
concurrently on the same application system, requires a computer network
facility for sharing of code, data and ideas. SUMEX has been used at PMC
for running programs developed concurrently by Stanford and PMC staff, and
data has been taken from the PMC computer system and transferred to SUMEX
on magnetic tape for program development and testing. The SUMEX staff has
developed a cooperating set of computer programs to allow the PMC computer
and the SUMEX/2020 systems to actively exchange files and program data and
output. This link is required for real-time testing of VM. SUMEX staff
had the necessary resources to design and implement this vital link
mechanism. The link is now undergoing final testing, and it will
dramatically contribute to the effectiveness of the research environment
for VM.

E. A. Feigenbaum 258 Privileged Communication
Section 9.2.6 PUFF/VM Project

We also use the SUMEX system for purposes other than program
development. A joint PMC-Stanford report of VM was prepared entirely
through the the word communications and processing capabilities of SUMEX.
Investigators from the two institutions have collaborated in writing
reports together; the separate contributions are prepared on SUMEX, edited
and merged with an exchange of messages but without ever requiring actual
meetings. We have also used the system for trading bibliographic
information with other AIM users, We have also experimentally run the
Internist program using SUMEX.,

B. Sharing and interactions with other SUMEX-AIM projects

We have participated in the AIM workshop and had very fruitful
interaction with a number of other SUMEX users, directly influencing our
perception of important problems and potentially appropriate solutions.
Personal contacts at other conferences, at Stanford AI weekly meetings, and
at PMC with visiting members of the AIM community, have also been very
helpful in keeping abreast of the current thinking of other members of the
AI community and with members of the medical community interested in
computer based physiological analysis and diagnosis. We believe that the
use of a common machine and the existence of the AIM conference encourages
increased recognition and better communication with other AIM workers.
Within AIM we most closely collaborate with the MYCIN, MOLGEN and DENDRAL
projects, who share common space, common techniques, and common attitudes.

C. Critique of resource management

The SUMEX community continues to be an extremely supportive
environment in which to do research on uses of artificial intelligence in
clinical medicine. The community has two equally vital resources -- the
people with knowledge and interest in AI and the facility on which AI
System development can proceed. They are equally excellent as resources,
helping hands when faced with problems, and friendly support for continued
productive research. The availability of INTERLISP; of a facility on which
routine data processing functions (eg. manipulating magnetic tapes and
making long listings) can take place; and of message-sending among remote
users are all vital functions for our project. SUMEX provides them in an
environment which is friendly and reliable. Management of the SUMEX
facility is consistent and excellent.

D. Needs and plans for other computational resources

The future goals of the project (as described above) will require
considerable computational requirements in the near future. These
requirements will come in the form of active development of a large
INTERLISP program, and extensive testing of the program in a clinical
environment. We hope to perform as much of the evaluation work as possible
on the 2020. System development of the program will probably continue on
SUMEX during off-hours or be off-loaded to the spare time on the 2020. AlI
subsidiary text processing tasks have been off-loaded from SUMEX to avoid
the high load average situation during the day. The storage of usable.
versions of the program and the test files used in the evaluation of the
program will require about 1000 additional pages on the SUMEX computer.

Privileged Communication 259 E. A. Feigenbaum
PUFF/VM Project Section 9.2.6

The process of validation then will require running VM in real time
so that PUFF/VM researchers can compare system interpretations of patient
State with the actual state as determined by careful concurrent clinical
evaluation. We believe that we can effectively use 3-4 hours per day of
running VM in a real time test mode during the initial validation period.
As the system operation becomes more predictable in 1981, longer running
times will be required to identify system problems, and we predict the need
to run the system for a full eight hour shift each day on an intermittent
basis.

E. Recommendations for future community and resource development

We perceive the evolution of our AI capability as moving from a
highly speculative development state, for which the interactive development
capabilities of SUMEX are vital, to a more stable but still changing
validation-and-evaluation state. Ultimately we foresee rather stable
specification of a program for routine clinical use. Thus, we see the need
to transfer our AI techniques from the SUMEX PDP-10 to a local host. For
this transfer, a principal long-range need is for software systems that
will allow us to run AI systems on a mini-computer after they have been
developed on the more powerful SUMEX facility. If the validation of
PUFF/VM in the PMC clinical setting shows the programs to be effective in
health care, then we hope and expect to be able to provide the capability
on a routine basis.

We would aiso like to encourage SUMEX's role as a facilitator of
information transfer between AIM users. This can happen by scheduling on-
line demonstrations that any other user can "connect to," or by providing a
common depository for AI and medicine information. This might take the
form of on-line bibliographies, collecting common user packages, or
connecting common research interests together. This communication service
would compliment the technical service facilities currently provided by the
SUMEX staff.

E. A. Feigenbaum 260 Privileged Communication
Section 9.2.7 Simulation of Cognitive Processes

9.2.7 Simulation of Cognitive Processes

 

Simulation of Cognitive Processes

James G. Greeno
Alan M. Lesgold
Learning Research and Development Center
University of Pittsburgh

SUMMARY OF RESEARCH PROGRAM
Project Rationale

Our goal continues to be contribution to increased theoretical
understanding of basic cognitive processes involved in reading, problem
solving, and other tasks requiring cognitive skills. The form in which we
theorize is computer simulation of human performance. Models of cognitive
processes stand as hypotheses about the components of human information
processing and the ways in which they interact in significant cognitive
tasks. ,

Medical Relevance

Increased understanding of basic cognitive processes is relevant to
medical needs in two ways. One form of relevance involves performance of
tasks in the practice of medicine. One of us (Lesgold) collaborates in
research on cognitive processes in radiology. Understanding of the nature
and organization of these processes, and those in other domains of medical
practice, should provide principles useful in the design of medical
training and the arrangement of conditions for more efficient delivery of
medical services. The second form of relevance of basic research in
cognition to medical needs is in development of understanding of the
cognitive requirements of elementary skills such as reading and arithmetic
computation, in which cognitive deficits can constitute sever disablement.
Improved understanding of these basic skills should provide principles
useful in improvement of diagnosis and therapy for learning disabilities.

Highlights of Research Progress
--Accomplishments this past year

Progress was made in the study of basic processes in reading skill,
where preliminary findings suggest that children whose speed of vocalizing
words develops slowly are destined to be slow in acquiring poor reading
skill (Lesgold, 1979). Progress in Anderson's ACT system and in our own
empirical work will enable more computer simulation work on reading in the
coming year. Comprehension of quantitative concepts was studied, with
development of a hypothesis relating the outcome of problem understanding
and choice of an arithmetic operation through an abstract representation of
a quantitative action (Heller, 1980). Developmental changes in .
quantitative understanding were identified in a study of children from 5 to
8 years of age (Riley & Robinson, 1980). Children's understanding of

Privileged Communication 261 E. A. Feigenbaum
Simulation of Cognitive Processes Section 9.2.7

computational procedures was studied, and instruction based on procedural
analogies was found to be helpful in remedying systematic procedural flaws
in children's performance (Resnick, 1979). A simulation model and a formal
analysis of preschool children's counting skills were developed, providing
some progress on the question of what constitutes understanding of a
general principle relevant to a cognitive procedure (Greeno, Gelman &
Riley, 1978). A theory of problem-solving set and constructions was
developed in the domain of high school geometry proof problems, based on an
idea of schematic knowledge (Greeno, Magone & Chaiklin, 1979).

--Research in progress
Research is continuing on all these topics.

We will briefly describe four research projects that depend on SUMEX
and are most directly relevant to AIM goals.

(1) Work has begun on a study of the acquisition of radiological
skills. The general strategy is to start with empirical data and proceed
to simulations of novice (first-year residents) and expert cognitive
processing during film reading. The final stage will be the development of
learning mechanisms that transform novice models into expert models. We
use protocols of beginning residents, fourth-year residents, and senior
radiologists gathered in relatively naturalistic film-reading situations
along with eye movement data and studies of what subjects see in the first
seconds of examining a film. Current work on the computer simulation part
of the project is directed at development of an anatomy database to
underpin representations of films and to provide a language for describing
feature analyzers and higher level knowledge structures and their outputs.
In general, it is expected that the novice model will be similar in form to
that of HEARSAY-II, while the expert model will have a somewhat more
"compiled" form, perhaps looking more like some of the diagnostic programs
on SUMEX. At this point, only the novice model has been considered in any
detail.

(2) Another study of spatial information processing is focused on
alternative cognitive representations of information in diagrams. Venn
diagrams are presented along with verbal keys that indicate probabilities
of events. In solving simple computational problems, subjects identify
figures in the diagrams that they use in organizing the numerical
information needed for calculations. Subjects differ in the level of
complexity of forms that they identify, indicating that individual
differences in spatial information processing affect performance in this
task in a fundamental way. Simulation models are being constructed using
Anderson's ACT program in the SUMEX system. These models represent
alternative forms of spatial information available to a problem solver, and
permit investigation of the consequences of alternative forms of spatial
information for the inferential processes required of a problem-solving
system,

(3) A collaborative project with John Anderson is focussed on

learning of problem-solving skills. Anderson and Greeno are developing
simulation models, using ACT, representing different stages in the

E. A. Feigenbaum 262 Privileged Communication
Section 9.2.7 Simulation of Cognitive Processes

acquisition of cognitive procedures for solving geometry proof problems.
-Greeno's contribution to the project involves simulation of learning new
procedural skills that make use of previously known schemata that are used
in representing problems. Problems being addressed include (a) acquisition
of productions for instantiating a schema in a new context, thus making
available the problem-solving procedures previously learned in different
contexts; (b) acquisition of new procedural attachments required for
solving new kinds of problems in a familiar domain; and (c) acquisition of
complex schemata formed by combining components of simpler schemata that
were known previously.

(4) We are conducting a formal analysis of acquisition of the syntax
of simple arithmetic sentences. The problem is a form of the language
acquisition problem, and we are developing a system patterned after:
Anderson's (1976) Language Acquisition System (LAS), which depends on
semantic representations of the referents of sentences in acquiring
Syntactic parsing rules. Our project involves an extension of this idea,
since the referents of arithmetic sentences are sequences of actions,
rather than spatial arrangements of objects as Anderson used. The
programming in this project is done in SAIL through the SUMEX system.

(5) Work continues on the longitudinal study of children's.
development of reading skill. This work is expected to facilitate
modelling of different forms of reading acquisition problems by providing
examples of different children's progress in acquiring various components
of effective word recognition.

List of Relevant Publications

Greeno, J. G., Gelman, R., & Riley, M. S. Young children’s counting and
understanding of principles. Paper presented at meetings of the
Psychonomic Society, San Antonio, November, 1978.

Greeno, J. G., Magone, M. E., & Chaiklin, S. Theory of constructions and
set in problem solving. Memory and Cognition, 1979, 7, 445-461.

Heller, J. I. The role of "focus" in children's understanding of arithmetic
word problems. Paper presented at meetings of the American Educational
Research Association, Boston, April, 1980.

Lesgold, A., & Curtis, M.E. Learning to read words efficiently. In A.M,
Lesgold & C.A. Perfetti (Eds.), Interactive processes in reading,
Hillsdale, NJ: Erlbaum, forthcoming.

Lesgold, A.M., Curtis, M.E., Roth, S.F., Resnick, L.B., & Beck, I.L. A
longitudinal study of reading. Paper presented at the Annual Meeting of
The American Educational Research Association, Boston, April, 1980.

Riley, M. S. & Robinson, M. A theoretical framework for word problem

research in arithmetic. Paper presented at meetings of the American
Educational Research Association, Boston, 1980.

Privileged Communication 263 E. A. Feigenbaum
Simulation of Cognitive Processes | Section 9.2.7

Funding Support

National Institute of Education

1.
2.

om

Office of

1.

2.

3.

4,
5.
6

Title: Research on Learning and Schooling

Principal Investigators: Robert Glaser, University Professor and

Co-Director of Learning Research and Development Center, and
Lauren B. Resnick, Professor of Psychology and Co-Director of
Learning Research and Development Center, University of
Pittsburgh

Funding Agency: National Institute of Education

Grant Number: NIE-G-80-0114

Total Award: 1 Dec 1979 to 30 November 1982, $7,879,729.

Current Period: 1 Dec 1979 to 30 Nov 1980, $2,625,520

(During the current period, $150,000 of the above has been
allocated for Greeno's Research and $67,000 for Lesgold's).

Naval Research and Advanced Research Projects Agency

Title: Cognitive and Instructional Factors in the Acquisition
and Maintenance of Skill

Principal Investigators: Robert Glaser, University Professor
and Co-Director of Learning Research and Development Center, and
Alan M. Lesgold, Research Assistant Professor of Psychology,
University of Pittsburgh

Funding Agency: Office of Naval Research (through funds
currently provided by the Advanced Research Projects Agency)
Contract Number: N00014-79-C-0215

Total Award: 1 Jan 1979 to 30 Sep 1981, $1,265,272.

Current Period: 1 Oct 1979 to 30 Sep 1980, $420,000.

National Science Foundation and National Institute of Education

1.

2.

3.

4.

Office of

1.

NM

OO SW

Title: Invention and Understanding in the Acquisition of
Computation

Principal Investigator: Lauren B. Resnick, Professor of
Psychology and Co-Director of Learning Research and Development
Center, University of Pittsburgh.

Funding Agencies: National Science Foundation and National
Institute of Education

Total and current Funding: 1 Dec 1978 to 31 May 1981, $161,238.

Naval Research
Title: Analysis of Formal and Informal Reasoning in Problem
Solving
Principal Investigator: James G. Greeno, University Professor,
University of Pittsburgh
Funding Agency: Office of Naval Research
Contract Number: N00014-78-C-0022
Total Award: 1 Oct 1977 to 30 sep 1980, $274,419.
Current Period: 1 Oct 1979 to 30 Sep 1980, $92,293.

E. A. Feigenbaum 264 Privileged Communication
Section 9.2.7 Simulation of Cognitive Processes

INTERACTIONS WITH THE SUMEX-AIM RESOURCE
Medical Collaborations and Program Dissemination via SUMEX

The work on development of radiology skills is being done in
collaboration with Dr. Yen Wang, Clinical Professor of Medicine, University
of Pittsburgh.

Sharing and Interactions with Other SUMEX-AIM Projects

Two of the five projects described in Section 1.3 involve use of
Anderson's ACT system in SUMEX. The skill acquisition project involves
direct collaboration and programming using the ACT system in Anderson's
directory. The project on spatial information processing with diagrams is
also programmed in ACT. Access to Anderson's programs through SUMEX has
allowed us to avoid costly duplication of his system, which would require
translation from INTERLISP into another dialect as well as unnecessary
duplication of disk files on another system, The reading work also
involves access to ACT, currently for development work and later for actual
building of models of reading.

Critique of Resource Management

None

RESEARCH PLANS
Project Goals and Plans

In the near term, we will complete our analysis of learning
arithmetic syntax, the analysis of acquiring geometry problem-solving
skill, and the analysis of spatial information processing with diagrams.
Work on reading will continue for several years. We expect to use new
versions of ACT that permit partial matching of production conditions to
Simulate one or more different types of low-reading-achievement children.

The radiology diagnosis modelling work is expected to continue
through an initial phase of novice modelling in interaction with empirical
work on chest film reading, after which we will proceed to the expert model
and to specification of learning mechanisms. Those mechanisms are expected
to include some of the mechanisms proposed by Anderson in his current work
as well as mechanisms that take particular account of the need to not have
all of the film viewing process excessively concentrated on the highest-
probability hypotheses. That is, we see a need to understand how good
radiologists come to be able to check films for unexpected pathology (such
as tumors) even when seeing evidence for entirely different disorders, We
hope that this work will lead to a better sense of how to teach
radiologists to exercise this additional care.

Another long-term project is development of a theory of learning -

elementary arithmetic. Arithmetic is a relatively well structured domain.
We now have a considerable body of empirical and theoretical knowledge

Privileged Communication 265 E. A. Feigenbaum
Simulation of Cognitive Processes Section 9.2.7

about the cognitive structures and processes that constitute knowledge of
elementary arithmetic. The development of a system that can acquire this
knowledge appears to be a feasible goal. At the same time, the task of
building a system that acquires both procedural skill and conceptual
understanding and integrates these aspects of knowledge raises theoretical
questions that seem nontrivial. Therefore, development of a learning
system for elementary arithmetic appears to be a productive project for our
research program during the next few years. As noted above, we will also
use ACT for the reading modeling work.

Justification and Requirements for Continued SUMEX Use

We anticipate continued use of SUMEX in development of simulation
programs, particularly in shared use of Anderson's ACT system. Anderson is
presently developing the learning capabilities of ACT in a systematic way,
and this is very likely to be an important resource for our long-range
project on the learning of arithmetic.

Needs and Plans for Other Computing Resources

We depend on SUMEX for a relatively modest, albeit significant, share
of our computing needs. We have installed a VAX-11/780 at LRDC and hope to
benefit from SUMEX-AIM's experience to continue to improve the cognitive
science resources we have locally. We are also exploring possibilities for
involvement in any cognitive science network that may develop. In any
event, though, having direct access to resources such as ACT as they are
developing plays a major role in allowing our work to proceed at the
current pace. Complete detachment from ACT would produce a major setback

and would waste a lot of staff time in re-inventing the work others already
have in place,

E. A. Feigenbaum 266 Privileged Communication
Section 9.2.8 Rutgers Computers in Biomedicine Project [Rutgers-AIM]

9.2.8 Rutgers Computers in Biomedicine Project [Rutgers-AIM]

 

Rutgers Computers in Biomedicine
Rutgers Research Resource--Computers in Biomedicine
Principal Investigator: Saul Amare]

Rutgers University, New Brunswick, New Jersey

IT. SUMMARY OF RESEARCH PROGRAM

 

A) Goals and Approach

 

The fundamental objective of the Rutgers Resource is to develop a
computer based framework for significant research in the biomedical
sciences and for the application of research results to the solution of
important problems in health care. The focal concept is to introduce
advanced methods of computer science - particularly in artificial
intelligence - into specific areas of biomedical inquiry. The computer is
used as an integral part of the inquiry process, both for the development
and organization of knowledge in a domain and for its utilization in
problem solving and in processes of experimentation and theory formation.

The Resource community includes 85 researchers and professionals - 37
members, 11 associates, 28 collaborators and 9 users. Members are mainly
located at Rutgers. Collaborators are located in several distant sites and
they interact, via the SUMEX-AIM and RUTGERS/LCSR facilities, with Resource
members on a variety of projects, ranging from system design/improvement to
clinical data gathering and testing of expert systems. Our collaborations
are described further in section B below. Resource users are located at
Harvard University, John Hopkins University, Ohio State University,
University of Pennsylvania, University of Pittsburgh, Stanford University
and the NIH campus.

Resource activities include research projects (collaborative research
and core research) training/dissemination projects, and computing services
in support of user projects. The research projects are organized in three
main AREAS OF STUDY. These areas of study and the senior investigators in
each of these are:

(1) Medical Modeling and Decision Making (C. Kulikowski)

(2) Modeling Belief Systems and Commonsense Reasoning
(C. Schmidt and N.S. Sridharan)

(3) Artificial Intelligence: Representations, Reasoning,
and System Development (S. Amarel).

The training/dissemination activities of the Rutgers Resource include

sponsorship of the Annual AIM Workshop - whose main objective is to
Strengthen interactions between AIM investigators, to disseminate research

Privileged Communication 267 E. A. Feigenbaum
Rutgers Computers in Biomedicine Project [Rutgers-AIM] Section 9.2.8

methodologies and results, and to stimulate collaborations and imaginative
resource sharing within the framework of AIM. Starting in 1979, the
Workshop is being organized and hosted on a rotational basis by the members
of the AIM community, in coordination with the Rutgers Research Resource.
The fifth AIM Workshop, organized by the MIT - Tufts Clinical Cognition
Project was held in Vermont in May, 1979. The Sixth Workshop is being
organized by the SUMEX-AIM Resource and is to be held at Stanford
University in August, 1980.

B) Medical Relevance; Collaborations;

During 1979-80 we continued the development of a versatile system for
building consultation programs, called EXPERT. This system is being used
extensively in the development and study of several medical consultation
models - in collaboration with clinical investigators from several
Specialties.

Problems in rheumatology are particularly important in health care,
given the high prevalence and chronic nature of arthritis and related
disorders. They also represent an active area of biomedical and clinical
research, in which a group of our medical collaborators at the University
of Missouri under Or. Gordon Sharp has been noted for its contributions.
The application of A.1. approaches to problems of medical decision making
in this domain was facilitated by our collaboration with Dr. Donald ,
Lindberg, Director of the Health Care Technology Center at the University
of Missouri.

Our experience with the design of the rheumatology model has shown us
that the knowledge engineering tools and know-how that we developed so far
in the Resource make it possible to move incrementally and rapidly in the
construction of a new medical knowledge base in collaboration with expert
clinical researchers. Moreover, this experience is leading us to the
development of a methodology for guiding the interaction of medical and
computer science researchers in model building. The sequence of
developments of a consultation models should follow a natural progression
aided at every step by an interplay between the clarification of medical
concepts and the application of logical methods of model design. Our work
in this area is contributing to a better understanding of a central problem
in the application of Artificial Intelligence to the design of expert
computer-based systems; namely, what are the representations, the processes
and the interface facilities that are needed to acquire, augment, and
refine knowledge bases of different types by interacting with specialists
in a domain.

In a single year we progressed from an initial model that represents
a framework of major findings and diagnostic categories for diffuse
connective tissue diseases to a refine model with a broad spectrum of well-
defined observational and decision criteria. It is now being validated and
further developed through a national network of rheumatology specialists
organized by our medical collaborators at the University of Missouri. This
work is directly contributing to the organization of clinical knowledge in
rheumatology. It has been a notable achievement to have been able to reach
a performance of over 90 percent of correct diagnoses on difficult cases of
disease at each step of model design.

E. A. Feigenbaum 268 Privileged Communication
Section 9.2.8 Rutgers Computers in Biomedicine Project [Rutgers-AIM]

In ophthalmology, the CASNET/Glaucoma knowledge base was translated
into the new EXPERT formalism. The development of the glaucoma knowledge
base built in conjunction with the investigators of ONET (ophthalmological
network) was supplemented by knowledge of Japanese variants of the disease
and the decision rules embodying the clinical judgment of Japanese glaucoma
experts. A model for neuro-ophthalmological consultation is being built in
collaboration with Dr. William Hart of the Washington University School of
Medicine, which is related to the automated interpretation of visual field
measurement.

Another collaboration has been in the area of endocrinology, where a
thyroid consultation knowledge base was developed in conjunction with Or.
R. A. Nordyke of the Pacific Health Research Institute.

All the above applications have shown the versatility of the basic
EAPERT representation scheme for rapidly developing medical knowledge
bases. By continued testing and development of various domain models, the
current boundaries of applicability of the EXPERT formalism are being
explored, and new facilities added as required to improve the consultative
performance of the programs developed.

In addition to the direct medical collaboration, we have continued
investigating problems of modeling in enzyme kinetics with Dr. David
Garfinkel of the University of Pennsylvania.

C) Highlights of Research Progress

1) Medical Modeling and Decision-Making

Research activities during the past year have concentrated on the
development and testing of the generalized consultative system scheme
(knowledge representation and associated strategies of inference), called
EXPERT, and its application to a number of different medical domains.

The structure of knowledge in EXPERT involves two data types:
findings and hypotheses. The hypotheses (diagnostic, prognostic and
treatment selection) are organized as a partially ordered network (PON)
using hierarchical and causal relationships. The findings are organized
according to observational constraints. Production rules are used to
encode inferences among findings, between findings and hypotheses, and
among hypotheses. Because of the PON organization of hypotheses, the
knowledge base can be pre-compiled with attendant space and time
efficiencies in the performance of the consultation programs that call on
the knowledge base for decision-making advice.

Knowledge bases in ophthalmology, rheumatology, and endocrinology
have served to test the versatility of the EXPERT formalism. (see I.B
above)

There have been a number of significant generalizations of the EXPERT
scheme during the current period, which fall in the following categories:

Privileged Communication 269 E. A. Feigenbaum
Rutgers Computers in Biomedicine Project [Rutgers-AIM] Section 9.2.8

1.1) Representations:

a) The context for Hypotheses-to-Hypothesis rules can be defined
(anchored) to include matching against a pattern of other hypotheses as
well as findings. This permits both very data-specific contexts, as well
as global contexts of disease domains and consultation environments, which
are used as triggers for the sets of applicable production rules.

b) Multiple visits of a patient can now be handled by the same
scheme. A time representation is being developed currently.

c) Internal functional and logical variables can now be defined for
use by the reasoning schemes, hence allowing specification of clinical
indices, discriminant functions, transformations among variables, etc.

d) Extension of the logical selector operation in the syntax to apply
to hypotheses as weil as findings.

 

e) The current version of EXPERT has been expanded to handle large
amounts of knowledge, up to approximately 600 hypotheses, 3,000 findings
and 20,000 rules, while retaining its processing efficiency.

1.2) Strateqies of Reasoning:

a) A focusing capability, which permits the system to concentrate on
only a preselected set of conclusions at a given time. Repeated
application of the focusing command gives the user direct control over the
"shifting of attention" in the reasoning sequence of the system, which may
be an attractive alternative to the program's usual control strategies. It
is also a powerful tool to test the effects of hypothesis-induced
partitions in the sets of production rules.

b) Strategy selection capabilities have been added, which permit the
user to pre-specify the type of scoring strategies used by the system in
assessing the effects of propagated uncertainties throughout the space of
hypotheses, while interpreting of a given patient's consultation results.
This has proven to be a useful tool in adjusting the scoring method to
match the degree of structural specification in a model. In particular,
for applications where few interdependencies among findings and hypotheses
are known and included in a model, it is desirable to strengthen the
cumulative components of a scoring function.

1.3) Facilities for Model Updating and Explanations:

a) Compiler of benchmark case changes: The compiling program, XP
been designed so that on each revision of the model for which it is
invoked, it will provide a summary of all significant changes in the
conclusions of a set of stored benchmark cases. These can then be
retrieved and analyzed for unexpected effects of changes in the rules or
the descriptive knowledge structure.

, has

b) The explanation facilities have been generalized, so that
Supportive evidence or chains of reasoning leading up to a particular

E. A. Feigenbaum 270 Privileged Communication
Section 9.2.8 Rutgers Computers in Biomedicine Project [Rutgers-AIM]

hypothesis or conclusion can be traced and assessed. The user can specify
the range of uncertainty weights for which he wishes to obtain explanations
of the conclusions (i.e., only those hypotheses that are strongly
confirmed, only those strongly denied, or any other alternative). Rules
can also be ordered according to weight criteria.

1.4) Knowledge-base Transfer Experiments with EXPERT

Experiments were carried out to test the facility with which
knowledge bases constructed for other representations can be transferred
into the EXPERT formalism. They primarily involved the CASNET model of
glaucoma, and demonstrated that the causal structure was, as designed,
representable in the EXPERT scheme. It allowed for the explicit
specification of hierarchical relationships and rules for the inference of
intermediate hypotheses from evidence and final conclusions from patterns
of intermediate hypotheses. There were some specialized features of
CASNET/Glaucoma that have yet to be added to the new System (use of
symmetry relations for binocular findings and hypotheses and the visit-to-
visit logic), that are currently under design.

Another experiment, carried out by Dr. Kitazawa in Japan, took part
of the CASNET knowledge base and transferred it into the EXPERT
representation, and then added new elements (findings, hypotheses, and
rules). to adapt the model to his clinical environment. Examination of the
INTERNIST-I representation showed the compatibilities between some of its
components and those expressible in the EXPERT scheme,

Problems of updating a knowledge base and learning decision rules
from a data base of case records are two other areas of investigation. A
program for rule learning by five different fuzzy-logic heuristic methods
was developed and tested using allergy case study data. Problems of the
transferability of large-scale consultation programs to a minicomputer
environment have also been investigated.

Clinical investigations in thyroid disease and hypertension (by
investigators at Pacific Health Research Institute and the Johns Hopkins
School of Medicine, respectively) have been aided by Resource support and
development of the BRIGHT system. ,

2) Modeling of Belief Systems and Commonsense Reasoning

The central role of commonsense reasoning in human thinking makes it
a particularly important form of reasoning to study and describe. However,
the theoretical frameworks, research methodologies, and analytical tools
that have been developed within psychology are not adequate for this task.
Consequently, over the past 10 to 15 years, psychologists interested in
investigating human reasoning in such "knowledge-rich” domains, have
increasingly looked to the research in artificial intelligence in the hope
that the tools and research strategies of this discipline can be borrowed,
customized, or extended to aid the psychologist in the investigation of
human reasoning.

Privileged Communication 271 FE. A. Feigenbaum
Rutgers Computers in Biomedicine Project [Rutgers-AIM] Section 9.2.8

At the broadest level, our research in commonsense reasoning
represents one of a handful of research projects that are exploring this
intersection of AI and cognitive psychology. We have borrowed, customized
and extended the conceptual tools of AI as a result of trying to state,
justify and test a knowledge-based theory of one aspect of commonsense
reasoning, namely, the problem of how persons recognize the plans and
intentions that quide the actions of another person.

 

 

Progress toward the achievement of the general goal of applying
and/or developing AI approaches for use in developing a cognitive science
in psychology can be attained only by providing, at some level of
approximation, solutions to four problems. First, a general system
framework must be developed within which a knowledge-based psychological
theory of some aspect of human cognition can be expressed. The AIMDS
system has been the continually evolving framework which we have developed
and used to express our theories about commonsense reasoning.

The second and most visible problem is that of representing the
knowledge and processes that constitute the psychological theory of plan
recognition, person perception and belief attribution. We invented the
term BELIEVER in order to distinguish, at least in our mind, those aspects
of the implemented code and architecture that represent information
processing structures and mechanism that constitute a psychological theory
of aspects of human cognitive performance. There have, of course, been
many versions of BELIEVER and there is a sense in which the code that
constitutes BELIEVER represents several theories. The process of plan
recognition requires that mechanisms of retrieval, matching, hypothesis
revision, plan generation, inference, categorization, concept
Specialization or customization and question answering be specified. In
non-AI based research in psychology, these various aspects are typically
treated as independent areas in which theories are formed and tested. In
an Al-based theory such as BELIEVER, there is a "unified" theory of these
phenomena in the sense that the way in which these processes interface and
communicate with each other has been worked out to yield a functioning
system.

The third problem that must be faced is that of developing
experimental paradigms which can yield a set of observations that are rich
enough to constrain and test an information processing theory of this type.
Data from typical psychological experiments yield only a few observations
on a single subject and rarely attempt to speak to the way in which various
processing mechanisms interface with each other to produce the observed
behavior,

Finally, not only must promising experimental paradigms be identified
and response protocols collected, but procedures must be devised for
representing such data and evaluating it against theoretically interesting
hypotheses. This is a very difficult task since the assumptions that
underlie the standard statistical tools used in psychology for this purpose
are usually at variance with the underlying assumptions of information
processing theories.

E. A. Feigenbaum 272 Privileged Communication
Section 9.2.8 Rutgers Computers in Biomedicine Project [Rutgers-AIM]

Over the lifetime of this project we have made forays into each of
these problem areas and have learned a good deal about the terrain of each
of these problems areas--that of devising the system framework, AIMDS, and
that of theory construction, BELIEVER. In the remaining two problem areas
our outposts are still at the fringes of the terrain to be searched
although we do feel that we at least now have an understanding of the
topographies of these areas. This emphasis reflects both our assumptions
about the precedence ordering that naturally falls over these problem areas
as well as the nature of the collaborative effort that exists between AI
and cognitive psychology within the Resource.

3) Artificial Intelligence; Representations and Systems Development

A major part of our effort in this core area continued to be directed
to collaborations with investigators in the other applications - oriented
projects of the Resource. These collaborations are having an impact on the
application areas of the Resource, and they are stimulating work on basic
AI issues that are related to designs of knowledge-based systems.

The following problems are providing foci of collaboration with
investigators in the Medical Systems area: (i) Develop a natural language
interface between a computer consulting system and a medical user; (ii)
Find methods for representing and effectively using several related bodies
of medical knowledge at various levels of resolution (anatomical,
physiological, causal-associational) for decision making in diagnosis and
therapy; (i111) Develop computer tools and design frameworks for
facilitating the construction and improvement of expert systems.

The joint intensive work between investigators in this core area and
researchers in the Belief Systems area is continuing. Developments in the
AIMDS system and in the BELIEVER theory are proceeding in parallel and they
are continuing to influence each other,

During this period we put new emphasis on basic problems of expertise
acquisition and on related problems of theory formation; we increased our
effort in problems of representation, interpretation and model-guided
control of natural processes; we continued basic work on problems of
knowledge acquisition in the context of a language learning task; and we
continued a modest level of effort in programming language development - to
provide a supportive programming environment for our research.

Our research on natural language processing has continued with the
objective to develop methods that facilitate communication between people
(domain experts, users, designers) and computers. We have taken a fresh
look at the problem of developing a convenient man machine interface for a
glaucoma consultation system. Building on our previous work in this area,
we have added several novel features to the design of our interface
processor.

We are continuing to study problems of language acquisition/learning
to gain insight into the general problem of knowledge acquisition in expert
Al systems. However, we have shifted emphasis this year to an approach
which assumes a more active teacher-learner dialogue in the language

Privileged Communication | 273 E. A. Feigenbaum
Rutgers Computers in Biomedicine Project [Rutgers-AIM] Section 9.2.8

acquisition process. This led to the identification of rules that govern
such a dialogue, and to the design of acquisition processes that embody
these rules.

Our commitment to a strong AI programming environment resulted in
improvements of the Rutgers/UCI LISP system, as well as in other systems
programming developments. These efforts are strengthening the tools for
design and experimentation that are available to Resource investigators on

the PUTGERS/LOSR commutar facility.

D) Up-to-Date List of Publications
The following is a list of books, papers and abstracts published in
1978 and 1979 by the Rutgers Resource:

Amarel, S., (1978) "Introduction and Overview for AI in Science and
Medicine," in 'Session on AI in Science and Medicine, National Computer
Conference 1978’, AFIPS Conference Proceedings, Vol. 47, 1978, AFIPS
Press,

Amarel. S. (1979) Invited participation in panel on "History of AI: 1956-
1961," with E. Feigenbaum (Chm.), J. McCarthy, H.A. Simon, IJCAI-79, —
Tokyo, Japan, August 1979; also, S. Amarel chaired two sessions on
Representations at the IJCAI meeting

Biesel, P., C. Kulikowski, S. Weiss, and Z. Aviur, (1979) "Computer-Aided
Acquisition and 3-Dimensional Display of Visual Field Data, Computers
in Ophthalmology Conf. St. Louis, April (1979).

Ciesielski, V., (1979) "Natural Language Input ts a Glaucoma Consultation
System," Presented at the Annual Meeting of the Association for
Computational Linguistics, San Diego, August 11-12, 1979.

Ciesielski, V. [ed.] (1979) "Proceedings of the Fourth Annual AIM Workshop,
"Computer Science Report CBM-TR-104, Rutgers University, August, 1979.

Ciesielski, V., D. Smith, and P. Biesel, (1979) "Artificial Intelligence in
Medicine: Contributions of the Rutgers Resource to the A.I. Handbook,"
Computer Science Report,

Goldberg, R. (1979) "BRIGHT User's Guide - Version 3.04", Report CBM-TR-95,
Department of Computer Science, Rutgers University, January 1979.

Hall, J.S. and N.S. Sridharan, (1978) "Modeling Actions and Processes in
AIMDS: An Example", Report CBM-TM-81, Department of Computer Science,
and Medicine, Vol. 11, (1 and 2), August 1978.

Kulikowski, C., (1979) "Expert Consultation Systems: Designs for
Generality", Hawaii Int. Conf. on Systems Science,

E. A. Feigenbaum 274 Privileged Communication
Section 9.2.8 Rutgers Computers in Biomedicine Project [Rutgers-AIM]

Kulikowski, €. and S. Weiss, (1979) "Representation of Expert Knowledge for
Consultation: The CASNET and EXPERT Projects", Proc. AAAS Annual
Meeting, Houston, January 1979, (in press) [also Rutgers Computer
Science Report CBM-TR-98].

Kulikowski, €., (1979) "Representation of Expert Knowledge for
Consultation; The CASNET and EXPERT Projects,” Report CBM-TR-96,
Department of Computer Science, Rutgers University, January 1979

Kulikowski, €., (1979) Current Research in Artificial Intelligence in
Medicine in the United States", Proc. Japan-AIM Workshop, August 1979.

Kulikowski, €., (1979) "Computer-based Medical Decision-Making”, Computers
in Ophthalmology Conf., St. Louis, April, 1979.

Kulikowski,C., (1978) "Artificial Intelligence Approaches to Medical
Consultation: A Tutorial Review, American Society of Clinical
Pathology, October, 1978.

Kulikowski, C. and S. Weiss, (1978) “Laboratory Computers in the Year
2000", Medical Laboratory Observer, Juiy, 1978, pp. 150-160.

Kulikowski, C.A. and Weiss, S.M., (1978) "The Evaluation of Performance in
Empirical and Theoretical Models of Medical Decision-Making", Proc.
TEEE Computer Society Workshop in Pattern Recognition and Artificial
Intelligence, Princeton, April 1978.

Kulikowski, C.A, (1978) "Artificial Intelligence Apnroaches to Medical
Consultation", Proc. fourth Illinois Conference on Medical Information
Systems, May, 1978.

Kulikowski, C.A, (1978) "Strategies of Glaucoma Treatment Planning",
National Computer Conference-78, AFIPS Conference Proceedings, Vol. 47
1978, AFIPS Press, Anaheim.

+

Mitchell, T.M., (1979) "An Analysis of Generalization as a Search Problem",

IJCAI, Tokyo, Japan, August 1979, [also Rutgers Computer Science Report
DCS-TR-78]

Mitchell, T.M., (1978) "Version Spaces: An Approach to Concept Learning”
PhD Dissertation, Stanford University, December 1978, (Also, Stanford
Computer Computer Science Report STAN-CS-78-711, HPP-79-2).

Mizoguchi, M.K., Maruyama, T. Yamada, Y. Kitazawa, M. Saito, and C.
Kulikowski, (1979), "A Case Study of EXPERT Formalism", Proc. IJCAI,
Tokyo, August 1979, pp. 589-588

Morgenstern, M. (1978) "Transferring Technology from Research Systems to
Users", Proc. The Third Jerusalem Conference on Information Technology,
Jerusalem, August, 1978.

Nagel, D., (1979) "An Experimentation in Extracting Some Properties of
Binary Relations", Department of Computer Science, Rutgers University,
CBM-TM (forthcoming)

Privileged Communication 275 E., A. Feigenbaum