4) SOLVER Project 4.70 "Problem Solving Expertise” Paul E. Johnson, Ph.D. William B. Thompson, Ph.D. University of Minnesota Control Data Corp. (Johnson) 1983-85 $90,000 Microelect. and Info. Ctr. Univ. of MN (Johnson, Thompson, Slagle, Wechsler, Yonas) 1984-1985 $500,000 NIH LM-00160 (Johnson, Connelly) 1984-1989 $712,573 McKnight Foundation (Johnson, Bailey) 1984-1985 $13,000 Dwan Family Fund, Univ. of MN Medical School (Johnson) 1985 $6,000 5) MENTOR Project §.41 “Medical Evaluation of Therapeutic Orders” Stuart M. Speedie, Ph.D. University of Maryland Terrence F. Blaschke, M.D. Stanford University National Center for Health Services Research 1 R18 HS 05263 1/85~-12/88 $485,134 1/85-12/85 $147,170 6) *** [Rutgers-AIM] *** Rutgers Research Resource 0.62 Artificial Intelligence in Medicine Casimir Kulikowski, Ph.D. Sholom Weiss, Ph.D. Rutgers U., New Brunswick NIH RR-02230-02 (Kulikowski, Weiss) 12/83-11/87 $3,198,075 12/84-11/85 $613,897 7) AIM Pilot Projects 69.84 8) AIM Administration 0.42 9) AIM Users 27.88 Community Totals 241.87 Privileged Communication 301 413.29 497.78 57.29 4292.54 57.86 3498.43 16289.49 Resource Operations and Usage Data 621 380 196 E. H. Shortliffe Resource Operations and Usage Data CPU Connect File Space Stanford Community (Hours) (Hours) (Pages) 1) GUIDON-NEOMYCIN Project 67.60 8225.93 6048 Bruce G. Buchanan, Ph.D. William J. Clancey, Ph.D. Dept. Computer Science ONR/ARI N00014-79-C-0302 3/79=3/85 $683,892(*) 2) MOLGEN Project 238.64 8358.21 11392 “Applications of Artificial Intelligence to Molecular Biology: Research in Theory Formation, Testing and Modification” Edward A. Feigenbaum, Ph.D. Peter Friedland, Ph.D. Charles Yanofsky, Ph.D. Depts. Computer Science/ Biology NSF MCS-8310236 (Feigenbaum, Yanofsky) 11/83-10/85 $270,836(*) 11/84-10/85 $131,621(*) 3) ONCOCIN Project 182.81 18869.06 16406 “Knowledge Engineering for Med. Consultation" Edward H. Shortliffe, M.D., Ph.D. Dept. Medicine NIH RR-01613 1/83-6/86 $624,455 7/84-6/85 $222,511 NIH LM-04136 8/83-7/86 $211,851 8/84-7/85 $69,875 H.J. Kaiser Family Fdn. 7/83-6/86 $150,000 1/84-6/85 $50,000 NSF IST83-12148 Bruce G. Buchanan (Shortliffe) 3/84-2/87 $330,000(*) 3/84-2/85 $101,308(*) NIH 1 T32 LM07033 7/84-6/89 $903,718 1/84-6/85 $79,059 NIH 1 R23 LM04316 2/85-1/88 $107,441 2/85-1/86 $37,500 E. H. Shortliffe 302 Privileged Communication Resource Operations and Usage Data 4) PROTEAN PROJECT 401.52 8539.01 13156 Oleg Jardetzky School of Medicine Bruce Buchanan Computer Science Department NSF PCM-84-02348 11/84-10/86 $100,000(*) 11/84-10/85 $50,000(*) 5) RADIX Project 33.23 2315.62 9168 "Deriving Medical Knowledge from Time Oriented Clinical Databases" Robert L. Blum, M.D. Gio C.M. Wiederhold, Ph.D. Depts. Computer Science/ Medicine NSF IST-8317858 (Blum) 3/84-3/86 $89,597(*) NIH LM-04334 (Wiederhold) $/84-11/86 $291,192 6) Stanford Pilot Projects 277.71 6545.02 5092 7) Core AI Research 139.65 9447.97 10358 8) Stanford Associates 11.40 1030.22 1127 9) Medical Information Sciences 16.52 2561.42 974 Community Totals 1369.08 65892.46 70901 Privileged Communication 303 E. H. Shortliffe Resource Operations and Usage Data KSL-AI Community For funding details please see page 105 1) Advanced Architectures 2) FOL | 2) Intelligent Agent 3) Pixie 4) KB VLSI 5) KSL Management 6) DART 7) MRS Community totals SUMEX Staff 1) Staff 2) System Associates Community Totals System Operations 1) Operations Resource Totals (*) Award includes indirect costs. E. H. Shortliffe CPU (Hours) 261.44 26.84 304 Connect (Hours) 11070.95 781.19 6934.73 1989.63 1275.64 21341.80 1497.89 9298.69 54190.52 Connect (Hours) 21450.55 1809.75 23260.30 Connect (Hours) 69589.10 229221.87 Privileged Communication File Space (Pages) File Space (Pages) 17051 4744 File Space (Pages) 131640 Ssesssa 297492 Resource Operations and Usage Data System Reliability System reliability for the DECsystem 2060 has significantly improved in this past period. We have had very few periods of particular hardware or software problems. The data below covers the period of May 1, 1984 to April 30, 1985. The actual downtime was rounded to the nearest hour. Table 1 : System Downtime Hours per Month - May 1984 through April 1985 13 1 16 5 9 17 1 N/A 26 9 8 9 May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr Table 2 : System Downtime Hours per Month - May 1984 through April 1985 Reporting period : 364 days, 19 hours, 13 minutes, and 25 seconds Total Up Time : 359 days, 11 hours, 32 minutes, and 18 seconds PM Downtime : 1 days, 6 hours, 8 minutes, and 1 seconds Actual Downtime : 4 days, 1 hours, 33 minutes, and 6 seconds Total Downtime : 5 days, 7 hours, 41 minutes, and 7 seconds Mtodf : 3 days, 14 hours, 16 minutes, and 31 seconds Uptime Percentage : 98.89 Network Usage Statistics The plots in Figure 18 and Figure 19 show the monthly network terminal connect time for the TYMNET and the INTERNET usage. The INTERNET is a broader term for what was previously referred to as Arpanet usage. Since many vendors now support the INTERNET protocols (IP/TCP) in addition to the Arpanet, which converted to IP/TCP in January of 1983, it is no longer possible to distinguish between Arpanet usage and Internet usage on our 2060 system. Privileged Communication 305 E. H. Shortliffe Resource Operations and Usage Data 1400° tVMNET Connect Time Hours/Month = - 1200+ L 1000 800t 600} i > | > 4001 200+ oO i de rf i sds dn i dew: ra deme i 1 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 [985 1986 Figure 18: TYMNET Terminal Connect Time 12007 aRPAnet Connect Time Hours/Month 1000 ¥ 800+ 600; 400; 200; oO a he L. 4 re i 4 he he i. 4 4 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 Figure 19: ARPANET Terminal Connect Time E. H. Shortliffe 306 Privileged Communication AIM Management Committee Membership Appendix C AIM Management Committee Membership Following are the current membership lists of the various SUMEX-AIM management committees: AIM Executive Committee: SHORTLIFFE, Edward H., M.D., Ph.D. (Chairman) Principal Investigator - SUMEX Medical Computer Science, TC135 Stanford University Medical Center Stanford, California 94305 (415) 497-6970 FEIGENBAUM, Edward A., Ph.D. Co-Principal Investigator - SUMEX Heuristic Programming Project Department of Computer Science 701 Welch Road, Building C Stanford University Stanford, California 94305 (415) 497-4879 KULIKOWSKI, Casimir, Ph.D. Department of Computer Science Rutgers University New Brunswick, New Jersey 08903 (201) 932-2006 LEDERBERG, Joshua, Ph.D. President The Rockefeller University 1230 York Avenue New York, New York 10021 (212) 570-8080, 570-8000 LINDBERG, Donald A.B., M.D. (Past Adv Grp Chrmn) Director, National Library of Medicine 8600 Rockville Pike Bethesda, Maryland 02114 (617) 726-8311 MYERS, Jack D., M.D. School of Medicine Scaife Hall, 1291 University of Pittsburgh Pittsburgh, Pennsylvania 15261 (412) 624-2649 Privileged Communication 307 E. H. Shortliffe AIM Management Committee Membership AIM Advisory Group: MYERS, Jack D., M.D. (Chairman) School of Medicine Scaife Hall, 1291 University of Pittsburgh Pittsburgh, Pennsylvania 15261 (412) 624-2649 AMAREL, Saul, Ph.D. Department of Computer Science Rutgers University New Brunswick, New Jersey 08903 (201) 932-3546 BAKER, William R., Jr., Ph.D. (Exec. Secretary) 10235 Gainsborough Road Potomac, Maryland 20854 FEIGENBAUM, Edward A., Ph.D. (Ex-officio) Co-Principai Investigator - SUMEX Heuristic Programming Project Department of Computer Science 701 Welch Road, Building C Stanford University Palo Alto, California 94305 (415) 497-4879 KULIKOWSKI, Casimir, Ph.D. Department of Computer Science Rutgers University New Brunswick, New Jersey 08903 (201) 932-2006 LEDERBERG, Joshua, Ph.D. President The Rockefeller University 1230 York Avenue New York, New York 10021 (212) 570-8080, 570-8000 LINDBERG, Donald A.B., M.D. Director, National Library of Medicine 8600 Rockville Pike Bethesda, Maryland 02114 (617) 726-8311 MINSKY, Marvin, Ph.D. Artificial Intelligence Laboratory Massachusetts Institute of Technology 545 Technology Square Cambridge, Massachusetts 02139 (617) 253-5864 E. H. Shortliffe 308 Privileged Communication AIM Management Committee Membership MOHLER, William C., M.D. Associate Director Division of Computer Research and Technology National Institutes of Health Building 12A, Room 3033 9000 Rockville Pike Bethesda, Maryland 20205 (301) 496-1168 PAUKER, Stephen G., M.D. Department of Medicine - Cardiology Tufts New England Medical Center Hospital 171 Harrison Avenue Boston, Massachusetts 02111 (617) 956-5910 SHORTLIFFE, Edward H., M.D., Ph.D. (Ex-officio) Principal Investigator - SUMEX | Medicai Computer Science, TC135 Stanford University Medical Center Stanford, California 94305 (415) 497-6970 SIMON, Herbert A., Ph.D. Department of Psychology Baker Hall, 339 Carnegie~Mellon University Schenley Park Pittsburgh, Pennsylvania 15213 (412) 578-2787, 578-2000 Privileged Communication 309 E. H. Shortliffe AIM Management Committee Membership Stanford Community Advisory Committee: FEIGENBAUM, Edward A., Ph.D. (Chairman) Heuristic Programming Project Department of Computer Science Margaret Jacks Halli Stanford University Stanford, California 94305 (415) 497-4879 LEVINTHAL, Elliott C., Ph.D. Departments of Mechanical and Electrical Engineering Building 530 Stanford University Stanford, California 94305 (415) 497-9037 SHORTLIFFE, Edward H., M.D., Ph.D. Principal Investigator ~ SUMEX Medical Computer Science, TC135 Stanford University Medical Center Stanford, California 94305 (415) 497-6970 E. H. Shortliffe 310 | Privileged Communication Collaborative Project Abstracts Appendix D Collaborative Project Abstracts The following are brief abstracts of our collaborative research projects. Privileged Communication 311 E. H. Shortliffe Collaborative Project Abstracts Stanford Project: GUIDON/NEOMYCIN -- KNOWLEDGE ENGINEERING FOR TEACHING MEDICAL DIAGNOSIS Principal Investigators: William J. Clancey, Ph.D. 701 Welch Road Department of Computer Science Stanford University Palo Alto, California 94304 (415) 497-1997 (CLANCEY@SUMEX-AIM) Bruce G. Buchanan, Ph.D. Computer Science Department 701 Welch Road Stanford University Palo Alto, California 94304 (415) 497-0935 (BUCHANAN@SUMEX-AIM) SOFTWARE AVAILABLE ON SUMEX GUIDON--A system developed for intelligent computer-aided instruction. Although it was developed in the context of MYCIN's infectious disease knowledge base, the tutorial Tules will operate upon any EMYCIN knowledge base. NEOMYCIN--A consulation system derived from MYCIN, with the knowledge base greatly extended and reconfigured for use in teaching. In contrast with MYCIN, diagnostic procedures, common sense facts, and disease hierarchies are factored out of the basic finding/disease associations. The diagnostic procedures are abstract (not specific to any problem domain) and model human reasoning, unlike the exhaustive, top-down approach implicit in MYCIN’s medical rules. This knowledge base will be used in the GUIDON2 family of instructional programs, being developed on D- machines. REFERENCES 1. Clancey, W.J.: Overview of GUIDON. In A. Barr and E.A. Feigenbaum (Eds.), THE HANDBOOK OF ARTIFICIAL INTELLIGENCE, Vol. 2. William Kaufmann Assoc., Los Altos, CA, 1982. (Also to appear in J. of Computer- based Instruction) 2. Clancey, WJ.: Methodology for building an intelligent tutoring system. In Kintsch, Polson, and Miller, (Eds., METHODS AND TACTICS IN COGNITIVE SCIENCE. L. Eribaum Assoc., Hillsdale, NJ. 1984. (Also STAN-CS-81-894, HPP 81-18) 3. Clancey, W.J.: Acquiring, representing, and evaluating a competence model of diagnosis. In Chi, Glaser, and Farr (Eds.), THE NATURE OF EXPERTISE. In preparation. HPP-84-2. E. H. Shortliffe 312 Privileged Communication Collaborative Project Abstracts Stanford Project: MOLGEN -- AN EXPERIMENT PLANNING SYSTEM FOR MOLECULAR GENETICS Principal Investigators: Edward A. Feigenbaum, Ph.D. Department of Computer Science Stanford University Charles Yanofsky, Ph.D. (YANOFSK Y@SUMEX-AIM) Department of Biology Stanford University Stanford, California 94305 (415) 497-2413 Contact: Dr. Peter FRIEDLAND@SUMEX-AIM (415) 497-3728 The goal of the MOLGEN Project is to apply the techniques of artificial intelligence to the domain of molecular biology with the aim of providing assistance to the experimental scientist. Previous work has focused on the task of experiment design. Two major approaches to this problem have been explored, one which instantiates abstracted experimental strategies with specific laboratory tools, and one which creates plans in toto, heavily influenced by the role played by interactions between plan steps. As part of the effort to build an experiment design system, a knowledge representation and acquisition package--the UNITS System, has been constructed. A large knowledge base, containing information about nucleic acid structures, laboratory techniques, and experiment-design strategies, has been developed using this tool. Smaller systems, such as programs which analyze primary sequence data for homologies and symmetries, have been built when needed. New work has begun on scientific theory formation, modification, and testing. This work will be done within the domain of regulatory genetics. We plan to explore fundamental issues in machine learning and discovery, as well as construct systems that will assist the laboratory scientist in accomplishing his intellectual goals. SOFTWARE AVAILABLE ON SUMEX SPEX system for experiment design. . UNITS system for knowledge representation and acquisition. SEQ system for nucleotide sequence analysis. REFERENCES 1. Friedland, P.E.: Knowledge-based experiment design in molecular genetics, (Ph.D. thesis). Stanford Computer Science Report, STAN-CS-79-771. 2 2. Friedland, P.E.: Knowledge-based experiment design in molecular genetics, Proc. Sixth IJCAI, Tokyo, August, 1979, pp. 285-287. 3. Stefik, MJ.: An examination of a frame-structured representation system, Proc. Sixth IJCAI, Tokyo, August, 1979, pp. 845-852. 4. Stefik, MJ.: Planning with constraints, (Ph.D. thesis). Stanford Computer Science Report, STAN-CS-80-784, March, 1980. Privileged Communication 313 E. H. Shortliffe Collaborative Project Abstracts Stanford Project: ONCOCIN -- KNOWLEDGE ENGINEERING FOR ONCOLOGY CHEMOTHERAPY CONSULTATION Principal Investigator: Edward H. Shortliffe, M.D., Ph.D. Departments of Medicine and Computer Science Stanford University Medical Center - Room TC135 Stanford, California 94305 (415) 497-6979 (SHORTLIFFE@SUMEX-AIM) Project Director: Dr. Lawrence M. Fagan The ONCOCIN Project is overseen by a collaborative group of physicians and computer scientists who are developing an intelligent system that uses the techniques of knowledge engineering to advise oncologists in the management of patients receiving cancer chemotherapy. The general research foci of the group members include knowledge acquisition, inexact reasoning, explanation, and the representation of time and of expert thinking patterns. Much of the work developed from research in the 1970's on the MYCIN and EMYCIN programs, early efforts that helped define the group's research directions for the coming decade. MYCIN and EMYCIN are still available on SUMEX for demonstration purposes. The prototype ONCOCIN system is in limited experimental use by oncologists in the Stanford Oncology Clinic. Thus much of the emphasis of this research has been on human engineering so that the physicians will accept the program as a useful adjunct to their patient care activities. OONCOCIN has generally been well-accepted since its introduction, and work is underway to transfer the program to professional workstations (rather than the central SUMEX computer) so that it can be implemented and evaluated at sites away from the University. SOFTWARE AVAILABLE ON SUMEX MYCIN-- A consultation system designed to assist physicians with the selection of antimicrobial therapy for severe infections. It has achieved expert level performance in formal evaluations of its ability to select therapy for bacteremia and meningitis. Although MYCIN is no longer the subject of an active research program, the system continues to be available on SUMEX for demonstration purposes and as a testing environment for other research projects. EMYCIN-- The “essential MYCIN” system is a generalization of the MYCIN knowledge representation and control structure. It is designed to facilitate the development of new expert consultation systems for both clinical and non-medical domains. ONCOCIN-- This system is in clinical use but is designed for special high speed terminals and therefore cannot be tested or demonstrated via network connections. Much of the knowledge in the domain of cancer chemotherapy is already well-specified in protocol documents, but expert judgments also need to be understood and modeled. E. H. Shortliffe 314 Privileged Communication Collaborative Project Abstracts REFERENCES 1. Shortliffe, E.H., Scott, A.C., Bischoff, M.B. Campbell, A.B., van Melle, W. and Jacobs, C.D: ONCOCIN: An expert system for oncology protocol management. Proc. Seventh IJCAI, pp. 876-881, Vancouver, B.C., August, 81. 2. Duda, R.O. and Shortliffe, E.H.: Expert systems research. Science 220:261-268, 1983. 3. Langlotz, C.P. and Shortliffe, EH: Adapting a consultation system to critique user plans. Int. J. Man-Machine Studies 19:479-496, 1983. 4. Bischoff, M.B., Shortliffe, E.H., Scott, A.C., Carlson, R.W. and Jacobs, C.D:: Integration of a computer-based consultant into the clinical Setting. Proceedings 7th Annual Symposium on Computer Applications in Medical Care, pp. 149-152, Baltimore, Maryland, October 1983. Privileged Communication 315 E. H. Shortliffe Collaborative Project Abstracts Stanford Project: PROTEAN Project Principal Investigators: Oleg Jardetzky (JARDETZKY@SUMEX-AIM) Nuclear Magnetic Resonance Lab, School of Medicine Stanford University Medical Center Stanford, California 94305 Bruce Buchanan, Ph.D. (BUCHANAN@SUMEX-AIM) Computer Science Department Stanford University Stanford, California 94305 The goals of this project are related both to biochemistry and artificial intelligence: (a) use existing AI methods to aid in the determination of the 3-dimensional structure of proteins in solution (not from x-ray crystallography proteins), and (b) use protein structure determination as a test problem for experiments with the AI problem solving structure known as the Blackboard Model. Empirical data from nuclear magnetic resonance (NMR) and other sources may provide enough constraints on structural descriptions to allow protein chemists to bypass the laborious methods of crystallizing a protein and using X-ray crystallography to determine its structure. This problem exhibits considerable complexity. Yet there is reason to believe that AI programs can be written that reason much as experts do to resolve these difficulties REFERENCES 1, Erman, L.D., Hayes-Roth, B., Lesser, V.R., Reddy, D.R:The HEARSAY-II Speech Understanding System: Integrating Knowledge to Resolve Uncertainty. ACM Computing Surveys 12(2):213-254, June, 1980. 2. Hayes-Roth, B.: The Blackboard Architecture: A General Framework for Problem Solving? Report HPP-83-30, Department of Computer Science, Stanford University, 1983. 3. Hayes-Roth, B: BB/: An Environment for Building Blackboard Systems that Control, Explain, and Learn about their own Behavior. Report HPP~84-16, Department of Computer Science, Stanford University, 1984. 4. Hayes-Roth, B.:A Blackboard Architecture for Control. Artificial Intelligence In Press, 1985. 5. Hayes-Roth, B. and Hewett, M.: Learning Control Heuristics in BBI. Report HPP-85-2, Department of Computer Science, 1985. 6. Jardetzky, O.: A Method for the Definition of the Solution Structure of Proteins from NMR and Other Physical Measurements: The LAC-Repressor Headpiece. Proceedings of the International Conference on the Frontiers of Biochemistry and Molecular Biology, Alma Alta, June 17-24, 1984, October, 1984. E. H. Shortliffe 316 Privileged Communication Collaborative Project Abstracts Stanford Project: RADIX -- DERIVING KNOWLEDGE FROM TIME-ORIENTED CLINICAL DATABASES Principal Investigators: Robert L. Blum, M.D. Departments of Medicine and Computer Science Stanford University Stanford, California 94305 (415) 497-9421 (BLUM@SUMEX-AIM) Gio C.M. Wiederhold, Ph.D. Department of Computer Science Stanford University Stanford, California 94305 (415) 497-0685 (WIEDERHOLD@SUMEX-AIM) The objective of clinical database (DB) systems is to derive medical knowledge from the Stored patient observations. However, the process of reliably deriving causal relationships has proven to be quite difficult because of the complexity of disease states and time relationships, strong sources of bias, and problems of missing and outlying data. The goal of the RADIX Project is to explore the usefulness of knowledge-based computational techniques in solving this problem of accurate knowledge inference from non-randomized, non-protocol patient records. Central to RADIX is a knowledge base (KB) of medicine and statistics, organized as a taxonomic tree consisting of frames with attached data and procedures. The KB is used to retrieve time-intervals of interest from the DB and to assist with the statistical analysis. Derived knowledge is incorporated automatically into the KB. The American Rheumatism Association DB containing records of 1700 patients is used. SOFTWARE AVAILABLE ON SUMEX RADIX--~(excluding the knowledge base and clinical database) consists of approximately 400 INTERLISP functions. The following groups of functions may be of interest apart from the RADIX environment: SPSS Interface Package -- Functions which create SPSS source decks and read SPSS listings from within INTERLISP. Statistical Tests in INTERLISP -- Translations of the Piezer-Pratt approximations for the T,F, and Chi-square tests into LISP. Time-Oriented Data Base and Graphics Package -- Autonomous package for maintaining a time-oriented database and displaying labelled time-intervals. Privileged Communication 317 E. H. Shortliffe Collaborative Project Abstracts REFERENCES Monograph Blum, R.L.: Discovery and representation of causal relationships from a large time-oriented clinical database: The RX project. IN D.A.B. Lindberg and P.L. Reichertz (Eds.), LECTURE NOTES IN MEDICAL INFORMATICS, Vol. 19, Springer-Verlag, New York, 1982. Journal Articles Blum, R.L.: Discovery, confirmation, and incorporation of causal relationships from a large time-oriented clinical database: The RX Project. Computers and Biomedical Research 15(2):164-187, April, 1982. Blum, R.L. Displaying clinical data from a time-oriented database. Computers in Biology and Medicine 11(4):197-210, 1981. Conference Proceeding Blum, R.L.: Modeling and encoding clinical causal relationships. Proc. SCAMC83, IEEE, Baltimore, MD, October, 1983. E. H. Shortliffe 318 Privileged Communication Collaborative Project Abstracts National AIM Project: CADUCEUS (formerly INTERNIST) Principal Investigators: Jack D. Myers, M.D. (MYERS@SUMEX-AIM) Harry E. Pople, Ph.D. (POPLE@SUMEX-AIM) University of Pittsburgh Pittsburgh, Pennsylvania 15261 Dr. Pople: (412) 624-3490 Dr. Myers: (412) 624-2649 The major goal of the CADUCEUS Project is to produce a reliable and adequately complete diagnostic consultative program in the field of internal medicine. Although this program is intended primarily to aid skilled internists in complicated medical probiems, the program may have spin-off as a diagnostic and triage aid to physicians’ assistants, rural health clinics, military medicine and space travel. In the design of CADUCEUS and its predecessor INTERNIST I, we have attempted to model the creative, problem-formulation aspect of the clinical reasoning process. The program employs a novel heuristic procedure that composes differential diagnoses, dynamically, on the basis of clinical evidence. During the course of a CADUCEUS or INTERNIST-1 consultation, it is not uncommon for a number of such conjectured problem foci to be proposed and investigated, with occasional major shifts taking place in the program's conceptualization of the task at hand. SOFTWARE AVAILABLE ON SUMEX Versions of INTERNIST are available for experimental use, but the project continues to be oriented primarily towards research and development; hence, a stable production version of the system is not yet available for general use. Privileged Communication 319 E. H. Shortliffe Collaborative Project Abstracts National AIM Project: CLIPR -- HIERARCHICAL MODELS OF HUMAN COGNITION Principal Investigators: Walter Kintsch, Ph.D. (KINTSCH@SUMEX-AIM) Peter G. Polson, Ph.D. (POLSON@SUMEX-AIM) Computer Laboratory for Instruction in Psychological Research (CLIPR) Department of Psychology University of Colorado Boulder, Colorado 80302 (303) 492-6991 Contact: Dr. Peter G. Polson (Polson@SUMEX-AIM) The CLIPR Project is concerned with the modeling of complex psychological processes. It is comprised of two research groups. The prose comprehension group has completed a project that carries out the text analysis described by van Dijk & Kintsch (1983) yielding predictions of the recall and readability of that text by human subjects. The human-computer interaction group is developing a quantitative theory of that predicts learning, transfer, and performance for a wide range of computer-tasks, e.g. text editing. SOFTWARE AVAILABLE ON SUMEX A set of programs has been developed to perform the microstructure text analysis described in van Dijk & Kintsch (1983) and Kintsch & Greeno (1985). The program accepts a propositionalized text as input, and produces indices that can be used to estimate the text's recall and readability. REFERENCES 1. Fletcher, R. C. Understanding and solving word arithmetic problems: A computer simulation. Technical Report NO. 135, Institute of Cognitive Science, Colorado, 1984. 2. Kieras, D.E. and Polson, P.G.: The formal analysis of user complexity. Int. J. Man-Machine Studies, In Press. 3. Kintsch, W. and van Dijk, T.A: Toward a model of text comprehension and production. Psychological Rev. 85:363-394, 1978. 4. Kintsch, W. and Greeno, J.G.:.Understanding and solving word arithmetic problems. Psychological Review, 1985, 92, 109-129. 5. Polson, P.G. and Kieras, D.E.: A formal description of users' knowledge of how to operate a device and user complexity. Behavior Research Methods, Instrumentation, & Computers, 1984, 16, 249-255, 6. Polson, P.G. and Kieras, D.E.: A quantitative model of the learning and performance of text editing knowledge. Proceedings of the CHI 1985 Conference on Human Factors in Computing. San Francisco, April 1985. 7. van Dijk, T.A. and = Kintsch, W.STRATEGIES OF DISCOURSE COMPREHENSION. Academic Press, New York, 1983. 8. Young, S. A theory and simulation of macrostructure. Technical Report No. 134, Institute of Cognitive Science, Colorado, 1984. E. H. Shortliffe 320 Privileged Communication Collaborative Project Abstracts 9. Walker, H.W. & KIntsch, W. Automatic and strategic aspects of knowledge retrieval. Cognitive Science, 1985, 9, 261-283. Privileged Communication 321 E. H. Shortliffe Collaborative Project Abstracts National AIM Project: MENTOR -- MEDICAL EVALUATION OF THERAPEUTIC ORDERS Principal Investigators: Stuart Speedie, Ph.D. (SPEEDIE@SUMEX-AIM) School of Pharmacy University of Maryland 20 N. Pine Street Baltimore, Maryland 21201 (301) 528-7650 Terrence F. Blaschke, M.D. (BLASCHKE@SUMEX-AIM) Department of Medicine Division of Clinical Pharmacology Stanford University Medical Center Stanford, California 94305 The goal of the MENTOR project is to implement and begin evaluation of a computer- based methodology for reducing therapeutic misadventures. The project will use principles of artificial intelligence to create an on-line expert system to continuously monitor the drug therapy of individual patients and generate specific warnings of potential and/or actual unintended effects of therapy. The appropriate patient information will be automatically acquired through interfaces to a hospital information system. This data will be monitored by a system that is capable of employing complex chains of reasoning to evaluate therapeutic decisions and arrive at valid conclusions in the context of all information available on the patient. The results reached by the system will be fed back to the responsible physicians to assist future decision making. Specific objectives of this proposal include: 1. Implement a prototype computer-based expert system to continuously monitor in- patient drug therapy. It will use a modular medical knowledge base and a separate inference engine to apply the knowledge to specific situations. 2. Select a small number of important and frequently occurring drug therapy problems that can lead to therapeutic misadventures and construct a comprehensive knowledge base necessary to detect these situations. 3. Design and begin implementation of an evaluation of the prototype MENTOR system with respect to its impact on the on the physicians’ therapeutic decision making as well as its effects on the patient in terms of specific mortality and morbidity measures. The work in the proposed project will build on the extensive previous work in drug monitoring done by these investigators in the Division of Clinical Pharmacology at Stanford and the University of Maryland School of Pharmacy. E. H. Shortliffe 322 Privileged Communication Collaborative Project Abstracts National AIM Project: SOLVER -- PROBLEM SOLVING EXPERTISE Principal Investigators: Paul E. Johnson, Ph.D., School of Management and Center for Research in Human Learning 205 Elliott Hail University of Minnesota Minneapolis, Minnesota 55455 (612) 376-2530 (PJOHNSON@SUMEX-AIM) William B. Thompson, Ph.D. Department of Computer Science 136 Lind Hall University of Minnesota Minneapolis, Minnesota 55455 (612) 373-0132 (THOMPSON@SUMEX-AIM) The Minnesota SOLVER project focuses upon the development of strategies for discovering and representing the knowledge and skill of expert problem solvers. Although in the last 15 years considerable progress has been made in synthesizing the expertise required for solving complex problems, most expert systems embody only a limited amount of expertise. What is still lacking is a theoretical framework capable of reducing dependence upon the expert's intuition or on the near exhaustive testing of possible organizations. Our methodology consists of: (1) extensive use of verbal thinking aloud protocols as a source of information from which to make inferences about underlying knowledge structures and processes; (2) development of computer models as a means of testing the adequacy of inferences derived from protocol studies: (3) testing and refinement of the cognitive models based upon the study of human and model performance in experimental settings. Currently, we are investigating problem- solving expertise in domains of medicine, financial auditing, management, and law. SOFTWARE AVAILABLE ON SUMEX A redesigned version of the Diagnoser simulation model, named Galen, has been implemented on SUMEX. REFERENCES 1. Johnson, P.E., Moen, J.B., and Thompson, W.B.: Garden Path Errors in Medical Diagnosis. IN Bloc, L. and Coombs, MJ. (Eds.), COMPUTER EXPERT SYSTEMS, Springer-Verlag (in press). 2. Johnson, P.E., Johnson, M.G., and Little, R.K.: Expertise in trial advocacy: Some considerations for inquiry into its nature and development, Campbell Law Review, (in press). 3. Johnson, P.E., “The Expert Mind: A New Challenge for the Information scientist,” in Beyond Productivity: Information System Development for Organizational Effectiveness, Th. M. A. Bemelmans (editor), Elsevier Science Publishers B. V. (North-Holland), 1984. 4. Johnson, P.E: What kind of expert should a system be? J. Medicine and Philosophy, 8:77-97, 1983. 5. Johnson, P.E., Duran, A., Hassebrock, F., Moller, J., Prietula, M., Feltovich, P. and Swanson, D.: Expertise and error in diagnostic reasoning. Cognitive Science 5:235-283, 1981. Privileged Communication 323 E. H. Shortliffe Collaborative Project Abstracts 6. Thompson, W.B., Johnson, P.E. and Moen, J.B.: Recognition-based diagnostic reasoning. Proc. Eighth IJCAI, Karlsruhe, West Germany, August, 1983. E. H. Shortliffe 324 Privileged Communication Collaborative Project Abstracts Stanford Pilot Project: THE COMPUTER-AIDED MEDICAL DECISION ANALYSIS (CAMDA) PROJECT Co-Principai Investigator: Samuel Holtzman Ronald A. Howard Department of Engineering-Economic Systems Stanford University Stanford, California 94305 Contact: Samuel Holtzman(HOLTZMAN@SUMEX-AIM) (415) 497-0486 The CAMDA project is a program of research in the area of medical decision making. The main focus of this effort is to combine decision analysis and artificial intelligence to develop systems that support medical decisions. Nearly two decades of experience in the application of decision analysis to problems in industry and government have shown that the technique constitutes an extremely helpful tool for making difficult choices. The potential benefit of decision analysis is particularly great when choices must be made in the presence of uncertainty and when the stakes involved are high. This situation is common in medical decisions. Partly as a result of the high cost of an individual decision analysis, and partly due to the inherent complexity of making choices which involve outcomes such as pain and death, medical decision analysis has remained essentially within the realm of the academic community. Therefore, the majority of patients and physicians have been deprived of the benefits of this powerful technique. Expert system technology makes it possible to bring decision analysis to the medical community in general. By providing a sophisticated modeling methodology, expert systems allow the process of decision analysis (within a specific medical context) to be formalized with sufficient accuracy to make much of the analysis amenable to computer automation. The resulting CAMDA systems could provide an attractive alternative to unaided decision making, and to the usually unaffordable option of analyzing medical decisions individually. Furthermore, these systems can help decision makers think more clearly about the difficult issues they face by providing them with a means to experiment with the logical consequences of their assumptions and preferences. A major focus of our research effort is the development of RACHEL, an intelligent decision system for infertile couples. The field of infertility was chosen for several reasons, including the prevalence of the condition, the complexity of the values that are usually attached to the possible outcomes in this field, the rapidly growing set of available tests and treatments, and the time-dependent nature of the human reproductive process. As part of the development of RACHEL, a substantial portion of the current CAMDA effort is aimed at the development of a general computer-based aid for medical decision analysis, which could be used in other medical decision domains. Privileged Communication 325 E. H. Shortliffe