Request for Use of the SUMEX-AIM System Submitted by: Alan M. Lesgold (412-624-4892) James G. Greeno (412-624-4885) Learning Research and Development Center University of Pittsburgh 3939 O'Hara Street Pittsburgh, PA 15260 Date submitted: 29 July 1977 LL. kage Danse 21 Lhsse Alan M. Lesgold sche es G. Greeno Abstract We are requesting an account in SUMEX-AIM for the purpose of further extending research on arithmetic and reading processes in children. This research is dependent upon the programs produced by the ACT project of John Anderson. Some of those programs have been imported to our own system at the University of Pittsburgh, but many of our needs cannot be handled without access to present and forthcoming versions of ACT that cannot be implemented here. The body of this request follows the form of the ‘questionnaire for potential users." A. MEDICAL AND COMPUTER SCIENCE GOALS 1. Describe the proposed research to be undertaken on the SUMEX-AIM resource a. Language processing in arithmetic Our research involves analysis of language processing and under- standing using production systems. The work to be done using SUMEX-AIM uses the ACT production system developed by Anderson and currently a SUMEX-AIM project. Collaborative work using the ACT system was begun when Anderson and Greeno were at Michigan in 1974-75. Greeno has developed a version of ACT for use in simulating geometry theorem proving (see attached papers; Greeno, 1976 and 1977). This version is now operating at the University of Pittsburgh. We have now begun developing models of language processing in arithmetic problem solving and in reading, using the system ACTE, a version of ACT that Anderson and Kline have made available and documented. Our preliminary work using the ACTE system has been conducted in SUMEX-AIM with a small directory of 50 pages under Anderson's account. Work on these models has been facilitated greatly by the ability to use Anderson's system through SUMEX-AIM, since we have not been required to translate the programs into UCI LISP, the version that is available through the University of Pittsburgh's computing center. Further work involving analysis of language processing will be similarly facilitated by use of the SUMEX-AIM resource, not only by continued use of the programs in their INTERLISP form, but also from the opportunity to use newer versions of ACT as they are developed in Anderson's project. The concepts that have been developed in Greeno's work on geometry using ACT include hypotheses about the nature of understanding in problem solving, representation of indefinite goal structures, and hypotheses about knowledge structures used by human problem solvers in producing simple constructions. The analysis of understanding processes has resulted in a problem-solving system called Perdix that generates a representation ef relations among components of the problem situation. The intuition expressed in this system is that understanding in problem solving is analogous to understanding language, where the understander constructs a representation that includes semantic relationships among the concepts that are mentioned in the linguistic input. Similarly, the representa- tion that Perdix generates in the process of solving a problem is one kind of semantic representation of the problem. A constraint on the repre- sentation of a problem is that it must include the relation that is given as the problem goal, but the representation includes other relations that are relevant to the problem goal, and shows how the various relations are interconnected. The representation of indefinite goal structures is relevant to the distinction between well~structured and ill-structured problems. In many ill-structured problems the problem goal is not specified in any definite way. A simple example of an indefinite goal occurs in geometry problems requiring proof that triangles are congruent. This can be accomplished in several alternative ways, and human problem solvers do not anticipate the specific pattern that is found in a given problem. This performance can be simulated in a straightforward way: The goal is represented by the problem solver as a pattern-recognition system. (In Perdix, it is organized as an EPAM-type net.) The extension of the theory of well-structured problem solving that is achieved by allowing goals to be represented as arbitrary pattern-recognition systems is quite modest, conceptually, but appears to accomplish a considerable expansion of the range of problems that can be encompassed by the theory. The third major aspect of Greeno's geometry project involves processes for generating constructions. This work is still in progress, but it appears that a satisfactory simulation of human problem-solving performance will be achieved by a system that selects a strategy for a problem using a top-down planning system similar to that of NOAH (Sacerdoti, 1974), and modifies the diagram using a schema-based discrepancy detecting system similar to that of MYCROFT (Goldstein, 1975). Solution of this problem will provide further insight into the nature of problem solving in Situations that have been considered to be ill structured, since these problems often require generation of materials needed for the solution of the problem as part of the problem-solving process. Further work on geometry theorem-proving is planned, involving collaboration with other investigators. A project for study of informal reasoning processes has been planned, which will involve collaboration with John Seely Brown and Richard Burton of Bolt, Beranek and Newman, Inc. We will study the interaction of formal, syntactic reasoning processes with spatial information processing that occurs in the domain of the semantic spatial models of geometric structure. Greeno's contribution to the project will include empirical studies and simulation models of knowledge about general concepts of proof, as well as simulations of informal reasoning processes. The theoretical work will involve further development of the Perdix problem-solving system, and thus will provide further development of problem-solving capabilities in the framework of ACT. An additional project has been discussed in a preliminary way. Anderson and Kline's current work developing a learning capability for ACT provides a promising possibility of collaborative work involving study of acquisition of problem-solving strategies. Anderson, Kline, and Greeno have begun to discuss the possibility of developing an analysis of strategic knowledge in theorem-proving, and attempting to simulate the acquisition of that knowledge using the ACT system. This work would require collaborative use of the adaptive ACTF system now being developed in Anderson's project. The research to be conducted on language processing in arithmetic is an attempt to analyze relationships involving language and computational procedures. We hope to be able to characterize what it means for a system to achieve understanding of a computational procedure, rather than merely knowing how to execute the procedure in a mechanical way. Thus, this work will draw on the results of earlier development of a system for solving problems with understanding. Winograd's (1972) results showed that pro- cedural representations can be developed in powerful ways to provide analyses of a broad range of knowledge structures that are communicated through language. Arithmetic word problems provide an interesting domain in which the process of understanding results in the execution of a well- specified set of procedures. The major analysis carried out previously on this problem was by Bobrow (1968), whose program used a minimum of semantic information in translating linguistic input into simultaneous equations. Our approach focuses attention on semantic processes, in the form of productions that perform inferences about the meanings of concepts and propositions. The system generates a structure of relatively abstract semantic relationships to represent the meaning of a word problem, and selects an arithmetic operation whose structure matches the meaning structure of the problem. With the limited space available in our present 50-page directory, we have constructed a set of productions that interprets the meaning of one kind of problem in which the answer is found by adding two given numbers. The problems that can be interpreted involve actions that increase the value of a quantitative variable. To test the validity of the approach, we will need to expand the system to include other semantic structures, such as combinations of sets and comparisons of quantities. These developments will require a moderate expansion of the disk space available for the work. With the space that we are requesting in this proposal, we anticipate that we can develop a system that is capable of interpreting the class of basic quantitative relationships that are ex- pressed in word problems given to children in their study of arithmetic through about the sixth grade. b. The reading project The purpose of the reading project is the building of a theory of reading that will guide the choice of reading instruction and especially reading practice activities for children who are having trouble learning to read. The primary goal is specification of post-primary-grades training principles for "normal" children in populations (e.g., urban) where reading is a problem. However, the nature of the theory and the modelling medium we propose will permit expansion to populations whose abnormalities can be characterized in terms of information processing speed and/or attentional variables. Use of the SUMEX-AIM system is proposed in order to obtain better access to Anderson's ACT system and to facilitate con- tinued interaction with that group. In the present project, the use of computer simulation is only a part (but an integral one) of a continuing theoretical and empirical analysis of the reading process wichh has been conducted under NIE con- tract support for several years. The work is best introduced in detail by a chapter written for the 1976 Carnegie Symposium (Perfetti & Lesgold, in press b, attached). Basically it is concerned with the reading process from several levels of analysis and is attempting to ascertain how sub- systems of the reading process interact. In this section, we give an overview of the project, then summarize ongoing empirical work, and finally discuss the simulation efforts for which SUMEX-AIM Support is being sought. Reading as an interaction of several levels of cognitive activity. The Perfetti-Lesgold model has much in common with other points of view, but is different in certain of its emphases. ——_ In essence, the model states that there are several different types or levels of thinking activity that interact to produce understanding of a written discourse. In order to understand the overall comprehension process, one must understand each level of component activity. Further, one must under- stand how these levels of processing interact. Research goals within this framework are determined recursively--preliminary hypotheses about the nature of component process interactions guide the choice of levels for further analysis. The two basic levels of analysis we currently foresee are the word recognition level and the level of sentence understanding in the context of preceding sentences. The word recognition level is an obvious choice, since "decoding" words from print is what distinguishes reading from other discourse understanding activities. It also is the level of the process that currently receives the brunt of curricular activity in schools. The sentence-comprehension-in-context level has been chosen for study for two reasons. First, it represents the point at which in- formation from a sentence is related to (understood in the context of) information acquired from earlier sentences and prior knowledge of the reader. Second, it is a plausible candidate for a level that interacts with word recognition processes. In claiming that word recognition and sentence encoding (compre- hending and remembering) processes interact, we are taking an increasingly common position on the nature of reading (Barron, personal communication; Levy, in press; Perfetti & Lesgold, in press a; Rumelhart, 1976). Further, earlier claims that reading is merely the addition of word recognition skills to the understanding skills that every child has or, alternatively, that since the fluent reader anticipates many words excessive emphasis ought not to be placed on word recognition have proven too simplistic. Neither level of processing clearly subsumes the other. What then is the nature of the interaction? Kahneman (1973) has suggested that processes which co-occur in time can interact in both 3 specific, structural ways and in a general (attentional) manner. The interactive reading model of Rumelhart (1976) deals with structural inter- actions. He is concerned with the way in which words may be recognized via of some combination of sensory activity (e.g., detecting fragments of letters) and semantic processes. That is, one need not be as sure of exactly what was seen if one knows what word "belongs" in a particular place in a sentence. Another form of interaction between subprocesses is nonspecific. Here, concern is not with the information one component sends to or receives from another but rather with the extent to which the general behavior of one component influences the success of another. One way in which this type of interaction occurs is the sharing of attentional resources. Human ability to perform attention-demanding tasks is quite limited. It is probably not possible for two attention-demanding processes to operate truly simultaneously (Newell & Simon, 1972; Schneider & Shiffrin, 1977). Just as in a computer, "simultaneous" processing of two tasks is probably done by time sharing, in which attention is directed alter- nately to the two tasks (see Moray, 1969). In the case of reading, for example, one could imagine attention being directed alternately to word recognition and understanding components of the process. This oscillation is quite apparent in the novice reader who alternately con- concentrates on sounding out a word and then on thinking about what he's read. The fact of time sharing would not be relevant to our under- Standing of reading problems if it meant only that processing levels successfully alternate. What is more relevant is our hypothesis that “Feading can fail even when the component processes of reading all work when tested separately. That is, characteristics of word recognition and higher level component processes may make time-sharing difficult or impossible. The basic problem, as we see it, is that each component needs to know what it should be doing when it gets its turn to function. Understanding processes need to know what was just understood a few instants ago and which words need to be understood now. Word recognition processes need to know which word is next to be read. The concern, then, is in how a component process keeps track of what it is to be doing while waiting its turn. How does the comprehension component keep track of where it is while it waits for a clause to be read by a word-processing component? This temporary memory problem has been studied extensively (Atkinson & Shiffrin, 1968; Craik & Levy, 1976; Miller, 1956). It is clear that this sort of temporary memory is limited, though we do not know exactly how it works. One hypothesis (Anderson, 1976) is that anything we know (which we will call a memory even if it is something we just found out a second ago) is in an inactive state unless it has very recently been activated. Memories become active when other memories to which they are closely associated are active. They also are active at the time they are first established. A memory's activity level fades rapidly, and the only way it can be reinstated instantly is if some mental process uses it. Sometimes, we invent mental processes solely to retain memory in an active state. For example, when we say the digits of a phone number over again as the phone is dialed, we are merely holding the memory for the number active, in case a busy signal requires redialing. These 10 rehearsal processes also take time, however, so they are not too helpful in solving memory problems due to time sharing. This time sharing and memory activation framework can be illustrated by a description of a common classroom sight. A beginning reader is trying to read the fifth word of a sentence. It is a hard word, so he has to try several different decoding strategies before he succeeds in reading it. To keep his memory for phonemes he has recognized from deactivating, he says them over and over to himself. Finally, he reads the whole word, but by then he has forgotten the earlier words, so he has to reread the sentence. It is not unknown for the word to be forgotten while the sentence is being re-read! (The man went to kuh, chu, chi, chi, chick, chicka, chicka, chicka, chicka, oh! Chicago! The man went to...Chi... Chicago). We can now restate the problem of general interactions of levels of the reading process as follows. Attention alternates between word recognition and sentence comprehension levels of activity. While there may be structural interactions (e.g., context guiding recognition) there is also general interaction. Here the problem is for each component to be fast enough so that other components don't “lose” essential memories while one component is trying to do its job. That is, the levels of processing must be synchronized with respect to the properties of temporary memory activation. The Chicago example just stated is a reminder that this synchrony is lacking in beginning readers. Our goal is to understand component synchrony in reading and to determine how it develops. We think there are three basic sources of synchronous reading activity. First, parts of the reading process can be automated. It appears that automated processes occur in parallel with the time-shared, limited capacity, conscious level of processing. Thus, they solve synchrony problems by getting more 11 done before needed memories deactivate. A second source of synchrony lies in the overall design of component processes. For example, compre- hension processes can feed forward a good guess about what the next word 4o be read might be. In this case, one process is generating the data that the next process needs to operate efficiently. Lower level processes can feed forward into higher level processes to a slight extent, but not as extensively as in the other (top-down) direction. A third source of synchrony is the discourse itself. Some discourses are so structured that only a small amount of information need be kept active at any time for comprehension to be successful. Such discourses will probably be better understood by the novice reader. Before describing specific work plans we will briefly comment in turn on subprocess automation, feed forward processes, and the structure of discourse. In recent years, automation of simple perception and comparison processes has been studied extensively. LaBerge and Samuels (1974), for example, proposed that word recognition processes can become automated through a series of stages. Initially, words are processed letter by letter and later in multi-letter units. Finally, words are recognized automatically and quickly, avoiding the processing capacity costs involved in putting together pieces of a word. Johnson (197 ) has presented related data, showing a progression from letter-by-letter to holistic, automated recognition. At the subword level, poor third-grade readers have been shown to have less ability to make fast, accurate syllable recognitions (Scheerer-Neumann, in press). It seems worthwhile to carefully examine the development of automated word and subword recognition in children as they learn how to read. Work to be done in this domain is described below. 12 It is important to realize that process automation is a better solution to reading problems when applied to lower-level decoding pro- cesses than when applied to higher level comprehension processes. This ie-because automation has both good and bad points (Schneider & Shriffin, .+-1977). Automated processes do not require attention nor use short-term processing capacity. They can operate in parallel with attention demanding processes. They require extensive practice to be learned and even more to unlearn. They are inflexible and not easily applied to novel situations. Controlled processes require attention but are modifiable easily, since they are under conscious control. Thus they permit more flexibility in processing. there is little flexibility required in perceiving a word; hence word perception is a sensible process to automate. On the other hand, nuances of meaning could be buried by automated (stereotypic) functioning at higher levels. Consequently, we have not addressed our automation studies beyond the word recognition level. It is necessary to study higher level processes for a different reason, though. In order to verify our hypotheses about the effects of unautomated word recognition on comprehension, we need to show that compre- hension of a sentence is enhanced when the memories to which it should be related are active at the time the sentence is read. Thus, we are con- ducting a series of studies showing the effects of activation level of relevant memory on comprehension. Assuming these studies come out as expected, it will be apparent that the structure of discourse exerts great influence on the extent to which word recognition expertise relates to effectiveness of comprehension. If there is nothing in preceding text to which a given sentence is relevant, there should be no memory activation problem since there will be no relevant memory. Similarly, when only a very small amount of very specific immediately prior information is relevant, it may stay active even in the poor word processor. We ‘will also be conducting some studies of ways in which discourse structure and even illustrations can be used to make information more available to a slow reader by minimizing memory activation —: ‘problems. The third possible solution to the asynchrony problems we have been discussing is to increase the extent to which comprehension feeds forward into word recognition. This approach will require a better understanding of specific interactions between comprehension and decoding of the sort being carried out by Rumelhart (1976). We have no immediate plans to work in this area but do realize that Rumelhart's work and related work of others is complementary to our own. Hence, any prescriptions we make for instructional practice will be sensitive to other research than ours. Empirical work. Our strategy is to first establish a number of basic operating characteristics of the discourse understanding processes that are involved in raading. This is being done by literature search supplemented by experiments as needed. Simultaneously, we are studying the development of word recognition skills in children as they learn how to read. We will be combining our analysis of both word processing and dis- course processing levels through the use of Anderson's ACT system. Since ACT is a plausible general modelling language for human cognitive processes, it will be possible to simulate the interaction of word recognition and understanding processes rather naturally. First, though, we need to tie down these processes so that they can form the basis for the simulation. In the area of discourse processing skills, we need to further study memory activation effects, as these are crucial to the asynchronies we believe are involved when inefficient word recognition leads to 14 incomplete comprehension. Ordinarily, in skilled readers, the main source of control over activation processes is the discourse structure itself, we hypothesize. Consequently, we have been conducting a variety of experiments “th which discourse is manipulated as we watch for changes in how it is processed by readers. The present and future work of this subproject is described in two attached documents (Lesgold & Perfetti; submitted; LRDC proposal to NIE, pp. 77~114). The longitudinal study, also described at length in the attached proposal excerpt (LRDC proposal to NIE), is a three year study of changes in childrens word recognition skills as they advance through a reading curriculum. The curriculum is individualized, so that it is possible to test children at points determined by their level of reading achievement. Thus, speed of visual and verbal word-coding performances is being related to level ani rate of reading achievement. At each test point, children also read a number of short passages aloud, providing a basis for an oral reading error analysis. This error analysis will form the basis of later tests of the decoding skills we build into our proposed model. The speed-of-processing data will also constrain our simulations of different levels of decoding expertise in children. Simulation plans. While the work described above is important for the development and application of the interactive model we described, it deals primarily with pieces of the model. We would also like to deal with the model as a whole. To do this we will, over the next three to five years, develop a computer simulation of enough of the components of the reading process to provide a rigorous test of it. This simulation effort may also generate additional experimental work, particularly in the individual differences approach. Our goal is to be able to predict qualitative aspects of comprehension from measurements of decoding facility. 15 We have already begun this work, using Anderson's (1976) ACT system, which is well-suited to our needs. ACT is a production system language for simulating cognition, but it is also a theory of cognition. In ACT, thinking takes place when productions execute. A production is a mental operation that is performed only if a particular memory pattern is already in memory. In ACT, though, only part of long-term memory is active, i.e., capable of satisfying a production's memory conditions. Some long-term memories become active because they are represented in short-term memory. Other activate because they are connected to currently active memory structures. Memories stay active only if they are, in some sense, represented in the limited capacity short-term store. If a fact has been comprehended from an earlier portion of a discourse but has not been referred to in subsequent portions, it will probably not be activated and hence will be less usable for the guidance of the comprehension process. Further, to the extent that information is forced into short-term meory, existing codes will be foreed out. Thus, when we change the subject in a discourse, the old subject and its related memories tend to lose activation. Similarly, new facts are related to old knowledge only if something causes the old knowledge to be activated. In ACT, then, the result of effective Socratic dialogue is a pattern of comprehension in which "appropriate" prior knowledge is activated by questions at just the time it is needed for more complete understanding of a new fact. Anderson's (1976) work has guided our thinking, and the ACT model has the capability of modelling activation processes nicely built in. Thus, it will be practical to model some of the ways in which foregrounding influences comprehension. Our primary concern, though, is with the effects of slow or less automatic and wholistic word recognition on comprehension. ACT seems capable of modelling these effects, also. Because of its design, ACT 16 has the property that the same decoding good readers do with a few complex productions requires many simple productions in poorer readers, then comprehension productions may be executed more slowly or with lower probability. Slower execution can result in less integration of related facts because of the activation problem. And, of course, failing to execute comprehension productions will also be detrimental. Both speed and quality of the decoding process can be varied in our planned model, with resulting comprehension effects. Specifically, a production executes only if its condition has been satisfied for a certain amount of time. The waiting period is longer for weaker productions. Overall, it is also increased as the number of productions in the queue increases. Essentially, then, ACT is a limited capacity processor with priority queueing. Priority is determined by production strength, and productions become stronger with practice. Applying this sort of reasoning to reader ability differences, one would argue that readers who have to go through more productions to achieve decoding will use up production execution capacity that would otherwise be allocated to weaker, but still relevant comprehension productions. One could imagine such a state of affairs resulting in less processing of details and of nuances of meaning. Also, an overall delay in production execution could result in processing being terminated by time constraints before preliminary products of comprehension could be further analyzed. Finally, the knowledge encoded by the poor reader from a test would be less well integrated because recently encoded facts would fade (deactivate) before new facts could be tied to them. This, in turn would have cumulative effect, since, in ACT, activation spreads faster in strong, tightly-connected memories. The faster activation spreads, relative to the rate of activation dampening, the more facts can be simultaneously active. 17 At this point, we have begun to simulate word recognition processes in ACT via our limited access to Anderson's account. We have convinced our- selves that the overall project is manageable, so long as we take the input _ fg, our system to be pre-recognized graphemic units. For our purposes, this will be adequate, as our concern is with the interactions between levels of processing and not with the decoding level's detailed functions per se. We can simulate visual whole-word and syllable recognition without having to specify the mechanics of subletter feature recognition. Given this approach, it is still possible to effectively simulate the word recognition skill. The primary method will involve use of ACTF, the version Anderson will finish this summer. That version permits modelling of learning of productions and differential strength of productions. The other two envisioned components for the model are a sentence parser and a proposition processor. The sentence parser will operate more or less in the manner of examples given in Anderson (1976). We have already produced a simple debugged version of this component, but we will want to expand it to accommodate a richer variety of sentence types. Also, we still rely too heavily on control variables (see Anderson, 1976), which do not fit very closely into existing models of human cognition. Thus, we will want to change to more human mechanisms of process control. The parts of the model worked on so far would be sufficient, if fleshed out a bit, to permit input of a string of graphemes and produce out- put of parsed simple sentences. What needs to be added is simulation of the process whereby the meaning of one sentence is integrated with what has already been understood from a text and with relevant prior knowledge. This part of the project has not been begun, though we know something of the form it might take. In the first approximation, simple productions will connect nodes in the nétwork structure that represent the current sentence to related 18 or equivalent nodes in the active part of memory. This will eventually be supplemented by productions that follow an agenda based upon the structure rules for discourse that children seem to know. The final stage in the process will be the use of the model to predict the kinds of knowledge that will be learned from text as a function of the decoding skills and memory operating parameters of the child being simulated. While we will actually run these simulations on a limited basis, it is not likely that a major operating system will be run for simulation at a statistical/quantitative level. Rather, the simulation undertaking is viewed as a means for structuring the further development of an inter- active model of reading and for generating, via simulation runs, information on the implications of particular hypotheses for the overall theory. 2. Research support The availability of SUMEX~AIM is not indicated in any proposals we are connected with. Our work is supported by contracts between National Institute of Education, DHEW, and Learning Research and Development Center, University of Pittsburgh. The current contract runs through November 30, 1980.