A research center for augmenting human intellect* by DOUGLAS C. ENGELBART and WILLIAM K. ENGLISH Stanford Research Institute Menlo Park, California 1 SUMMARY la This paper describes a multisponsor re- search center at Stanford Research Institute in man-computer interaction. jai For its laboratory facility, the Center has a time-sharing computer (65K, 24-bit core) with a 4.5 megabyte swapping drum and a 96 megabyte file-storage disk. This serves twelve CRT work stations simultane- ously. lJaia Specjal hardware completely removes from the CPU the burden of display re- freshing and input sampling, even though these are done directly out of and into core. 1a1b The display in a user’s office appears on a high-resolution (875-line) commercial television monitor, and provides both char- acter and vector portrayals. A relatively standard typewriter keyboard is supple- mented by a five-key handset used (option- ally) for entry of control codes and brief literals. An SRI cursor device called the “mouse” is used for screen pointing and selection. Ja1bi1 The “mouse” is a hand-held X-Y transducer usable on any flat surface; it is described in greater detail further on. 1a2 Special-purpose high-level languages and associated compilers provide rapid, flexible development and modification of the reper- toire of service functions and of their control procedures (the latter being the detailed user *Principal sponsors are: Advanced Research Projects Agency and National. Aeronautics and Space Agency (NAS1~7897), and Rome Air Development Center F30602-68-C-0286. 2 395 Reprinted from — AFIPS — Conference Proceedings, Volume 33 Copyright © 1968 by Thompson Book Company 383 National Press Building Washington, D.C. 20004 actions and computer feedback involved in controlling the application of these service functions). 1b User files are organized as_ hierarchical structures of data entities, each composed of arbitrary combinations of text and figures. A repertoire of coordinated service features en- ables a skilled user to compose, study, and mod- ify these files with great speed and flexibility, and to have searches, analyses data manipula- tion, etc. executed. In particular, special sets of conventions, functions, and working methods have been developed to air programming, logi- cal design, documentation, retrieval, project management, team interaction, and hard-copy production. INTRODUCTION 2a In the Augmented Human Intellect (AHI) Research Center at Stanford Research Institute a group of researchers is developing an experi- mental laboratory around an interactive, multi- console computer-display system, and is work- ing to learn the principles by which interactive computer aids can augment their intellectual capability. 2b The research objective is to develop prin- ciples and techniques for designing an “aug- mentation system.” 2b1 This includes concern not only for the technology of providing interactive computer service, but also for changes both in ways of conceptualizing, visualizing, and organizing working material, and in procedures and methods for working individually and coop- eratively. APRS 1974 3954 396 ~=Fall Joint Computer Conference, 1968 2c The research approach is strongly empirical. At the workplace of each member of the subject group we aim to provide nearly full-time avail- ability of a CRT work station, and then to work continuously to improve both the service avail- able at the stations and the aggregate value de- rived therefrom by the group over the entire range of its roles and activities. 2d Thus the research group is also the subject group in the experiment. 2d1 Among the special activities of the group are the evolutionary development of a com- plex hardware-software system, the design of new task procedures for the system’s users, and careful documentation of the evolving system designs and user procedures. 2d2 The group also has the usual activities of managing its activities, keeping up with outside developments, publishing reports, etc. 2d8 Hence, the particulars of the augmenta- tion system evolving here will reflect the na- ture of these tasks—i.e., the system is aimed at augmenting a system-development project team. Though the primary research goal is to develop principles of analysis and design so as to understand how to augment human capability, choosing the researchers them- selves as subjects yields as valuable secondary benefit a system tailored to help develop com- plex computer-based systems. 2e This “bootstrap” group has the interesting (recursive) assignment of developing tools and techniques to make it more effective at carrying out its assignment. 2e1 Its tangible product is a developing aug- mentation system to provide increased capa- bility for developing and studying augmenta- tion systems. 2e2 This system can hopefully be transferred, as a whole or by pieces of concept, principle and technique, to help others develop augmen- tation systems for aiding many other dis- ciplines and activities. 2f In other words we are concentrating fully upon reaching the point where we can do all of our work on line—placing in computer store all of our specifications, plans, designs, pro- grams, documentation, reports, memos, bibliog- raphy and reference notes, etc., and doing all of our scratch work, planning, designing, de- bugging, etc., and a good deal of our intercom- munication, via the consoles, 2f1 We are trying to maximize the coverage of our documentation, using it as a dynamic and plastic structure that we continually de- velop and alter to represent the current state of our evolving goals, plans, progress, knowl- edge, designs, procedures, and data. 2g The display-computer system to support this experiment is just (at this writing) becoming operational. Its functional features serve a basic display-oriented user system that we have evolved over five years and through three other computers. Below are described the principal features of these systems. THE USER SYSTEM 8a _ Basic Facility 3a1 As “seen” by the user, the basic facility has the following characteristics: 3aia 12 CRT consoles, of which 10 are normally located in offices of AHI research staff. 8aib The consoles are served by an SDS 940 time-sharing computer dedicated to full-time service for this staff, and each console may operate entirely independently of the others. 8aiec Each individual has private file space, and the group has community space, on a high-speed disc with a capacity of 96 mil- lion characters. 3a2 The system is not intended to serve a general community of time-sharing users, but is being shaped in its entire design toward the special needs of the “bootstrapping” ex- periment, 8b Work Stations 8b1 As noted above, each work station is equipped with a display, an alphanumeric keyboard, a mouse, and a five-key handset. 862 The display at each of the work stations (see Figure 1) is provided on a high-resolu- tion, closed-circuit television monitor. Research Center for Augmenting Human Intellect 397 FIGURE 2—Underside of mouse relative speed and accuracy obtained with this and other selection devices showed the mouse to be better than a light pen or a joy- stick (see Refs. English 1 and English 2). 3b4c1 Compared to a light pen, it is gen- erally less awkward and fatiguing to use, and it has a decided advantage for use with raster-scan, write-through storage tube, projection, or multiviewer display systems. 3b5 The five-key handset has 31 chords or FIGURE 1—Typical work station, with TV display, typewriter unique key-stroke combinations, in five keyboard, mouse, and chord handset “cases.” 803 The alphanumeric keyboard is similar to a Teletype keyboard. It has 96 normal char- acters in two cases. A third-case shift key provides for future expansion, and two spe- cial keys are used for system control. 8b4 The mouse produces two analog voltages ast het wo wheels (see Figure 2) rotate, each changing in proportion to the X or Y move- ment over the table top. 8b4a These voltages control—via an A/D converter, the computer’s memory, and the display generator—the coordinates of a tracking spot with which the user may “noint” to positions on the screen. 3640 Three buttons on top of the mouse are used for special control. 3b4c A set of experiments, comparing (within our techniques of interaction) the 3b5a The first four cases contain lower- and upper-case letters and punctuation, digits, and special characters. (The chords for the letters correspond to the binary numbers from 1 to 26.) 3656 The fifth case is “control case.” A particular chord (the same chord in each case) will always transfer subsequent in- put-chord interpretations to control case. 3b5e¢ In control case, one can “backspace” through recent input, specify underlining for subsequent input, transfer to another case, visit another case for one character or one word, etc. 365d One-handed typing with the handset is slower than two-handed typing with the standard keyboard. However, when the user works with one hand on the handset and one on the mouse, the coordinated in- 398 Fall Joint Computer Conference, 1968 terspersion of control characters and short literal strings from one hand with mouse- control actions from the other yields con- siderable advantage in speed and smooth- ness of operation. 8b5d1 For literal strings longer than about ten characters, one tends to trans- fer from the handset to the normal key- board. 3b5d2 Both from general experience and from specific experiment, it seems that enough handset skill to make its use worthwhile can generally be achieved with about five hours of practice. Be- yond this, skill grows with usage. 8c Structure of Files 8c Our working information is organized into files, with flexible means for users to set up indices and directories, and to hop from file to file by display-selection or by typed-in filename designations. Each file is highly structured in its internal organization. 8cia The specific structure of a given file is determined by the user, and is an im- portant part of his conceptual and “study- manipulate” treatment of the file. 8c2 The introduction of explicit “structur- ing” to our working information stems from a very basic feature of our conceptual frame- work (see Refs, Engelbart1 and Engelbart2) regarding means for augmenting human in- tellect. 8c2a With the view that the symbols one works with are supposed to represent a mapping of one’s associated concepts, and further that one’s concepts exist in a “net- work” of relationships as opposed to the essentially linear form of actual printed records, it was decided that the concept- manipulation aids derivable from real-time computer support could be appreciably en- hanced by structuring conventions that would make explicit (for both the user and the computer) the various types of network relationships among concepts. 8c2b As an experiment with this concept, we adopted some years ago the convention of organizing all information into explicit hierarchical structures, with provisions for arbitrary cross-referencing among the ele- ments of a hierarchy. 3c2b1 The principal manifestation of this hierarchical structure is the break- ing up of text into arbitrary segments called “statements,” each of which bears a number showing its serial location in the text and its “level” in an “outline” of the text. This paper is an example of hierarchical text structure. 3cZe To set up a reference link from State- ment A to Statement B, one may refer in Statement A either to the location number of B or to the “name” of B. The difference is that the number is vulnerable to subse- quent structural change, whereas the name stays with the statement through changes in the structure around it. 3c2c1 By convention, the first word of a statement is treated as the name of the statement, if it is enclosed in paren- theses. For instance, Statement O on the screen of Figure 1 is named “FJCC.” 8c2c2 References to these names may be embedded anywhere in other statements, for instance as “see(AFI),” where spe- cial format informs the viewer explicitly that this refers to a statement named “AFI,” or merely as a string of char- acters in a context such that the viewer can infer the referencing. 8e2c3 This naminga nd linking, when added to the basic hierarchical form, yields a highly flexible general structur- ing capability. These structuring con- ventions are expected to evolve relatively rapidly as our research progresses. 8c3 For some material, the structured- statement form may be undesirable. In these cases, there are means for suppress- ing the special formatting in the final print- out of the structured text. 8c4 The basic validity of the structured- text approach has been well established by our subsequent experience. 3c4a We have found that in both off-line and on-line computer aids, the concep- Research Center for Augmenting Human Intellect 399 tion, stipulation, and execution of sig- nificant manipulations are made much easier by the structuring conventions. 8c46 Also, in working on line at a CRT console, not only is manipulation made much easier and more powerful by the structure, but a user’s ability to get about very quickly within his data, and to have special “views” of it generated to suit his need, are significantly aided by the structure. 3c4e We have come to write all of our documentation, notes, reports, and pro- posals according to these conventions, because of the resulting increase in our ability to study and manipulate them during composition, modification, and usage. Our programming systems also incorporate the conventions. We have found it to be fairly universal that after an initial period of negative reaction in reading explicitly structured material, one comes to prefer it to material printed in the normal form. 3d File Studying 3d1 The computer aids are used for two prin- cipal “studying” operations, both concerned with construction of the user’s “views,” i.e., the portion of his working text that he sees on the screen at a given moment. 8dia Display Start 3dial The first operation is finding a particular statement in the file (called the “display start”); the view will then begin with that statement. This is equiva- lent to finding the beginning of a par- ticular passage in a hard-copy document. 8dib Form of View 8d1b1 The second operation is the speci- fication of a “form” of view—it may simply consist of a screenful of text which sequentially follows the point spec- ified as the display start, or it may be constructed in other ways, frequently so as to give the effect of an outline, Sdic In normal, off-line document study- ing, one often does the first type of opera- tion, but the second is like a sissors-and- staple job and is rarely done just to aid one’s studying. Sdid (A third type of service operation that will undoubtedly be of significant aid to studying is question answering. We do not have this type of service.) 38d2 Specification of Display Start 8d2a The display start may be specified in several ways: 8d2a1 By direct selection of a statement which is on the display—the user simply points to any character in the statement, using the mouse. 3d2a2 If the desired display start is not on the display, it may be selected in- directly if it bears a “marker.” 38d2a2a Markers are normally invisi- ble. A marker has a name of up to five characters, and is attached to a char- acter of the text. Referring to the marker by name (while holding down a special button) is exactly equivalent to pointing to the character with the mouse, 38d2a2b The control procedures make it extremely quick and easy to fix and call markers. 8d2a3 By furnishing either the name or the location number of the statement, which can be done in either of two basic ways: 8d2a3a Typing from the keyboard 8d2a3b Selecting an occurrence of the name or number in the text. This may be done either directly or via an in- direct marker selection. 3d2b After identifying a statement by one of the above means, the user may request to be taken directly there for his next view. Alternately, he may request instead that he be taken to some statement bearing a specified structure relationship to the one specifically identified. For instance, when the user identifies Statement 3E4 by one of the above means (assume it to be a member of the list 83E1 through 3E7), he may ask to be taken to 400 Fall Joint Computer Conference, 1968 8d2b1 Its successor, i.e., Statement 3E5 8d2b2 Its predecessor, i.e., Statement 3E3 34263 Ita list tail, ie, Statement 3E7 8d2b4 Its list head, i.e., Statement 3E1 $4265 Its list source, i.e., Statement 3E 8@2b6 Its subhead, i.e., Statement 3E4A 8d2c Besides being taken to an explicitly identified statement, a user may ask to go to the first statement in the file (or the next after the current location) that cen- tains a specified word or string of char- acters. 3d2c1 He may specify the search string by typing it in, by direct (mouse) selec- tion, or by indirect (marker) selection. 3d3 Specification of Form of View $d3a The “normal” view beginning at a given location is like a frame cut out from a long scroll upon which the hierarchical set of statements is printed in sequential order. Such a view is displayed in Figure 1. 8d3b Otherwise, three independently vari- able view-specification conditions may be applied to the construction of the displayed view: level clipping, line truncation, and content filtering. The view is simultaneous- ly affected by all three of these. 8d3b1 Level: Given a specified level parameter, L (L—1, 2,..., ALL), the view generator will display only those statements whose “depth” is less than or equal to L. (For example, Statement 3E4 is third level, 83E second, 4B2C1 fifth, etc.) Thus it is possible to see only first- level statements, or only first-, second-, and third level statements, for example. 8d362 Truncation: Given a specified truncation parameter, T (T —1, 2,..., ALL), the view generator will show only the first T lines of each statement being displayed. 8d8b3 Content: Given a specification for desired content (written in a special high-level content-analysis language) the view generator optionally can be directed to display only those statements that have the specified content. 8d$b8a One can specify simple strings, or logical combinations thereof, or such things as having the word “memory” within four words of the word “alloca- tion.” 8d3b8b Content specifications are writ- ten as text, anywhere in the file. Thus the full power of the system may be used for composing and modifying them. 3d3b3c Any one content specification can then be chosen for application (by select- ing it directly or indirectly). It is com- piled immediately to produce a machine- code content-analysis routine, which is then ready to “filter” statements for the view generator. 8d8ec In addition, the following format fea- tures of the display may be independently varied: indentation of statements accord- ing to level, suppression of location num- bers and/or names of statements, and sepa- ration of statements by blank lines. $d3d.The user controls these view speci- fications by means of brief, mnemonic char- acter codes. A skilled user will readjust his view to suit immediate needs very quickly and frequently; for example, he may change level and truncation settings several times in as many seconds. 8d4 “Freezing” Statements 8d4a One may also pre-empt an arbitrary amount of the upper portion of the screen for holding a collection of “frozen” state- ments. The remaining lower portion is treated as a reduced-size scanning frame, and the view generator follows the same rules for filling it as described above. 3di4b The frozen statements may be inde- pendently chosen or dismissed, each may have line truncation independent of the rest, and the order in which they are dis- played is arbitrary and readily changed. Any screen-select operand for any com- mand may be selected from any portion of the display (including the frozen state- ments). Research Center for Augmenting Human Intellect 401 8d5 Examples 3d5a Figures 3 and 4 show views generated from the same starting point with different level-clipping parameters. This example hap- pens to be of a program written in our Ma- chine-Oriented language (MOL, see below). 8d5b Figure 5, demonstrates the freezing feature with a view of a program (the same one shown in Figure 8) written in our Con- trol Metalanguage (CML, see below). State- ments 3C, 3C2, 2B, 2B1, 2B2, 2B3, and 2B4 are frozen, and statements from 2J on are shown normally with L —3, T = 1. 8d5b1 The freezing here was used to hold for simultaneous view four different func- tionally related process descriptions. The subroutines (+ BUGISPEC) and sat tersssae tenths eos Patents STATEREK! r DOCU NemNR a0, meMARAA, KacHiad, LINECEOT, Lt RCLANS Un teal, LISTE. Ne leRsd, LF 1STE, CHE 19SR, SPACEL ORs BeCLAMe Stam nt anne +, PRERED Se; Se OR t FIGURE 3—View of an MOL program, with level parameter set to 3 and truncation to | PDWRMAAT PROCEDURES SCRE; DECLARE KCHARS ED, MCMAR EG, vAXcHSEO. UINEL EOS. fy GECCARE EXTERWAL LISTS! NuINEsO, LPS TSI@, CRS ISSR, SPACE: OB: DECKARE STAR: A ARPOU. +) PREKED- 6) IP PRERED THEN meee PEERED: By RETLAOLLTMECNCHAR ST END IF WOHAR GE ROHAN Tae eel ~ POR 1 PRON O 2C 1 TO MARCH DO ROAR © HAICH: Oro wes (thy MOHAR © 1 imgts IF UIST THEN HAA > 01 AP LINE ERD RUSE BLP CHAR: eT mL NEL NCHAR! : FIGURE 4—Same program as Figure 3, but with level parameier changed to 6 (several levels still remain hidden from view) 42 te? SSEDWOLSPULOLE EC am RLS Cesk eOan «Rt SISPEO & {ALPES PL Pe ky 2m (*RUSISPECT fA. CASE 2B1 $Cb) gnto [5° 282 BLS) Sh GS) AP swArT RETRY ' 23 14WALT) BPO. CASE 2d) 106) sore Cs} 23d (CA) RETURN 243 ENDCASE REPEAT Of} 2K (*GEL) @R1 50 STB © SF(BH) F3.F4 SECHET: GELPTRIPT #2) Goro [8! 20 C4WORI LAL AZ, AJ. AGAS) CH GLO TAS+As ISP TAS CLAS) SAMPLE WORDS 2H CAINSRT) SPECIE sabsiRBT, TRUE) Litas S$ EW f> PLO‘ FIGURE 5—View of CML program, showing six frozen state- ments and illustrating use of reference hopping (+ WAIT were located by use of the hop-to-name feature described above, File Modification 3e1 Here we use a standard set of editing operations, specifying with each operation a particular type of text entity. 3ei1a Operations: Delete, Insert, Replace, Move, Copy. 8e1b Entities (within text of statements) : Character, Text (arbitrary strings), Word, Visible (print string), Invisible (gap string). 3ele Entities (for structure manipula- tion): Statement, Branch (statement plus all substructure), Group (sublist of branch- es), Plex (complete list of branches). 3e2 Structure may also be modified by join- ing statements, or breaking a statement into two at a specified point. 3e3 Generally, an operation and an entity make up a command, such as “Delete Word.” To specify the command, the user types the first letter of each word in the command: thus “DW” specifies ‘Delete Word.” There are occasional cases where a third word is used or where the first letter cannot be used because of ambiguities. 3f File Output 3f1 Files may be sent to any of a number of different output devices to produce hard copy—an upper/lower-case line printer, an 402 Fall Joint Computer Conference, 1968 on-line high-quality typewriter, or paper tape to drive various typewriters. 8fia In future it will be possible to send files via magnetic tape to an off-line CRT- to-film system from which we can produce Xérox prints, Multilith masters, or micro- form records. 8f2 Flexible format control may be exercised in this process by means of specially coded directives embedded in the files—running headers, page numbering, line lengths, line centering, suppression of location numbers, indenting, right justification (hyphenless), ete., are controllable features. Sg Compiling and Debugging $91 Source code files written in any of our 8i3 We are also experimenting with it in compiler languages (see below), or in the SDS 940 assembly language (ARPAS, in FIGURE 6—On-line conference arrangement project meetings, using it not only to keep track of agenda items and changes but also which our compiler output is produced) may to log progress notes, action notes, etc. The be compiled under on-line control. For de- review aid is of course highly useful here bugging, we have made a trivial addition to also. the SDS 940’s DDT loader-debugger so as to operate it from the CRT displays. Though it $14 We are anxious to see what special con- was designed to operate from a Teletype ventions and procedures will evolve to allow terminal, this system gains a great deal in us to harness a number of independent con- speed and power by merely. showing with a soles within a conference group. This obvi- display the last 26 lines of what would have ously has considerable potential. been on the Teletype output. 4 SERVICE-SYSTEM SOFTWARF Sh Calculating 4a The User’s Control Language 8h1 The same small innovation as mentioned 441 Consider the service a user gets from above for DDT enables us to use the CAL the computer to be in the form of discrete system from a display terminal. operations—i. e., the execution of individual $i Conferencing “service functions” from a repertoire com- prising a “service system.” Si1 We have set up a room specially equipped for on-line conferencing. Six displays are arranged in the center of a square table (see Figure 6) so that each of twenty participants 4ala Examples of service functions are deleting a word, replacing a character, hopping to a name, etc. has good visibility. One participant controls 402 Associated with each function of this the system, and all displays show the same repertoire is a “control-dialogue procedure.” view. The other participants have mice that This procedure involves selecting a service control a large arrow on the screen, for use function from the repertoire, setting up the as a pointer (with no control function). necessary parameter designations for a par- 8t2 As a quick means of finding and display- ticular ‘application, recovering from user ing (with appropriate forms of view) any errors, and calling for the execution of the desired material from a very large collection, . this system is a powerful aid to presentation 4a2a The procedure is made up of the and review conferences, sequence of keystrokes, select actions, etc. Research Center for Augmenting Human Intellect 403 made by the user, together with the inter- spersed feedback messages from the com- puter. 4a3 The repertoire of service functions, to- gether with their control-dialogue procedures, constitutes the user’s “control language.” This is a language for a “master-slave” dia- logue, enabling the user to control applica- tion of the computer’s capabilities to his own service. 408a It seems clear that significant aug- mentation of one’s intellectual effectiveness from the harnessing of computer services will require development of a broad and sophisticated control-language vocabulary. 4a3b It follows that the evolution of such a control language is a very important part of augmentation-system research. 4a4 For the designer of user systems, it is important to have good means for specifying the nature of the functions and their respec- tive control-dialogue procedures, so that a design specification will be 4a4a Concise, so that its essential features are easily seen 4a4b Unambiguous, so that questions about the design may be answered clearly 4a4e Canonical, so that information is easily located 4a4d Natural, so that the form of the de- scription fits the conceptual] frame of the design 4a4e Easy to compose, study, and modify, so that the process of evolutionary design can be facilitated, 445 It is also important for the user to have a description of the service functions and their control-dialogue procedures. 4a5a The description must again be con- cise, unambiguous, canonical, and natural; furthermore, it must be accurate, in that everything relevant to the user about the service functions and their control-dialogue procedures is described, and everything de- scribed actually works as indicated. = FIGURE 7—State-chart portrayal of part of the text-manipuls- tion control language 4b State-Chart Representation of Control-Lan- guage Design 4b1 Figure 7 shows a charting method that was used in earlier stages of our work for designing and specifying the control-pro- cedure portions of the control language. Even though limited to describing only the control-dialogue procedures, this representa- tion nonetheless served very well and led us to develop the successive techniques described below. 462 Figure 7 shows actual control procedures for four service functions from the repertoire of an interactive system: Delete Word, De- lete Text, Place Up Statement, and Forward Statement. 4b2a The boxes contain abbreviated de- scriptions of relevant display-feedback conditions, representing the intermediate states between successive user actions. Both to illustrate how the charting con- ventions are used and to give some feeling for the dynamics of our user-system con- trol procedures, we describe briefly below both the chart symbols and the associated display-feedback conventions that we have developed, 4b2a1 The writing at the top of each box indicates what is to be shown as “command feedback” at the top of the display (see Figures 3, 4 and 5). 4b2a1a An uparrow sometimes ap- Fall Joint Computer Conference, 1968 pears under the first character of one of the words of Command Feedback. 4b2a1a1 This indicates to the user that the next character he types will be interpreted as designating a new term to replace that being pointed to—no uparrow under Command Feedback signifies that keyboard ac- tion will not affect the command designation. 4b2a1b “Entity” represents the entity word (i.e., “character,” “word, “state- ment,” etc.) that was last used as part of a fully specified command. 4b2a1b1 The computer often “‘of- fers” the user an entity option. 4b2a2 The circle in the box indicates the character to be used for the “bug” (the tracking spot), which alternates between the characters uparrow and plus. 4b2a2a The uparrow indicates that a select action is appropriate, and the plus indicates that a select action is inappropriate. 4b2a3 The string of X’s. with under- lines, indicates that the selected char- acters are to be underlined as a means of showing the user what the computer thinks he has selected. 462b There is frequently an X on the out- put line from a box on the chart. This indi- cates that the computer is to wait until the user has made another action. 4b62b1 After this next action, the com- puter follows a branching path, depend- ing upon what the action was (as indi- cated on the chart) to reach another state-description box or one of the func- tion-execution processes, 4c The Control Metalanguage 4c1 In search for an improvement over the state chart, we looked for the following spe- cial features, as well as the general features listed above: 4cla A representational form using struc- tural text so as to harness the power of our on-line text-manipulation techniques for composing, studying, and modifying our designs. 4c1b A form that would allow us to specify the service functions as well as the control- dialogue procedures. 4c1c A form such that a design-descrip- tion file could be translated by a computer program into the actual implementation of the control language. 4c2 Using our Tree Meta compiler-compiler (described below), we have developed a next step forward in our means of designing, specifying, implementing and documenting our on-line control languages. The result is called “Control Metalanguage” (CML). 4e2a Figure 8 shows a portion of the de- scription for the current control language, written in Control Metalanguage. 4c2a1 This language is the means for describing both the service functions and their control-dialogue procedures. 4c2b The Contro] Metalanguage Trans- lator (CMLT) can process a file contain- ing such a description, to produce a corre- sponding version of an interactive system which responds to user actions exactly as described in the file. 4c8 There is a strong correspondence be- tween the conventions for representing the control procedures in Control Metalanguage and in the state chart, as a comparison of Figures 8 and 7 will reveal. 4c8a The particular example printed out for Figure 8 was chosen because it specifies some of the same procedures as in Figure 7. 4c3b For instance, the steps of display- feedback states, leading to execution of the “Delete Word” function, can readily be fol- lowed in the state chart. _4c3b1 The steps are produced by the user typing “D,” then “W,” then select- ing a character in a given word, and then hitting “command accept” (the CA key). 4ce8b2 The corresponding steps are out- lined below for the Control Metalanguage description of Figure 8, progressing from Statement 3, to Statement 3c, to Research Center for Augmenting Human Intellect 405 Statement 3c2, to Subroutine + BUG- 4c8c These same steps are indicated in SPEC, etc. Figure 8, starting from Statement 3: 4c3b3 The points or regions in Figure 7 4c3c1 “D” sets up the state described in corresponding to these statements and Statement 3C subroutines are marked by (8), (8C), (3C2), and (+ BUGISPEC), to help 4c8c2 “W” sets up the state described compare the two representations. in Statement 3C2 FIGURE 8—-Metalanguage description of part of control language 3 (we:) zap case 3A (b) [edit] dsp(backward tes*) . case 3B 3C (c) [edit] dsp(copy tes*) :s true => adjl: . case 3B1 (c) s*=cc dsp(tcopy character) e*=c,cha . = racter +bug2spec +cdlim(b1,p1,p2,p3,p4) *cdlim(b2,p5, p6,p7, p8) BaP +cpchtx(bi,p2,p4,pS,po) ; 3B2 (w) s*=cw dsp(tcopy word) e*=w,word +bug2spec +wdr2 (b1,p1,p2,p3,p4) +wdr2(b2,p5,p6,p7,p8) +cpwdvs (b1,p2,p4,p5,p6) ; 3B3 Q) s*=cl dsp(tcopy line) e*=1,line +bug2spec +1dlim(b1,p1,p2,p3,p4) +1dlim(b2,p5,p6,p7,p8) :c st bl+sf(b1) p2, tsi 7p p2>pl cr: then (cr) else (null) , pS p6, p4 se(bl): goto s 3B4 (v) s*=cv dsp(tcopy visible) e*=v,visible +bug2spec +vdr2(b1,p!,p2,p3,p4) +vdr2(b2,p5,p6,p7,p8) +cpwdvs(b1,p2,p4,p5,p6) ; 3b10 endcase +caqm ; (d) [edit] dsp(delete tes*) . case 3C1 (c) s*=xde dsp(tdelete character) e*=c,character +bug1 ( = spec +cdlim(b1,p1,p2,p3,p4) +del; , ad 3C2 (w) s*=dw dsp(tdelete word) e*=w,word +bug! glispec ¢+wdr (b1,p1,p2,p3,p4) del ; , . 3C3 (1) s*#dl dsp(tdelete line) e*#1,line +bugispec... 406 Fall Joint Computer Conference, 1968 4c8c3 The subroutine +BUGISPEC waits for the select-word (1) and CA (2) actions leading to the execution of the delete-word function. 4c38c3a Then the TWDR subroutine takes the bug-position parameter and sets pointers P1 through P4 to delimit the word in the text data. 4ce8¢36 Finally, the + DEL subroutine deletes what the pointers delimit, and then returns to the last-defined state (i.e., to where S* — DW). 4d Basic Organization of the On-Line System (NLS) 4d1 Figure 9 shows the relationships among the major components of NLS. 4d2 The Tree Meta Translator is a processor specially designed to produce new translators. 4d2a There is a special language—the Tree Meta Language—for use in describing the translator to be produced. 4d2b A special Tree Meta library of sub- routines must be used, along with the out- put of the Tree Meta Translator, to pro- duce a functioning new translator. The same library serves for every translator it produces. CMLT DISCAIPTION, IN TREE META LANGUAGE TREE META TREE META LIBRARY TRANSLATOR CRT WORK STATION c CONTROL-LANGUAGE | 0 CONTROL DISCRIPTION, IN META- a ——eooo —-474 CONTROL META- LANGUAGE LANGUAGE TRANSLATOR | .€ bl | CONTROL LIBRARY OF | PROCESSOR SUBROUTINES G F | LIBRARY SuB- ROUTINE DESCRIP- a MOL TION IN) MOL TRANSLATOR | cara OPERATING ON LINE SYSTEM (NLS) FIGURE 9—Basic organization of NLS showing use of compilers and compiler-compiler to implement it 4d3 For programming the various subrou- tines used in our 940 systems, we have de- veloped a special Machine-Oriented Language (MOL), together with an MOL Translator to convert MOL program descriptions into ma- chine code (see Ref. Hayl for a complete description). 4d3a The MOL is designed to facilitate system programming, by providing a high- level language for iterative, conditional, and arithmetic operations, etc., along with a block structure and conventions for label- ing that fit our structured-statement on- line manipulation aids. 4d8a1 These permit sophisticated com- puter aid where suitable, and also allow the programmer to switch to machine- level coding (with full access to varia- ' bles, labels, etc.) where core space, speed, timing, core-mapping arrangements, etc., are critical. 4d4 The NLS is organized as follows (letters refer to Figure 9): 4d4a The Control Processor (E) receives and processes successive user actions, and calls upon subroutines in the library (H) to provide it such services as the following: 4d4a1 Putting display feedback on the screen 4d4a2 Locating certain data in the file 4d¢4a3 Manipulating certain working data 4d4a4 Constructing a display view of specified data according to given view- ing parameters, etc. 4d4b The NLS library subroutines (H) are produced from MOL programs (F), as translated by the MOL Translator (G). 4d4e The Control Processor is produced from the control-language description (D), written in Control Metalanguage, as trans- lated by the CMLT (C). 4d4d The CMLT, in turn, is produced from a description (A) written in Tree Meta, as translated by the Tree Meta Translator (B). Research Center for Augmenting Human Intellect 407 =~ 2 4d5 Advantages of Metalanguage Approach to NLS Implementation 4d5a The metalanguage approach gives us improved means for control-language spec- ification, in terms of being unambiguous, concise, canonical, natural and easy to com- pose, study and modify. 4d5b Moreover, the Control Metalanguage specification promises to provide in itself a users’ documentation that is completely accurate, and also has the above desirable characteristics to facilitate study and refer- ence. 4a5¢ Modifying the control-dialogue pro- cedures for existing functions, or making a reasonable range of changes or additions to these functions, can often be accom- plished solely by additions or changes to the control-language record (in CML). 4d5c1 With our on-line studying, ma- nipulating and compiling techniques, system additions or changes at this level can be thought out and implemented (and automatically documented) very quickly. 4a5d New functions that require basic operations not available through existing subroutines in the NLS library will need to have new subroutines specified and pro- grammed (in MOL), and then will need new terms in CML to permit these new functions to be called upon. This latter requires a change in the record (A), and a new compilation of CMLT by means of the Tree Meta Translator. 4d5d1 On-line techniques for writing and modifying the MOL source code (F), for executing the compilations, and for debugging the routines, greatly reduce the effort involved in this process. SERVICE-SYSTEM HARDWARE (OTHER THAN SDS 940) 5a In addition to the SDS 940, the facility in- cludes peripheral equipment made by other manufacturers and equipment designed and constructed at SRI. 56 All of the non-SDS equipment is interfaced through the special devices channel which con- ne cts to the second memory buss through the SDS memory interface connection (MIC). 5c 561 This equipment, together with the RAD, is a significant load on the second memory buss, Not including the proposed “special operations” equipment, the maximum ex- pected data rate is approximately 264,000 words per second or one out of every 2.1 memory cycles. However, with the 940 varia- ble priority scheme for memory access (see Pirtle:), we expect less than 1 percent de- gradation in CPU efficiency due to this load. 562 This channel and the controllers (with the exception of the disc controller) were de- signed and constructed at SRI. 5b62a In the design of the hardware serv- ing the work stations, we have attempted to minimize the CPU burden by making the system as automatic as possible in its access to memory and by formatting the data in memory so as to minimize the executive time necessary to process it for the users. Figure 10 is a block diagram of the special- devices channel and associated equipment. The major components are as follows: 5c1 Executive Control 5e1a This is essentially a sophisticated multiplexer that allows independent, asyn- chronous access to core from any of the 6 controllers connected to it. Its functions are the following: 5cla1 Decoding instructions from the computer and passing them along as signals to the controllers. 5claz Accepting addresses and requests for memory access (input or output) from the controllers, determining rela- tive priority among the controllers, syn- chronizing to the computer clock, and passing the requests along to memory via the MIC. 5c1b The executive control includes a com- prehensive debugging panel that allows any of the 6 controllers to be operated off- line without interfering with the operation of other controllers. 408 Fall Joint Computer Conference, 1968 5c2 Disc File 5c2a This is a Model 4061 Bryant disc, selected for compatibility with the con- tinued 940-system development by Berke- ley’s Project GENIE, where extensive file- handling software was developed. 5c2b As formatted for our use, the disc will have a storage capacity of approxi- mately 32 million words, with a data-trans- fer rate of roughly 40,000 words per second and average access time of 85 milliseconds. 5e2ze The disc controller was designed by Bryant in close cooperation with SRI and Project GENIE. 5c3 Display System 5c3a The display systems consists of two identical subsystems, each with display con- troller, display generator, 6 CRT’s, and 6 closed-circuit television systems. 5c3b The display controllers process dis- play-command tables and display lists that are resident in core, and pass along dis- play-buffer contents to the display genera- tors. 5ce8e The display generators and CRT’s were developed by Tasker Industries to our specifications. Each has general character- vector plotting capability. They will accept display buffers consisting of instructions (beam motion, character writing, etc.) from the controller. Each will drive six 5-inch high-resolution CRT’s on which the display pictures are produced. 5c3c1 Character writing time is approxi- mately 8 microseconds, allowing an aver- FIGURE 10—Special devices channel DISC BRYANT Ty CONTROL DISC 5” : 17’ MONITOR CAMERA C.R.T, (875 LINES) Oo—_ Mouse C) C>— | Keyser — ! i i C>— KEYBOARD DISPLAY DISPLAY a | { 1 CONTROL GENERATOR JJ y i ! I 1 1 fr 9 i i § — ' ’ CAMERA i LL. | ( | CONTROL 1 i ! ! wd | ! ISTATIONS 1 ! — becom } t TO MEMORY — | 1 INTERFACE EXECUTIVE | | ' ' i CONNECTOR HARDWARE 1 I ' 1 DISPLAY DISPLAY p— | ! | ’ CONTROL GENERATOR ba ! ! \ ' 2 2 4 | ! ’ ‘ § 1 p- O =H woe INPUT | beet DEVICES -—L. KEYBOARD CONTROL L SPECIAL Ty OPERATIONS Low LINE mt PRIORITY PRINTER DEVICES >= * DURA/ PLOTTER ‘Research Center for Augmenting Human Intellect 409 age of 1000 characters on each of the six monitors when regenerating at 20 eps. 5c38d A high-resolution (875-line) closed- circuit television system transmits display pictures from each CRT to a television monitor at the corresponding work-station console, 5c8e This system was developed as a “best solution” to our experimental-laboratory needs, but it turned out to have properties which seem valuable for more widespread use: 5e8e1 Since only all-black or all-white signal levels are being treated, the scan- beam current on the cameras can be re- duced to achieve a short-term image- storage effect that yields flicker-free TV output even when the display refresh rate is as low as 15 eps. This allows a display generator to sustain about four times more displayed material than if the users were viewing direct-view re- freshed tubes. 5c8e2 The total cost of small CRT, TV camera, amplifier-controller, and monitor came to about $5500 per work station— where a random-deflection, display-qual- ity CRT of similar size would cost con- siderably more and would be harder to drive remotely. 5c3e3 Another cost feature which is very important in some system environ- ments favors this TV approach: The ex- pensive part is centrally located; each outlying monitor costs only about $600, so terminals can be set up even where usage will be low, with some video switching in the central establishment to take one terminal down and put another up. 5c8e4 An interesting feature of the video system is that with the flick of a switch the video signal can be inverted, so that the image picked up as bright lines on dim background may be viewed as black lines on a light background. There is a definite user preference for this inverted form of display. 5c8f In addition to the advantages noted above, the television display also invites the use of such commercially available de- vices as extra cameras, scan converters, video switches, and video mixers to enrich system service. 5c3f1 For example, the video image of a user’s computer-generated display could be mixed with the image from a camera focused on a collaborator at another ter- minal; the two users could communicate through both the computer and a voice intercom. Each user would then see the other’s face superimposed on the display of data under discussion. 5e8f2 Superimposed views from cameras focused on film images or drawings, or on the computer hardware, might also be useful. 5e3f3 We have experimented with these techniques (see Figure 11) and found them to be very effective. They promise to add a great deal to the value of re- mote display terminals. 5c4 Input-Device Controller 5c4a In addition to the television monitor, each work-station console has a keyboard, binary keyset, and mouse, 5c4b The controller reads the state of these FIGURE 11—Television display obtained by mixing the video signal from a remote camera with that from the computer- generated display 410 Fall Joint Computer Conference, 1968 devices at a preset interval (about 30 milli- seconds) and writes it into a fixed location table in core. 5c4b1 Bits are added to information from the keyboards, keysets and mouse switches to indicate when a new char- acter has been received or a switch has changed state since the last sample. If there is a new character or switch change, an interrupt is issued after the sample period. 5c4b2 The mouse coordinates are for- matted as a beam-positioning instruction to the display generator. Provisions are made in the display controller for in- cluding an entry in the mouse-position table as a display buffer. This allows the mouse position to be continuously displayed without any attention from the CPU. 5ce5 Special Operations 5c5a The box with this label in Figure 10 is at this time only a provision in the execu- tive control for the addition of a high-speed device. We have tentative plans for add- ing special hardware here to provide opera- tions not available in the 940 instruction set, such as character-string moves and string-pattern matching, 5c6 Low-Priority Devices 5c6a This controller accommodates three devices with relatively low data-transfer 6 6a 6b 6c 6d 6e 6f rates. At this time only the line printer is implemented, with provisions for adding an on-line typewriter (Dura), a plotter, and a terminal for the proposed ARPA computer network. 5c6al The line printer is a Potter Model HSP-3502 chain printer with 96 print- ing characters and a speed of about 230 lines per minute. REFERENCES (English 1) W K ENGLISH'D C ENGELBART B HUDDART Computer-aided display control Final Report Contract NAS 1-3988 SRI Project 5061 Stan- ford Research Institute Menlo Park California July 1965 (English2) W K ENGLISH D C ENGELBART M L BERMAN Display-selection techniques for text manipulation IEEE Trans on Human Factors in Electronics Vol HFE-& No 1 1967 (Engelbart1) D C ENGLEBART Augmenting human intellect: A conceptual framework Summary Report Contract AF 49 638 1024 SRI Project 3578 Stanford Research Institute Menlo Park California October 1962 (Engelbart2) DC ENGELBART A conceptual framework for the augmentation of man’s intellect In Vistas in Information Handling Vol 1 D W Howerton and D C Weeks eds Spartan Books Washington D C 1663 (Hayl) REHAY JF RULIFSON MOL940 Preliminary speifications for an ALGOL Ike machine-oriented language for the SDS 940 Interim Technical Report Contract NAS 1-5940 SRI Project 5890 Stanford Research Institute Menlo Park California March 1968 (Pirtlel) M PIRTLE Intercommunication of Processors and memory Proce Fall Joint Computer Conference Anaheim California November 1967