ADDING USER PROGRAMMABILITY TO A DEDICATED AUGMENTATIVE COMMUNICATION DEVICE

Clifford A. Kushler Prentke Romich Co., Wooster, Ohio

ABSTRACT

An augmentative communication (AAC) device has been enhanced with sophisticated macro capabilities. Previously, the system was capable of creating macros which could execute any combination of operations normally available to the system operator. This included a macro recorder which enabled the system operator to create a macro simply by performing the desired sequence of operations using the system's standard interface. A number of new features have been added to the system which significantly increase its power and programmability. These include: control structures WHILE/ENDWHILE and IF/ELSE/ENDIF; user-definable variables; complex expressions including arithmetic, logical, and string manipulation operators; and a variety of hooks into the operating system (e.g. switch and keyboard events). A number of "application macros" have been developed to test and demonstrate the power of the system.

BACKGROUND

Devices which have been specifically designed to serve as AAC systems generally have a number of characteristics such as improved durability, extended battery life, and control interfaces which have been optimized for individuals with severe motor impairments. Operating systems for such devices include a variety of functions which have been designed to be potentially useful for such individuals, especially for face-to-face communication. While such dedicated systems are not generally able to run standard computer applications, keyboard- and mouse-emulating interfaces can be used to enable such systems to be used for computer access. This allows the same language interface used for communication to be used for text generation on a computer. Many tasks performed by system operators involve the repetition of simple functions, or repeated sequences of functions that vary in a fixed or predictable manner. Such tasks are obviously good candidates for software implementation. However, this is generally not an option since most dedicated systems support the selection of various features and options, but lack the ability to program truly novel modes of operation. The needs of system operators vary so greatly that it is simply not feasible to include in an operating system every possible combination of functions, features, and/or modes of operation that might prove useful to some individual. An alternative is to provide a macro language which is flexible and powerful enough to enable the end user or other support personnel to implement solutions to support such tasks. A major revision of the operating system has been developed for a device which uses semantic compaction as the primary vocabulary organization technique. The previous revision included extensive macro capabilities, including a macro recorder which enables the creation of a macro by performing the desired sequence of operations using the system's standard interface. However, the ability of the macro language to respond to operator input was limited to allowing selections from system menus, selecting icons in certain situations, and entering text strings requested by the operating system. One of the primary goals of the new revision was to enhance the macro capabilities to provide the functionality described above. Once this was achieved, it was found that some of the other features to be included in the revision could be implemented using macros, and did not require further modification to the operating system software.

STATEMENT OF THE PROBLEM

The device in question has been used in field for over four years. The developers have been in close contact with a wide range of system operators during that time, and have responded to a large number of requests for operational modifications to the system. On many occasions, such requests have been able to be satisfied immediately by providing a macro to perform the desired function. However, these requests have often required revisions to the operating system (which is stored in flash EEPROM in the device, and can be upgraded in the field). Such revisions are only released periodically, so such requests may actually be satisfied only after a lengthy delay.

RATIONALE

In order to satisfy such user requests in a more timely fashion, it is necessary to support true programmability at the macro level in the device. This required the implementation of a number of new functions and features, many of which were well known at the outset of defining the enhancements. The first requirement was to introduce basic program control structures. To keep the programming language as simple as possible, only WHILE/ENDWHILE and IF/ELSE/ENDIF were used since all of the needed programming constructs could be created from these. User-definable variables were another necessary addition, along with the ability to process complex expressions including arithmetic, logical, and string manipulation operators. A variety of hooks into the operating system were also needed such as switch and keyboard events, and the ability to test and manipulate various system resources such as text and icon buffers. Other needed functions included the ability to delay for fixed intervals and to wait for specific events. The need for additional specialized functions became apparent only after some experience of trying to program specific macro functions using the first version of the revised system. Some of these examples will be described in this paper.

DESIGN / DEVELOPMENT

Previous versions of the operating system already supported the creation of "named macros" which can be invoked from other macros using the function <MACRO(m_name)>. A single named macro is limited to a maximum length of 246 bytes. This results in a "natural modularity" since more complex procedures must be broken down into functional components. In any case, since the system's display is limited to an 8-line LCD screen, this approach is more manageable at the programming level than large monolithic blocks of macro code. It also tends to be more efficient since many modularized macro functions may be re-used in other complex macros. The two programming structures available in the macro language are IF/ELSE/ENDIF and WHILE/ ENDWHILE. Since the macro "code" is interpreted rather than compiled, one constraint of the system is that the matching parts of a single control structure must occur within the same named macro. Since the body of a structure can consist of a call to another macro, this does not limit the capabilities of the system. Furthermore, attempting to construct macros that violated this requirement would be both difficult and highly error-prone. Other functions that affect the flow of execution include <MACRO-EXIT>, which immediately returns to the macro which invoked the current macro, and <MACRO-RETURN>, which does the same but first calculates and returns the value of the following expression, which can be of any type (integer, floating point, character or string). The type of the return value is automatically recognized by the expression evaluator, and converted appropriately when necessary. <MACRO-QUIT> exits all the way up to and out of the top level macro, so that the system waits for the next user input. <PAUSE> evaluates the following expression as an integer and unconditionally waits for that many 1/64's of a second. <WAIT-EVENT> must be followed by an event specifier, and optionally by a time-out limit (in 1/64's of a second). The <ANY-EVENT(...)> (or <ALL-EVENTS(...)>) function specifies which combination of switch and/or keyboard events is of interest and waits until any one (or the entire set) of the events specified occurs (or until the time- out limit expires). Finally, it is possible to specify a "start-up" macro that, if it exists, is automatically invoked every time the device is powered up. Simple temporary variables are set up for convenience to initialize and use "locally" within a single macro "program." User-defined, named variables can be created as needed to store values which serve specific functions. They may be set up as arrays, and can be initialized to more complex sets of data (such as the number of days in each month, or an array of macros). Variables can be set to expression values, or the user can be prompted to input a value. A variety of random number generation functions are available for implementing games, therapy drills, and so forth. One set of specialized functions was created to test and manipulate various system buffers and interfaces. The characters in the neighborhood of the text cursor can be tested and manipulated, as can the icons in the icon buffer and the data structure controlling which LEDs in the keyboard matrix are lit ("icon prediction"). One example macro using such functions simply spells out loud the word under or preceding the text cursor (e.g. for spelling an unfamiliar name over the telephone). For an early emerging literacy application, the speech output could be "A as in apple, T as in...", and so forth. Another macro pluralizes the preceding word, generating the correct spelling according to the morphology of the word. One system operator who utilizes auditory scanning for his selection technique requested that the system be set up to read the complete contents of the icon buffer as each new icon is added that does not complete an icon sequence. To accomplish this with a macro required that the macro be continuously executing "in the background," which meant that the macro needed to be able to fetch keystrokes, have the system perform them, and then test the result to see if the icon buffer was to be spoken. The <KEY- FETCH> and <KEY-PERFORM> functions were created to fill this need, and have been found to be useful in a variety of similar applications in which the macro executes in the background. It is also possible to generate informative messages that appear as if they were generated by the operating system. Fraser Shein has noted that, for an individual who uses scanning as their input technique, it is much more efficient to directly scan and manipulate the text itself, rather than to scan a separate keyboard matrix to select functions which move the cursor through the text [1]. A macro was developed to implement this approach, making frequent calls to a macro which simply waited until the switch was depressed, then returned 0 or 1 depending on whether the switch was depressed for a time exceeding a certain threshold (set in a named variable). By responding to audible prompts from the macro, the system operator can choose to move forward or backward, then choose how many paragraphs, sentences, words, and finally letters to move the cursor. A more complex example implements a completely new selection technique which enables the operator to use a joystick to navigate through a hierarchical system of categories based on auditory prompts. Moving the joystick to one of the eight compass points generates the auditory prompt for the category or selection item at that node. If the joystick is held in that position past a pre-set time threshold, that category or item is selected. If nothing has been programmed at a given node, the prompt "empty" is automatically generated, and the system remains in the current category. Two other features have been added to enhance the utility of the macro system. One is user-definable alarms which can be set up to trigger any macro. They can be set up as one-time only alarms, or to repeat at any desired frequency. They can be programmed to trigger only when the device is turned on at the time of the alarm, or to turn the device on, or to wait until the device is manually turned on. The other feature is the ability to create "menu macros." The operating system interface commonly uses menus on the 8-line LCD as the interface for configuring system options. Menu macros allow the programmer to create a similar "look and feel," where each menu selection triggers a macro. Thus, a menu system might be created in which each selection sends a command out the serial port to an environmental control device, organized as a menu of devices with sub- menus of relevant commands. As a final example, a background data collection macro has been created at the request of a number of different researchers. The macro is designed to time-stamp and record every event of interest (switch closures, key selections, text generated, speech output, etc.) in order to collect data for research purposes. The data is recorded in a hidden file in the background which can be downloaded to a computer for later analysis. To enable this macro to perform this function without distracting the system operator, a number of very specialized functions had to be added. These included functions to: save and restore the state of the system (before and after recording the data for each event); automatically record and subsequently duplicate any text and/or speech generated by a key selection, and to temporarily disable the system from modifying or refreshing any visible display (to keep the display and LEDs from flickering during the data recording process).

EVALUATION

Formal beta-testing of the system began in July, 1995, and has been in daily use by about 20 system operators ranging from young children to adults. It has also been demonstrated to a large number of professionals, including several speech language pathologists specializing in AAC. Much useful feedback has been received resulting in the improvements and refinements described above. General release of the operating system is scheduled for the spring of 1996.

DISCUSSION

The purpose of the enhanced macro capabilities of the system is to provide a simple, rapid and generally available method for developing customized functions for a widely used, commercially available AAC device. A number of complex and arguably useful "application macros" have been developed and tested. The system has been enthusiastically received by the individuals involved in beta-testing. It is anticipated that a disk of "shareware" macros will be developed over time to facilitate the widespread distribution of macros found to be particularly useful.

REFERENCES

1) Shein, F., Haataja, S., Brownlow, N. D., Treviranus, J., Milner, M., and Parnes, P. (1990) Direct Manipulation of Text by Scanning. Proceedings of the 13th Annual RESNA Conference, pp. 147-148. Washington, DC: RESNA.

Clifford A. Kushler, Ph.D. Director of Research, Prentke Romich Co. 1022 Heyl Rd., Wooster, OH 44691 Tel: (216)262-1984 ext. 220 Fax: (216)263-4829 E-mail: cak@prentrom.com