ScripTalker - An AAC System Incorporating Scripts
Richard Dye
Norman Alm
John L. Arnott
Iain R. Murray
Gillian Harper
Applied Computing
University of Dundee
Dundee, DD1 4HN, Scotland
tel: +44 1382 344150
fax: +44 1382 345509
email: rdye@mic.dundee.ac.uk
1. Summary
Transactional conversations, that is conversations which are carried out in order to accomplish a particular task, are a significant obstacle for disabled non-speaking people when they wish to become more independent. Incorporating scripted conversations into AAC (Augmentative and Alternative Communication) technology can help to overcome this obstacle. An AAC system is described that uses scripted conversations to help non-speaking people accomplish transactional communication more effectively.
| Top |
2. Introduction
The need to perform simple transactional interactions, that is conversations which are carried out in order to accomplish a particular task, is a significant obstacle to many disabled non-speaking people when they wish to become more independent. Disabled non-speaking people can be assisted by AAC (Augmentative and Alternative Communication) technology to communicate, but the resulting conversations can still be slow and problematic. This is especially the case when they are interacting with people who are unfamiliar with their method of communicating. A need therefore exists for new AAC techniques which will help non-speaking people take a more effective part in accomplishing everyday tasks through communication.
One way of speeding up the rate of interaction is to provide AAC users with a large number of complete phrases, and a mechanism by which these phrases can be accessed quickly and efficiently. Many existing AAC devices use techniques for accessing a large number of phrases which rely on users memorising large numbers of symbol sequences or remembering paths through a hierarchical storage structure. This places a significant cognitive load on users particularly when they are learning to use the device.
| Top |
3. Scripts in AAC
This paper describes a method of organising large numbers of pre-stored utterances in a way which is designed to overcome the memory load problem. It is proposed that a user's memory load can be minimised by exploiting the user's existing long term memory to help them locate and select appropriate utterances within an AAC system. Schank and Abelson [9] proposed a theory that people remember frequently encountered situations in structures in long term memory which they called "Scripts". A script captures the essence of a stereotypical situation, and allows people to make sense of what is happening, and to anticipate what will happen next. The AAC potential of using the concept of scripts to organise large numbers of complete phrases has been recognised for some time, and preliminary experimentation has encouraged and justified further research on their use [2, 10]. Scripts are most useful in transactional type conversations, as these tend to be well structured, and predictable in flow and content.
| Top |
4. ScripTalker
An AAC system, called ScripTalker, has been developed that incorporates the concept of scripts. ScripTalker was developed by a consortium of European research institutes and commercial partners under the ALADIN project (TIDE Project 1035).
In order to build a practical device that is useful outwith a limited number of scripted situations, the script concept was incorporated into a more general model of conversation [1, 2]. This conversation model allows users quick access to many frequently used conversation remarks, and also allows the users to create their own unique utterance when required.
As many AAC users have literacy problems it was decided to make the user interface of ScripTalker as graphical as possible. A pictorial scene-based user interface was devised to provide access to a set of scripted conversations. Fig 1. shows the interface displaying a scene within a script. Research into picture recognition and memory structures has demonstrated that groups of objects organised into realistic scenes corresponding to stereotypical situations assist recognition and memory, compared to groups of arbitrarily placed objects [5, 6, 7]. The scene approach gives users a pictorial indication of subject matter, and allows for users with varying levels of literacy skills.
Fig 1. The interface showing a scene within a script
After consultation with a group of AAC users [3] it was decided to develop a set of seven scripts. These scripts were chosen because they represented situations that users thought important, but found great difficulty communicating. The seven scripts were: "Activities of daily living", "Restaurant", "Doctor", "Shopping", "Meeting someone new", "Talking about emotions" and "Telephone". The scripts were developed by considering all the conversation tasks or goals a user would wish to achieve in any given situation. These conversation tasks were further broken down until each sub-task could be performed with a single phrase. A suitable phrase was then composed for each of these conversation sub-tasks. These phrases were grouped into sequences and scenes, and then mapped onto suitable props. The complexity of each scene was carefully controlled to make the scenes easy to use, and easy to learn to use. Wherever possible the number of props within a scene was restricted to nine, as this is approximately the limit of human short-term memory [8].
ScripTalker was developed as a software product which would run on a portable PC running MS Windows. It was designed so that it could be easily localised for different countries and languages: German, Dutch and English language versions are currently available. ScripTalker is now a commercial product and is being marketed in The Netherlands and Germany, and will soon be available in the UK.
| Top |
5. Future work
At present, there are only a limited number of scripts that are available within ScripTalker and this may be a problem in getting the product widely accepted. A simple authoring system for entering new scripts is available, but it is not easy to use for a non-expert. Further work is underway in the REACT project (Telematics project: DE 4207) to allow ScripTalker to be easily extended to cope with the wide-ranging and expanding requirements of AAC users. In the REACT project software will be developed for speech and language therapists to enable them to easily create new scripts, and modify and share existing scripts.
ScripTalker will also be developed further to automatically adapt to a user's pattern of communication, and better predict appropriate phrases. This will mean that users should be able to speak more quickly, and in the case of switch users, the amount of physical effort required will be reduced.
| Top |
6. Conclusions
The incorporation of scripts into AAC devices has great potential for helping non-speaking disabled people take an effective part in transactional conversations. The success of this type of AAC device will depend largely on the number and quality of the scripts that are available. In order to solve the problem of only having a limited number of scripts a new piece of software is being developed which will allow speech and language therapists to easily create new scripts and to collaborate with other therapists through the sharing of scripts.
| Top |
7. Acknowledgements
The development of ScripTalker was supported by the TIDE Programme of the Commission of the European Union under the ALADIN project (TP 1035). The project partners were The University of Dundee (UK), Instituut vor Revalidatievraagstukken (The Netherlands), Kompagne BV(The Netherlands), and IGEL GmbH(Germany). Further work in this field is being conducted under the REACT project (Telematics project DE 4207), involving these four partners and Instituut voor Doven (The Netherlands).
| Top |
8. References
[1] Alm, N., Arnott, J.L., and Newell, A.F. (1992) Prediction and conversational momentum in an augmentative communication system, Communications of the ACM, Vol. 35, No. 5, pp. 46-57.
[2] Alm, N., Morrison, A. and Arnott, J.L. (1995) A communication system based on scripts, plans and goals for enabling non-speaking people to conduct telephone conversations, Proceedings of IEEE Conference on Systems, Man & Cybernetics, Vancouver, Canada, 22-25 October, pp. 2408-2412
[3] Alm, N., Dye, R. and Harper, G. (1995) ALADIN - Advanced Language Device for Interaction, Proceedings of ECART 3 Conference, Lisbon, Portugal, 10-13 October, pp. 150-151
[4] McLaughlin, M. and Cody, M. (1982) Awkward silences: Behavioural antecedents and consequences of the conversational lapse, Human Communication Research, Vol. 8, pp. 299-316.
[5] Mandler, J.M. and Parker, R.E. (1976) Memory for descriptive and spatial information in complex pictures, Journal of Experimental Psychology: Human Learning and Memory, 2, pp. 38-48.
[6] Mandler, J.M. (1979) Categorical and schematic organization in memory, In: Puff, C.R. (ed.) Memory Organisation and Structure, pp. 259-299, New York: Academic Press.
[7] Mandler, J.M. (1984) Stories, Scripts and Scenes: Aspects of Schema Theory, Hillsdale: Lawrence Erlbaum Associates.
[8] Miller, G.A. (1956) The magic number seven, plus or minus two: some limits on our capacity to process information, Psychological Review, 63, 81-97.
[9] Schank, R. and Abelson, R. (1977) Scripts, Plans, Goals, and Understanding, New Jersey: Lawrence Erlbaum.
[10] Vanderheyden, P.B., Demasco, P.W., McCoy, K.F. and Pennington, C.A. (1996) A preliminary study into schema-based access and organization of re-usable text in AAC, Proceedings of the RESNA '96 Conference, Salt Lake City, Utah, USA, 17th.-12th. June pp.59-61.
| Top | |TIDE 98 Papers |