Web Posted on: February 1, 1999
MULTI-MODAL INFORMATION PRESENTATION FOR COMPUTER USERS WITH DYSLEXIA, READING DISABILITIES OR VISUAL IMPAIRMENT
Peter Ridge
Architecture & Design
pridge@mindmaker.com
Tim Musgrove
Natural Language Technologies
tmusgrove@mindmaker.com
Jeff Savage
Intelligent Personal Assistants
jsavage@mindmaker.com
Mindmaker, Inc.
224 Airport Pkwy. #550
San Jose, CA 95110
(408) 467-9200
http://www.mindmaker.com
Overview
Today, the Internet has become a part of our personal and professional life. Accessing the wealth of text information on the World Wide Web is a must, but not everyone is able to realize the full potential of computer systems or the Web because reading or comprehension dysfunctions make using documents on a computer difficult. Moreover, for those who suffer from visual impairments like acute myopia, simply seeing the information can present a significant challenge. In order to increase the viewing enjoyment and understanding of text-based information for these users, a multi-modal software system that can present text in an audible and easily readable form with simultaneous associated graphics and sound is needed.
TextAssist
Millions of users already know the former version of TextAssist, developed by Creative. Mindmaker developed a new version of TextAssist that is bundled with Creative audio boards and is also available for use with all Windows-compatible sound devices. It is based on Lernout & Hauspie Speech Product's TruVoice text-to-speech synthesis technology and can read aloud text from any Windows application. By means of familiar CD player-like controls, the user can "play" text from applications such as word processors, e-mail programs and web browsers.
TextAssist is fully customizable, offering several different voices that can be modified in pitch, volume, and speed. Individual voice characteristics can also be tailored to the user's taste and associated with specific applications. For example, a voice with a bright and peppy feel can be associated with a word processing application while a serious and conservative voice could speak when reading spreadsheets.
TextAssist DX
Mindmaker not only improved the basic functionality of TextAssist since its conception as a text reader, but has developed a multi-modal version, TextAssist DX, as a specialized tool for the assistive market.
TextAssist DX toolbar
TextAssist DX window
TextAssist DX extends the base product with the following features:
- Enlarged view of the text being read
- Synchronized highlighting of the word being read
- Display of images associated with the words being read
- Playing of sound effects associated with the words being read
Enlarged, synchronized text highlighting
The highlighting feature displays a three window dialog to the user while text is being read. The upper window displays the text being read in an enlarged font to increase legibility. The size and color of the font and window background are configurable to the needs of the user. As the text is read, each word is synchronously highlighted to indicate the current word being read within the body of the text. This feature makes it much easier to follow the text being read.
In addition to the enlarged font, another window displays a smaller font view of the text. This "zoomed out" view gives the user a sense of where they are within the body of the document.
Associated images and sound
In a third window, images and animation are displayed as the text is being read. These images are triggered by the content of the document. For example, a sentence that describes an eagle triggers an image of an eagle as a graphic reinforcement of the word eagle. In addition to the image, sounds are also triggered as further enhancement to the content being read. In this example, the screeching of an eagle can be heard in the background.
Additional applications of the technology
In addition to aiding users with reading, comprehension or visual disabilities, TextAssist DX is also applicable to other areas. Children who are learning to read are benefited by the synchronized highlighting so that they know what words sound and look like. Also, the meaning of words that are unfamiliar to them are easily inferred by the associated graphics and audio effects. The same is true for adults who are learning a second language. Although TextAssist currently speaks English, Mindmaker will be adding support for other languages to bring the technology to more users.
The multimedia presentation capability of TextAssist DX takes the "books on tape" concept to the next level. Not only does the user hear the story, but animation, background sounds and sound effects add to the environment. With the addition of three dimensional sound generation technologies, the user can be fully immersed into the story.
Conclusion
Computer users with dyslexia, reading disabilities or visual impairment can get more out of text-based information by presenting it via multiple media channels. TextAssist DX displays text with a magnified font size, sequentially highlights the text as it reads it out loud, displays images that correspond to the context of the highlighted text, and generates pertinent sound effects. This simultaneous reinforcement effectively enhances the user's comprehension and enjoyment of the information.